[patch] AES XTS: supporting custom iv from openssl enc command

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
24 messages Options
12
Reply | Threaded
Open this post in threaded view
|

[patch] AES XTS: supporting custom iv from openssl enc command

Jitendra Lulla
Hi,

openssl enc command with -aes-xxx-xts doesnt work if an IV is specified
as below:
openssl enc -engine af_alg -aes-256-xts -in <plaintext_file> -out
<output_encrypted_file> -K
0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef -iv
00000000000000000000000000000000

I am proposing a minor enhancement in EVP_CipherInit_ex() to include
case EVP_CIPH_XTS_MODE which currently is not present.

Please consider the patch [attached as well as pasted below]
--- /root/jlulla/evp_enc.c    2014-07-04 04:23:48.000000000 -0700
+++ crypto/evp/evp_enc.c    2014-07-04 03:21:29.000000000 -0700
@@ -242,6 +242,10 @@ skip_to_init:
             if(iv)
                 memcpy(ctx->iv, iv, EVP_CIPHER_CTX_iv_length(ctx));
             break;
+            case EVP_CIPH_XTS_MODE:
+            if(iv)
+                memcpy(ctx->iv, iv, EVP_CIPHER_CTX_iv_length(ctx));
+            break;
 
             default:
             return 0;

~Jitendra Lulla

openssl_xts_patch (534 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

[openssl.org #3442] [patch] AES XTS: supporting custom iv from openssl enc command

Rich Salz via RT
Hi,

openssl enc command with -aes-xxx-xts doesnt work if an IV is specified
as below:
openssl enc -engine af_alg -aes-256-xts -in <plaintext_file> -out
<output_encrypted_file> -K
0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef -iv
00000000000000000000000000000000

I am proposing a minor enhancement in EVP_CipherInit_ex() to include
case EVP_CIPH_XTS_MODE which currently is not present.

Please consider the patch [attached as well as pasted below]
--- /root/jlulla/evp_enc.c    2014-07-04 04:23:48.000000000 -0700
+++ crypto/evp/evp_enc.c    2014-07-04 03:21:29.000000000 -0700
@@ -242,6 +242,10 @@ skip_to_init:
             if(iv)
                 memcpy(ctx->iv, iv, EVP_CIPHER_CTX_iv_length(ctx));
             break;
+            case EVP_CIPH_XTS_MODE:
+            if(iv)
+                memcpy(ctx->iv, iv, EVP_CIPHER_CTX_iv_length(ctx));
+            break;
 
             default:
             return 0;

~Jitendra Lulla

openssl_xts_patch (534 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: [openssl.org #3442] [patch] AES XTS: supporting custom iv from openssl enc command

Jitendra Lulla


[with pull request now]

Hi,


openssl enc command with -aes-xxx-xts doesnt work if an IV is specified
as below:
openssl enc -engine af_alg -aes-256-xts -in <plaintext_file> -out
<output_encrypted_file> -K
0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef -iv
00000000000000000000000000000000

I am proposing a minor enhancement in EVP_CipherInit_ex() to include
case EVP_CIPH_XTS_MODE which currently is not present.

pull request: https://github.com/openssl/openssl/pull/150

Regards,
Jitendra Lulla




----- Original Message -----
From: Jitendra Lulla via RT <[hidden email]>
To:
Cc: [hidden email]
Sent: Wednesday, July 9, 2014 7:54 PM
Subject: [openssl.org #3442] [patch] AES XTS: supporting custom iv from openssl enc command




Hi,

openssl enc command with -aes-xxx-xts doesnt work if an IV is specified
as below:
openssl enc -engine af_alg -aes-256-xts -in <plaintext_file> -out
<output_encrypted_file> -K
0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef -iv
00000000000000000000000000000000

I am proposing a minor enhancement in EVP_CipherInit_ex() to include
case EVP_CIPH_XTS_MODE which currently is not present.

Please consider the patch [attached as well as pasted below]
--- /root/jlulla/evp_enc.c    2014-07-04 04:23:48.000000000 -0700
+++ crypto/evp/evp_enc.c    2014-07-04 03:21:29.000000000 -0700
@@ -242,6 +242,10 @@ skip_to_init:
             if(iv)
                 memcpy(ctx->iv, iv, EVP_CIPHER_CTX_iv_length(ctx));
             break;
+            case EVP_CIPH_XTS_MODE:
+            if(iv)
+                memcpy(ctx->iv, iv, EVP_CIPHER_CTX_iv_length(ctx));
+            break;
 
             default:
             return 0;

~Jitendra Lulla

openssl_xts_patch (534 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

[openssl.org #3442] [patch] AES XTS: supporting custom iv from openssl enc command

Rich Salz via RT
In reply to this post by Jitendra Lulla
On Wed Jul 09 16:24:04 2014, [hidden email] wrote:
> Hi,
>
> openssl enc command with -aes-xxx-xts doesnt work if an IV is specified

When you say it "doesn't work", what do you mean? Do you get an error? If so
what is it?



> as below:
> openssl enc -engine af_alg -aes-256-xts -in <plaintext_file> -out
> <output_encrypted_file> -K
> 0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef -iv
> 00000000000000000000000000000000

I notice you have installed a custom engine. Does it advertise XTS support?
What happens if you do not use the engine?

Running this command (without the engine parameter) works for me. Which version
of openssl are you running?

Note: although I don't think it explains your problem, the key you are using
here is too short. XTS is unusual in that it requires double length keys, hence
aes-256-xts requires a 512 bit key.

>
> I am proposing a minor enhancement in EVP_CipherInit_ex() to include
> case EVP_CIPH_XTS_MODE which currently is not present.
>
> Please consider the patch [attached as well as pasted below]
> --- /root/jlulla/evp_enc.c 2014-07-04 04:23:48.000000000 -0700
> +++ crypto/evp/evp_enc.c 2014-07-04 03:21:29.000000000 -0700
> @@ -242,6 +242,10 @@ skip_to_init:
> if(iv)
> memcpy(ctx->iv, iv, EVP_CIPHER_CTX_iv_length(ctx));
> break;
> + case EVP_CIPH_XTS_MODE:
> + if(iv)
> + memcpy(ctx->iv, iv, EVP_CIPHER_CTX_iv_length(ctx));
> + break;
>
> default:
> return 0;

This will not work. This section of code only runs if the flag
EVP_CIPH_CUSTOM_IV is not set - which it is for XTS.

Matt

______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
Development Mailing List                       [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: [openssl.org #3442] [patch] AES XTS: supporting custom iv from openssl enc command

Rich Salz via RT
>> openssl enc command with -aes-xxx-xts doesnt work if an IV is specified
>
> When you say it "doesn't work", what do you mean? Do you get an error? If so
> what is it?

If only it was the actual problem. The thing is that *if* one wants to
make enc work with XTS, it has to be treated specially, i.e. not as any
other cipher. See http://marc.info/?t=136844751600003&r=1&w=2 for
additional info. Another alternative to custom header mentioned in
referred thread can be to adhere to pre-defined fixed block size and
read 16 bytes ahead, so that when one hits end of file, and finds that
total_size%fixed_block_size<16, one can expand last block with
total_size%fixed_block_size. I mean last block would be variable size
from 16 up to fixed_block_size + 15 bytes, so that one doesn't have to
make up padding scheme.

>> as below:
>> openssl enc -engine af_alg -aes-256-xts -in <plaintext_file> -out
>> <output_encrypted_file> -K
>> 0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef -iv
>> 00000000000000000000000000000000
>
> I notice you have installed a custom engine. Does it advertise XTS support?
> What happens if you do not use the engine?

I'm not saying that it's the case here, but it should be noted that in
this case engine can impose own behaviour on enc. Most notably it can
trick enc to treating whole file as one single sector [which is not
necessarily cryptographically sound].

Bottom line [still] is that enc is not the place to perform XTS,
*unless* it's treated specially. In other words question should not be
about setting IV, but about *if* XTS should be supported by enc, and if
so, how exactly.


______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
Development Mailing List                       [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: [openssl.org #3442] [patch] AES XTS: supporting custom iv from openssl enc command

Ben Laurie-2
On 11 July 2014 08:30, Andy Polyakov via RT <[hidden email]> wrote:

>>> openssl enc command with -aes-xxx-xts doesnt work if an IV is specified
>>
>> When you say it "doesn't work", what do you mean? Do you get an error? If so
>> what is it?
>
> If only it was the actual problem. The thing is that *if* one wants to
> make enc work with XTS, it has to be treated specially, i.e. not as any
> other cipher. See http://marc.info/?t=136844751600003&r=1&w=2 for
> additional info. Another alternative to custom header mentioned in
> referred thread can be to adhere to pre-defined fixed block size and
> read 16 bytes ahead, so that when one hits end of file, and finds that
> total_size%fixed_block_size<16, one can expand last block with
> total_size%fixed_block_size. I mean last block would be variable size
> from 16 up to fixed_block_size + 15 bytes, so that one doesn't have to
> make up padding scheme.
>
>>> as below:
>>> openssl enc -engine af_alg -aes-256-xts -in <plaintext_file> -out
>>> <output_encrypted_file> -K
>>> 0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef -iv
>>> 00000000000000000000000000000000
>>
>> I notice you have installed a custom engine. Does it advertise XTS support?
>> What happens if you do not use the engine?
>
> I'm not saying that it's the case here, but it should be noted that in
> this case engine can impose own behaviour on enc. Most notably it can
> trick enc to treating whole file as one single sector [which is not
> necessarily cryptographically sound].
>
> Bottom line [still] is that enc is not the place to perform XTS,
> *unless* it's treated specially. In other words question should not be
> about setting IV, but about *if* XTS should be supported by enc, and if
> so, how exactly.

It seems to me this is why jamming modes like XTS into standard EVP as
if they were like other modes is a less than great idea.
______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
Development Mailing List                       [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: [openssl.org #3442] [patch] AES XTS: supporting custom iv from openssl enc command

Andy Polyakov-2
>> Bottom line [still] is that enc is not the place to perform XTS,
>> *unless* it's treated specially. In other words question should not be
>> about setting IV, but about *if* XTS should be supported by enc, and if
>> so, how exactly.
>
> It seems to me this is why jamming modes like XTS into standard EVP as
> if they were like other modes is a less than great idea.

But providing own interface for every specific mode is also hardly fun.
I mean there ought to be balance. Now we have EVP that implies different
semantic in different modes. In other words application might have to
perform extra calls depending on mode (and in this particular case
problem is that enc doesn't do those calls). What would be alternative?
Distinct interface for every class of modes? Can we define what makes up
such class? What do we expect from interface? Also note that either way,
the fact that it needs to be treated in enc in special way doesn't
change. It's not like I'm rejecting alternatives to EVP, but discussion
have to be more constructive.
______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
Development Mailing List                       [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re : Re: [openssl.org #3442] [patch] AES XTS: supporting custom iv from openssl enc command

nicolas.kox
Hi and sorry to interfere,

I had to review all ciphers available in openssl, and what seemed weird to me is that algorithms are always given as a combination of a symmetric cipher and a mode of operation
This approach was convenient since stream ciphers and block cipher with a mode of operation didn't need to be distinguished, and it worked fine as long as there were not so much chaining modes and none requiring more than an IV.

However, with much modes available now, and with these mode being quite independent from the block cipher used, I wonder if it wouldn't be nice to add something like EVP_MODE and EVP_MODE_CTX structures to manage modes independently. Practically, modes only need to know from a block cipher ecb encryption and decryption routines and block size. Thus treating block ciphers and modes separately could improve modularity (eg. Blowfish in GCM mode doesn't seems to exist, while both BF and GCM are implemented)

EVP_MODE would provide init, update and final routines, depending on encrypt/decrypt routines from block cipher, and EVP_MODE_CTX would store Ivs, AADs, buffers and any other mode specific data.
If I'm not mistaken, this is compliant with the API philosophy.


Obviously, it can be tricky to make it compatible with the current API but it does not appear to be impossible.
For example using a function that could create an EVP_CIPHER from a block cipher and a mode (basically by using the encrypt/decrypt routines from the block cipher).
Or given all block ciphers and modes, one can also declare all possible combinations, without too big efforts since it would mainly consist in setting some pointers.

With this approach, some functions EVP_MODE_CTX_set_xxx can also be added to manage mode-specific operations (setting IV length in XTS, or AAD in GCM) while maintaining backward compatibility.

I hope this is a more constructive proposition. However I think this could be nice to have such feature from an user point-of-view, and that it would also make the library easier to maintain.


Best regards
Nicolas


----- Mail d'origine -----
De: Andy Polyakov <[hidden email]>
À: [hidden email]
Cc: [hidden email]
Envoyé: Fri, 11 Jul 2014 12:56:50 +0200 (CEST)
Objet: Re: [openssl.org #3442] [patch] AES XTS: supporting custom iv from openssl enc command

>> Bottom line [still] is that enc is not the place to perform XTS,
>> *unless* it's treated specially. In other words question should not be
>> about setting IV, but about *if* XTS should be supported by enc, and if
>> so, how exactly.
>
> It seems to me this is why jamming modes like XTS into standard EVP as
> if they were like other modes is a less than great idea.

But providing own interface for every specific mode is also hardly fun.
I mean there ought to be balance. Now we have EVP that implies different
semantic in different modes. In other words application might have to
perform extra calls depending on mode (and in this particular case
problem is that enc doesn't do those calls). What would be alternative?
Distinct interface for every class of modes? Can we define what makes up
such class? What do we expect from interface? Also note that either way,
the fact that it needs to be treated in enc in special way doesn't
change. It's not like I'm rejecting alternatives to EVP, but discussion
have to be more constructive.
______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
Development Mailing List                       [hidden email]
Automated List Manager                           [hidden email]

______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
Development Mailing List                       [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re : Re: [openssl.org #3442] [patch] AES XTS: supporting custom iv from openssl enc command

Peter Waltenberg
Or extend EVP_CIPHER_CTX_ctrl() to handle things like changing IV's ?

Modes like XTS may gain a lot from that, you could use EVP_CIPHER_CTX_copy() to avoid repeated key expansion costs, change the IV with EVP_CIPHER_CTX_ctrl() and do the next block.

Peter

-----[hidden email] wrote: -----
To: [hidden email]
From: [hidden email]
Sent by: [hidden email]
Date: 07/11/2014 11:46PM
Subject: Re : Re: [openssl.org #3442] [patch] AES XTS: supporting custom iv from openssl enc command

Hi and sorry to interfere,

I had to review all ciphers available in openssl, and what seemed weird to me is that algorithms are always given as a combination of a symmetric cipher and a mode of operation
This approach was convenient since stream ciphers and block cipher with a mode of operation didn't need to be distinguished, and it worked fine as long as there were not so much chaining modes and none requiring more than an IV.

However, with much modes available now, and with these mode being quite independent from the block cipher used, I wonder if it wouldn't be nice to add something like EVP_MODE and EVP_MODE_CTX structures to manage modes independently. Practically, modes only need to know from a block cipher ecb encryption and decryption routines and block size. Thus treating block ciphers and modes separately could improve modularity (eg. Blowfish in GCM mode doesn't seems to exist, while both BF and GCM are implemented)

EVP_MODE would provide init, update and final routines, depending on encrypt/decrypt routines from block cipher, and EVP_MODE_CTX would store Ivs, AADs, buffers and any other mode specific data.
If I'm not mistaken, this is compliant with the API philosophy.


Obviously, it can be tricky to make it compatible with the current API but it does not appear to be impossible.
For example using a function that could create an EVP_CIPHER from a block cipher and a mode (basically by using the encrypt/decrypt routines from the block cipher).
Or given all block ciphers and modes, one can also declare all possible combinations, without too big efforts since it would mainly consist in setting some pointers.

With this approach, some functions EVP_MODE_CTX_set_xxx can also be added to manage mode-specific operations (setting IV length in XTS, or AAD in GCM) while maintaining backward compatibility.

I hope this is a more constructive proposition. However I think this could be nice to have such feature from an user point-of-view, and that it would also make the library easier to maintain.


Best regards
Nicolas


----- Mail d'origine -----
De: Andy Polyakov <[hidden email]>
À: [hidden email]
Cc: [hidden email]
Envoyé: Fri, 11 Jul 2014 12:56:50 +0200 (CEST)
Objet: Re: [openssl.org #3442] [patch] AES XTS: supporting custom iv from openssl enc command

>> Bottom line [still] is that enc is not the place to perform XTS,
>> *unless* it's treated specially. In other words question should not be
>> about setting IV, but about *if* XTS should be supported by enc, and if
>> so, how exactly.
>
> It seems to me this is why jamming modes like XTS into standard EVP as
> if they were like other modes is a less than great idea.

But providing own interface for every specific mode is also hardly fun.
I mean there ought to be balance. Now we have EVP that implies different
semantic in different modes. In other words application might have to
perform extra calls depending on mode (and in this particular case
problem is that enc doesn't do those calls). What would be alternative?
Distinct interface for every class of modes? Can we define what makes up
such class? What do we expect from interface? Also note that either way,
the fact that it needs to be treated in enc in special way doesn't
change. It's not like I'm rejecting alternatives to EVP, but discussion
have to be more constructive.
______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
Development Mailing List                       [hidden email]
Automated List Manager                           [hidden email]

______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
Development Mailing List                       [hidden email]
Automated List Manager                           [hidden email]

______________________________________________________________________ OpenSSL Project http://www.openssl.org Development Mailing List [hidden email] Automated List Manager [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: [openssl.org #3442] [patch] AES XTS: supporting custom iv from openssl enc command

Ben Laurie-2
In reply to this post by Andy Polyakov-2
On 11 July 2014 11:56, Andy Polyakov <[hidden email]> wrote:

>>> Bottom line [still] is that enc is not the place to perform XTS,
>>> *unless* it's treated specially. In other words question should not be
>>> about setting IV, but about *if* XTS should be supported by enc, and if
>>> so, how exactly.
>>
>> It seems to me this is why jamming modes like XTS into standard EVP as
>> if they were like other modes is a less than great idea.
>
> But providing own interface for every specific mode is also hardly fun.
> I mean there ought to be balance. Now we have EVP that implies different
> semantic in different modes. In other words application might have to
> perform extra calls depending on mode (and in this particular case
> problem is that enc doesn't do those calls). What would be alternative?
> Distinct interface for every class of modes?

Yes.

> Can we define what makes up
> such class? What do we expect from interface? Also note that either way,
> the fact that it needs to be treated in enc in special way doesn't
> change. It's not like I'm rejecting alternatives to EVP, but discussion
> have to be more constructive.

Right, agree that there has to be support for it, whatever happens.
The problem with the current situation is that because the same
interface is used (sometimes in strange ways) for incompatible modes,
s/w can _think_ it is capable of doing, e.g., XTS, when in fact it is
not.

It doesn't seem helpful to give that appearance by re-using the API
when it isn't actually true.

That said, there's certainly something to be said for having a common
API for the common parts.

Perhaps the thing to do is to make it so when you create an XTS_EVP,
that has the extra stuff needed to support XTS, but you can also
extract from it an EVP (or whatever the common interface is) that can
be used for the common subset.
______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
Development Mailing List                       [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: Re : Re: [openssl.org #3442] [patch] AES XTS: supporting custom iv from openssl enc command

Dr. Stephen Henson
In reply to this post by Peter Waltenberg
On Sat, Jul 12, 2014, Peter Waltenberg wrote:

> Or extend EVP_CIPHER_CTX_ctrl() to handle things like changing IV's ?  Modes
> like XTS may gain a lot from that, you could use EVP_CIPHER_CTX_copy() to
> avoid repeated key expansion costs, change the IV with EVP_CIPHER_CTX_ctrl()
> and do the next block.

There is already a method to change IVs without expnanding the key again which
should work for XTS (looking at code, not tried it explicitly). You set all
parameters to EVP_EcnryptInit_ex et al to NULL apart from the context and IV.
Subsequenty calls to EVP_EncryptUpdate etc should then use the new IV.

Steve.
--
Dr Stephen N. Henson. OpenSSL project core developer.
Commercial tech support now available see: http://www.openssl.org
______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
Development Mailing List                       [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: Re : Re: [openssl.org #3442] [patch] AES XTS: supporting custom iv from openssl enc command

Peter Waltenberg

Doh. 

Thanks
Pete


-----[hidden email] wrote: -----
To: [hidden email]
From: "Dr. Stephen Henson"
Sent by: [hidden email]
Date: 07/12/2014 10:16PM
Subject: Re: Re : Re: [openssl.org #3442] [patch] AES XTS: supporting custom iv from openssl enc command

On Sat, Jul 12, 2014, Peter Waltenberg wrote:

> Or extend EVP_CIPHER_CTX_ctrl() to handle things like changing IV's ?  Modes
> like XTS may gain a lot from that, you could use EVP_CIPHER_CTX_copy() to
> avoid repeated key expansion costs, change the IV with EVP_CIPHER_CTX_ctrl()
> and do the next block.

There is already a method to change IVs without expnanding the key again which
should work for XTS (looking at code, not tried it explicitly). You set all
parameters to EVP_EcnryptInit_ex et al to NULL apart from the context and IV.
Subsequenty calls to EVP_EncryptUpdate etc should then use the new IV.

Steve.
--
Dr Stephen N. Henson. OpenSSL project core developer.
Commercial tech support now available see: http://www.openssl.org
______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
Development Mailing List                       [hidden email]
Automated List Manager                           [hidden email]

______________________________________________________________________ OpenSSL Project http://www.openssl.org Development Mailing List [hidden email] Automated List Manager [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: [openssl.org #3442] [patch] AES XTS: supporting custom iv from openssl enc command

Jitendra Lulla
In reply to this post by Rich Salz via RT

Sorry for responding late.
I am using openssl-1.0.1h.
My af_alg engine does support xts.

Following are the findings:
1. The command works fine if I dont make any changes in the openssl.cnf file:
root@bodhi64vm:/home/jlulla/install/bin# ./openssl enc -aes-128-xts -in data_32 -out enc_data_32 -K 0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef -iv 00000000000000000000000000000000
root@bodhi64vm:/home/jlulla/install/bin# ./openssl enc -aes-128-xts -in enc_data_32 -out dec_data_32 -K 0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef -iv 00000000000000000000000000000000 -d
root@bodhi64vm:/home/jlulla/install/bin# md5sum *data_32
8fdbeaeafab909e9d9d81e23c06ef4d2  data_32
8fdbeaeafab909e9d9d81e23c06ef4d2  dec_data_32
3e38c0dba1f59c5901a7319524b97b45  enc_data_32

2. A. (without specifying engine in command) If I modify openssl.cnf by adding aes-128-xts in CIPHERS, the command gives me
"Error setting cipher AES-128-XTS"
root@bodhi64vm:/home/jlulla/install/bin# ./openssl enc -aes-128-xts -in data_32 -out enc_data_32 -K 0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef -iv 00000000000000000000000000000000
Error setting cipher AES-128-XTS
2.B. (with engine specified in command) If I modify openssl.cnf by adding aes-128-xts in CIPHERS, the command gives methe same error again:
root@bodhi64vm:/home/jlulla/install/bin# ./openssl enc -engine af_alg -aes-128-xts -in data_32 -out enc_data_32 -K 0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef -iv 00000000000000000000000000000000
engine "af_alg" set.
Error setting cipher AES-128-XTS

3. My af_alg engine does support xts and that only works if I make the changes in evp_enc.c
root@bodhi64vm:/home/jlulla/install/bin# ./openssl enc -engine af_alg -aes-128-xts -in data_32 -out enc_data_32_af_alg -K 0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef -iv 00000000000000000000000000000000
engine "af_alg" set.
root@bodhi64vm:/home/jlulla/install/bin# ./openssl enc -engine af_alg -aes-128-xts -in enc_data_32_af_alg -out dec_data_32_af_alg -K 0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef -iv 00000000000000000000000000000000 -d
engine "af_alg" set.
root@bodhi64vm:/home/jlulla/install/bin# md5sum data_32 dec_data_32 dec_data_32_af_alg
8fdbeaeafab909e9d9d81e23c06ef4d2  data_32
8fdbeaeafab909e9d9d81e23c06ef4d2  dec_data_32
8fdbeaeafab909e9d9d81e23c06ef4d2  dec_data_32_af_alg
root@bodhi64vm:/home/jlulla/install/bin#

My objective was to try linux kernel's crypto for xts.
I know that I could have written custom driver or custom user space applications (by using socket options and af_alg as the socket family) to try kernel's crypto without involving openssl.
But I wanted to use openssl and go to the linux kernel's crypto code for doing xts.

Based on the discussions on this thread so far, I now know that doing xts with openssl enc command is not advisable. I should try some other app with openssl.
Also the changes I am proposing may not be acceptable for similar reasons.


Another question I have is: (off topic though..)
why the af_alg patch submitted some time ago to openssl was not accepted?
http://www.mail-archive.com/openssl-dev%40openssl.org/msg29411.html

It seems that it has some performance issues but the linux crypto people still advocate that af_alg should be used for new crypto projects and af_alg engine for openssl should be extended:
Here’s a very recent set of slides [may 18 2014] which says that af_alg should be the choice for new projects [2nd last slide/page] for crypto accelerators in linux.
 

~Jitendra


From: Matt Caswell via RT <[hidden email]>
To: [hidden email]
Cc: [hidden email]
Sent: Friday, July 11, 2014 3:50 AM
Subject: [openssl.org #3442] [patch] AES XTS: supporting custom iv from openssl enc command

On Wed Jul 09 16:24:04 2014, [hidden email] wrote:
> Hi,
>
> openssl enc command with -aes-xxx-xts doesnt work if an IV is specified

When you say it "doesn't work", what do you mean? Do you get an error? If so
what is it?



> as below:
> openssl enc -engine af_alg -aes-256-xts -in <plaintext_file> -out
> <output_encrypted_file> -K
> 0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef -iv
> 00000000000000000000000000000000

I notice you have installed a custom engine. Does it advertise XTS support?
What happens if you do not use the engine?

Running this command (without the engine parameter) works for me. Which version
of openssl are you running?

Note: although I don't think it explains your problem, the key you are using
here is too short. XTS is unusual in that it requires double length keys, hence
aes-256-xts requires a 512 bit key.


>
> I am proposing a minor enhancement in EVP_CipherInit_ex() to include
> case EVP_CIPH_XTS_MODE which currently is not present.
>
> Please consider the patch [attached as well as pasted below]
> --- /root/jlulla/evp_enc.c 2014-07-04 04:23:48.000000000 -0700
> +++ crypto/evp/evp_enc.c 2014-07-04 03:21:29.000000000 -0700
> @@ -242,6 +242,10 @@ skip_to_init:
> if(iv)
> memcpy(ctx->iv, iv, EVP_CIPHER_CTX_iv_length(ctx));
> break;
> + case EVP_CIPH_XTS_MODE:
> + if(iv)
> + memcpy(ctx->iv, iv, EVP_CIPHER_CTX_iv_length(ctx));
> + break;
>
> default:
> return 0;

This will not work. This section of code only runs if the flag
EVP_CIPH_CUSTOM_IV is not set - which it is for XTS.

Matt

______________________________________________________________________
OpenSSL Project                                http://www.openssl.org
Development Mailing List                      [hidden email]
Automated List Manager                          [hidden email]



Reply | Threaded
Open this post in threaded view
|

Re: [openssl.org #3442] [patch] AES XTS: supporting custom iv from openssl enc command

Dr. Stephen Henson
On Sat, Jul 12, 2014, Jitendra Lulla wrote:

>
>
> Based on the discussions on this thread so far, I now know that doing
> xts with openssl enc command is not advisable. I should try some other
> app with openssl.

Not only not advisable it wont work properly. The enc command assumes that the
underlying cipher can stream whereas XTS is a one shot version. We should
really block use of XTS mode in enc completely.

> Also the changes I am proposing may not be acceptable for similar reasons.
>

It may be that your implementation of an AF_ALG EVP_CIPHER for XTS can be
changed so it works with unmodified OpenSSL. The OpenSSL XTS implementation
is a software implementation and some techniques it uses wont be appropriate.

Is the EVP_CIPHER code you're using for XTS available somewhere?

Steve.
--
Dr Stephen N. Henson. OpenSSL project core developer.
Commercial tech support now available see: http://www.openssl.org
______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
Development Mailing List                       [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: [openssl.org #3442] [patch] AES XTS: supporting custom iv from openssl enc command

Dr. Stephen Henson
On Sun, Jul 13, 2014, Dr. Stephen Henson wrote:

> On Sat, Jul 12, 2014, Jitendra Lulla wrote:
>
> > Also the changes I am proposing may not be acceptable for similar reasons.
> >
>
> It may be that your implementation of an AF_ALG EVP_CIPHER for XTS can be
> changed so it works with unmodified OpenSSL. The OpenSSL XTS implementation
> is a software implementation and some techniques it uses wont be appropriate.
>
> Is the EVP_CIPHER code you're using for XTS available somewhere?
>

If you look through the existing code for XTS mode in e_aes.c you'll see it
copies the IV manually. If your EVP_CIPHER implementation includes the
EVP_CIPH_ALWAYS_CALL_INIT flags you can do the same. If you handle that
appropriately you shouldn't need to modify OpenSSL at all.

Steve.
--
Dr Stephen N. Henson. OpenSSL project core developer.
Commercial tech support now available see: http://www.openssl.org
______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
Development Mailing List                       [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: [openssl.org #3442] [patch] AES XTS: supporting custom iv from openssl enc command

Andy Polyakov-2
>>> Also the changes I am proposing may not be acceptable for similar reasons.
>>>
>> It may be that your implementation of an AF_ALG EVP_CIPHER for XTS can be
>> changed so it works with unmodified OpenSSL. The OpenSSL XTS implementation
>> is a software implementation and some techniques it uses wont be appropriate.
>>
>> Is the EVP_CIPHER code you're using for XTS available somewhere?
>>
>
> If you look through the existing code for XTS mode in e_aes.c you'll see it
> copies the IV manually. If your EVP_CIPHER implementation includes the
> EVP_CIPH_ALWAYS_CALL_INIT flags you can do the same. If you handle that
> appropriately you shouldn't need to modify OpenSSL at all.

Let's not loose our heads. What is the objective? To produce arbitrary
result? I mean suggested patch doesn't actually solve real problem in
sense that modified code doesn't produce cryptographically sound output,
nor does it handle corner cases. Same applies to tweaking flags! If goal
is to encrypt files with XTS with enc utility, then it's *not* the way
to solve the problem. There are two possible options: 1. Forbid usage or
XTS with enc. 2. Decide how to handle it (multiple options, real
mode-specific, non-API ones, discussed earlier) and implement it in
enc(*). One can do both in the given order. If goal is to compare
performance, then speed -evp aes-128-xts with and without -engine
argument would suffice.

(*) if XTS layer is extended with notion of block size, it's possible to
minimize enc's involvement to just initially setting block size. In
which case XTS layer would have to take care of tweak chaining, IV
increment (basically counter, but in little-endian order), perhaps even
adaptive padding...

______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
Development Mailing List                       [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: [openssl.org #3442] [patch] AES XTS: supporting custom iv from openssl enc command

Dr. Stephen Henson
On Sun, Jul 13, 2014, Andy Polyakov wrote:

> >>> Also the changes I am proposing may not be acceptable for similar reasons.
> >>>
> >> It may be that your implementation of an AF_ALG EVP_CIPHER for XTS can be
> >> changed so it works with unmodified OpenSSL. The OpenSSL XTS implementation
> >> is a software implementation and some techniques it uses wont be appropriate.
> >>
> >> Is the EVP_CIPHER code you're using for XTS available somewhere?
> >>
> >
> > If you look through the existing code for XTS mode in e_aes.c you'll see it
> > copies the IV manually. If your EVP_CIPHER implementation includes the
> > EVP_CIPH_ALWAYS_CALL_INIT flags you can do the same. If you handle that
> > appropriately you shouldn't need to modify OpenSSL at all.
>
> Let's not loose our heads. What is the objective? To produce arbitrary
> result? I mean suggested patch doesn't actually solve real problem in
> sense that modified code doesn't produce cryptographically sound output,
> nor does it handle corner cases. Same applies to tweaking flags!

There are AFAICS two separate problems. One is that enc can't handle XTS mode
I don't think that can be fixed easily and I'd consider it acceptable to just
indicate that XTS mode isn't support by enc.

The separate problem is that the OP has written an ENGINE that supports XTS
mode and the requested patch was to make XTS mode work in the ENGINE. I'm
suggesting that the OPs ENGINE implementation of XTS mode in an EVP_CIPHER
has set the flags incorrectly (perhaps it's a generic EVP_CIPHER that handles
all cases identically). Using the correct flags in the ENGINE EVP_CIPHER and
not making any changes to OpenSSL should solve the second problem.

Steve.
--
Dr Stephen N. Henson. OpenSSL project core developer.
Commercial tech support now available see: http://www.openssl.org
______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
Development Mailing List                       [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: [openssl.org #3442] [patch] AES XTS: supporting custom iv from openssl enc command

Matt Caswell-2


On 13/07/14 22:28, Dr. Stephen Henson wrote:
> The separate problem is that the OP has written an ENGINE that supports XTS
> mode and the requested patch was to make XTS mode work in the ENGINE. I'm
> suggesting that the OPs ENGINE implementation of XTS mode in an EVP_CIPHER
> has set the flags incorrectly (perhaps it's a generic EVP_CIPHER that handles
> all cases identically). Using the correct flags in the ENGINE EVP_CIPHER and
> not making any changes to OpenSSL should solve the second problem.

It's not clear to me that the engine being used was actually written by
the OP...or whether he was just using it. See RT #2554.

Matt

______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
Development Mailing List                       [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: [openssl.org #3442] [patch] AES XTS: supporting custom iv from openssl enc command

Andy Polyakov-2
In reply to this post by Dr. Stephen Henson
Dr. Stephen Henson wrote:

> On Sun, Jul 13, 2014, Andy Polyakov wrote:
>
>>>>> Also the changes I am proposing may not be acceptable for similar reasons.
>>>>>
>>>> It may be that your implementation of an AF_ALG EVP_CIPHER for XTS can be
>>>> changed so it works with unmodified OpenSSL. The OpenSSL XTS implementation
>>>> is a software implementation and some techniques it uses wont be appropriate.
>>>>
>>>> Is the EVP_CIPHER code you're using for XTS available somewhere?
>>>>
>>> If you look through the existing code for XTS mode in e_aes.c you'll see it
>>> copies the IV manually. If your EVP_CIPHER implementation includes the
>>> EVP_CIPH_ALWAYS_CALL_INIT flags you can do the same. If you handle that
>>> appropriately you shouldn't need to modify OpenSSL at all.
>> Let's not loose our heads. What is the objective? To produce arbitrary
>> result? I mean suggested patch doesn't actually solve real problem in
>> sense that modified code doesn't produce cryptographically sound output,
>> nor does it handle corner cases. Same applies to tweaking flags!
>
> There are AFAICS two separate problems. One is that enc can't handle XTS mode
> I don't think that can be fixed easily and I'd consider it acceptable to just
> indicate that XTS mode isn't support by enc.
>
> The separate problem is that the OP has written an ENGINE that supports XTS
> mode and the requested patch was to make XTS mode work in the ENGINE. I'm
> suggesting that the OPs ENGINE implementation of XTS mode in an EVP_CIPHER
> has set the flags incorrectly (perhaps it's a generic EVP_CIPHER that handles
> all cases identically). Using the correct flags in the ENGINE EVP_CIPHER and
> not making any changes to OpenSSL should solve the second problem.

The fact that modified enc interoperates with engine in question means
that engine suffers from exactly same problems, i.e. produces unsound
result and doesn't handle corner cases. In other words it requires
similar adjustments, and adjustments need to be harmonized.
______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
Development Mailing List                       [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: [openssl.org #3442] [patch] AES XTS: supporting custom iv from openssl enc command

Dr. Stephen Henson
In reply to this post by Matt Caswell-2
On Sun, Jul 13, 2014, Matt Caswell wrote:

>
>
> On 13/07/14 22:28, Dr. Stephen Henson wrote:
> > The separate problem is that the OP has written an ENGINE that supports XTS
> > mode and the requested patch was to make XTS mode work in the ENGINE. I'm
> > suggesting that the OPs ENGINE implementation of XTS mode in an EVP_CIPHER
> > has set the flags incorrectly (perhaps it's a generic EVP_CIPHER that handles
> > all cases identically). Using the correct flags in the ENGINE EVP_CIPHER and
> > not making any changes to OpenSSL should solve the second problem.
>
> It's not clear to me that the engine being used was actually written by
> the OP...or whether he was just using it. See RT #2554.
>

That version (and other AF_ALG versions I've seen) does not support XTS mode
AFAICS. If you try and treat XTS mode like any other EVP_CIPHER in an ENGINE
it will fail, hence my comment about the flags.

Steve.
--
Dr Stephen N. Henson. OpenSSL project core developer.
Commercial tech support now available see: http://www.openssl.org
______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
Development Mailing List                       [hidden email]
Automated List Manager                           [hidden email]
12