issue with EVP_EncryptUpdate in XTS mode?

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

issue with EVP_EncryptUpdate in XTS mode?

Andrew Tucker
I was doing some comparisons of XTS and GCM mode using the EVP APIs and found a discrepancy that seems to be an issue with XTS.

In GCM mode if the buffer is encrypted in one call to EVP_EncryptUpdate or with several calls with smaller buffers the resulting ciphertext is the same, as I would expect.   With XTS mode, calling EVP_EncryptUpdate results in the same ciphertext for the same plaintext and does not match the results when the buffer is encrypted with one call to EVP_EncryptUpdate.

I would expect that the counter is incremented in both XTS and GCM mode in the same way and that in both cases the output would match regardless of the encryption block size.

A simple repro test is attached.    If you run it you can see that the output "GCM in one block" matches the output for "GCM in 16 byte blocks" and the outputs do not match for XTS.

I am using OpenSSL v1.02p but I have tried with other versions and got the same results.

Am I misunderstanding the use of XTS mode or is this an issue with OpenSSL?

Thanks!


--
openssl-users mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-users

xtsgcmtest.c (4K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: issue with EVP_EncryptUpdate in XTS mode?

Matt Caswell-2


On 25/01/2019 20:16, Andrew Tucker wrote:

> I was doing some comparisons of XTS and GCM mode using the EVP APIs and found a
> discrepancy that seems to be an issue with XTS.
>
> In GCM mode if the buffer is encrypted in one call to EVP_EncryptUpdate or with
> several calls with smaller buffers the resulting ciphertext is the same, as I
> would expect.   With XTS mode, calling EVP_EncryptUpdate results in the same
> ciphertext for the same plaintext and does not match the results when the buffer
> is encrypted with one call to EVP_EncryptUpdate.
>
> I would expect that the counter is incremented in both XTS and GCM mode in the
> same way and that in both cases the output would match regardless of the
> encryption block size.
>
> A simple repro test is attached.    If you run it you can see that the output
> "GCM in one block" matches the output for "GCM in 16 byte blocks" and the
> outputs do not match for XTS.
>
> I am using OpenSSL v1.02p but I have tried with other versions and got the same
> results.
>
> Am I misunderstanding the use of XTS mode or is this an issue with OpenSSL?

The documentation for XTS mode seems to be particularly dreadful (i.e.
practically non-existent).

XTS mode was designed for disk encryption and works quite differently to modes
like GCM. XTS encrypts blocks at a time where each block has a separate "tweak"
value (and by "block" here I'm talking about disk blocks not encryption blocks).
There is no "streaming" concept - there is not necessarily any relationship with
the tweak from one block to be encrypted to the next one. AFAICT from code
inspection you are expected to set the tweak in the IV field and then encrypt an
entire (disk) block in one go with EVP_EncryptUpdate().

This seems to be supported by this post from Steve Henson where he describes the
XTS implementation as not being able to stream and as a "one shot" version:

http://openssl.6102.n7.nabble.com/patch-AES-XTS-supporting-custom-iv-from-openssl-enc-command-tt51949.html#a52009

In this later post in the same thread Steve talks about calling
EVP_EncryptInit_ex again with all the params set to NULL except the ctx and iv
(i.e. tweak) in order to set a new tweak and then call EVP_EncryptUpdate again.

http://openssl.6102.n7.nabble.com/patch-AES-XTS-supporting-custom-iv-from-openssl-enc-command-tp51949p52006.html

In other words I think you are supposed to use it something like this (error
handling omitted for brevity):

EVP_CIPHER_CTX *ctx = EVP_CIPHER_CTX_new();

EVP_EncryptInit_ex(ctx, EVP_aes_256_xts(), NULL, aes_key, NULL);

EVP_EncryptInit_ex(ctx, NULL, NULL, NULL, tweak1);
EVP_EncryptUpdate(ctx, out, &outl, block1, blocklen1);
EVP_EncryptInit_ex(ctx, NULL, NULL, NULL, tweak2);
EVP_EncryptUpdate(ctx, out, &outl, block2, blocklen2);
EVP_EncryptInit_ex(ctx, NULL, NULL, NULL, tweak3);
EVP_EncryptUpdate(ctx, out, &outl, block3, blocklen3);

That really needs to go in the man pages.

Matt





--
openssl-users mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-users
Reply | Threaded
Open this post in threaded view
|

Re: issue with EVP_EncryptUpdate in XTS mode?

Andrew Tucker
Thanks for the feedback Matt.

I definitely missed this difference between XTS and GCM and didnt realize one supports "streaming" and one doesnt.

For our application we only ever call EncryptUpdate once so XTS works well but its great to understand the limitations and make sure future changes dont screw something up.

On Fri, Jan 25, 2019 at 4:18 PM Matt Caswell <[hidden email]> wrote:


On 25/01/2019 20:16, Andrew Tucker wrote:
> I was doing some comparisons of XTS and GCM mode using the EVP APIs and found a
> discrepancy that seems to be an issue with XTS.
>
> In GCM mode if the buffer is encrypted in one call to EVP_EncryptUpdate or with
> several calls with smaller buffers the resulting ciphertext is the same, as I
> would expect.   With XTS mode, calling EVP_EncryptUpdate results in the same
> ciphertext for the same plaintext and does not match the results when the buffer
> is encrypted with one call to EVP_EncryptUpdate.
>
> I would expect that the counter is incremented in both XTS and GCM mode in the
> same way and that in both cases the output would match regardless of the
> encryption block size.
>
> A simple repro test is attached.    If you run it you can see that the output
> "GCM in one block" matches the output for "GCM in 16 byte blocks" and the
> outputs do not match for XTS.
>
> I am using OpenSSL v1.02p but I have tried with other versions and got the same
> results.
>
> Am I misunderstanding the use of XTS mode or is this an issue with OpenSSL?

The documentation for XTS mode seems to be particularly dreadful (i.e.
practically non-existent).

XTS mode was designed for disk encryption and works quite differently to modes
like GCM. XTS encrypts blocks at a time where each block has a separate "tweak"
value (and by "block" here I'm talking about disk blocks not encryption blocks).
There is no "streaming" concept - there is not necessarily any relationship with
the tweak from one block to be encrypted to the next one. AFAICT from code
inspection you are expected to set the tweak in the IV field and then encrypt an
entire (disk) block in one go with EVP_EncryptUpdate().

This seems to be supported by this post from Steve Henson where he describes the
XTS implementation as not being able to stream and as a "one shot" version:

http://openssl.6102.n7.nabble.com/patch-AES-XTS-supporting-custom-iv-from-openssl-enc-command-tt51949.html#a52009

In this later post in the same thread Steve talks about calling
EVP_EncryptInit_ex again with all the params set to NULL except the ctx and iv
(i.e. tweak) in order to set a new tweak and then call EVP_EncryptUpdate again.

http://openssl.6102.n7.nabble.com/patch-AES-XTS-supporting-custom-iv-from-openssl-enc-command-tp51949p52006.html

In other words I think you are supposed to use it something like this (error
handling omitted for brevity):

EVP_CIPHER_CTX *ctx = EVP_CIPHER_CTX_new();

EVP_EncryptInit_ex(ctx, EVP_aes_256_xts(), NULL, aes_key, NULL);

EVP_EncryptInit_ex(ctx, NULL, NULL, NULL, tweak1);
EVP_EncryptUpdate(ctx, out, &outl, block1, blocklen1);
EVP_EncryptInit_ex(ctx, NULL, NULL, NULL, tweak2);
EVP_EncryptUpdate(ctx, out, &outl, block2, blocklen2);
EVP_EncryptInit_ex(ctx, NULL, NULL, NULL, tweak3);
EVP_EncryptUpdate(ctx, out, &outl, block3, blocklen3);

That really needs to go in the man pages.

Matt





--
openssl-users mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-users

--
openssl-users mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-users
Reply | Threaded
Open this post in threaded view
|

Re: issue with EVP_EncryptUpdate in XTS mode?

Matt Caswell-2
In reply to this post by Andrew Tucker


On 25/01/2019 20:16, Andrew Tucker wrote:

> I was doing some comparisons of XTS and GCM mode using the EVP APIs and found a
> discrepancy that seems to be an issue with XTS.
>
> In GCM mode if the buffer is encrypted in one call to EVP_EncryptUpdate or with
> several calls with smaller buffers the resulting ciphertext is the same, as I
> would expect.   With XTS mode, calling EVP_EncryptUpdate results in the same
> ciphertext for the same plaintext and does not match the results when the buffer
> is encrypted with one call to EVP_EncryptUpdate.
>
> I would expect that the counter is incremented in both XTS and GCM mode in the
> same way and that in both cases the output would match regardless of the
> encryption block size.
>
> A simple repro test is attached.    If you run it you can see that the output
> "GCM in one block" matches the output for "GCM in 16 byte blocks" and the
> outputs do not match for XTS.
>
> I am using OpenSSL v1.02p but I have tried with other versions and got the same
> results.
>
> Am I misunderstanding the use of XTS mode or is this an issue with OpenSSL?

Please see my previous post on this topic here:

https://mta.openssl.org/pipermail/openssl-users/2019-January/009781.html

PRs welcome to improve the documentation in this area.

Matt