EVP_DecryptUpdate: why is this failing when out == in?

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

EVP_DecryptUpdate: why is this failing when out == in?

Paul Smith
As I understand it, it's legal to provide the exact same input and
output buffer to EVP_EncryptUpdate and EVP_DecryptUpdate, but it's not
legal to provide pointers into different parts of the same buffer.
That's a good check.

However, my implementation is getting triggered by this code in
EVP_DecryptUpdate():

    if (ctx->final_used) {
        /* see comment about PTRDIFF_T comparison above */
=>      if (((PTRDIFF_T)out == (PTRDIFF_T)in)
            || is_partially_overlapping(out, in, b)) {
            EVPerr(EVP_F_EVP_DECRYPTUPDATE, EVP_R_PARTIALLY_OVERLAPPING);
            return 0;
        }

Can someone explain why, only in this specific situation where we're
decrypting the final block, we require that OUT and IN not be the same
buffer?  Everywhere else we check is_partially_overlapping() only,
without equality.

I read the comment about PTRDIFF_T but I didn't come up with a reason
for the equality check.  This check was added back in 2016 in SHA
5fc77684f1 FWIW.

--
openssl-users mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-users
Reply | Threaded
Open this post in threaded view
|

Re: EVP_DecryptUpdate: why is this failing when out == in?

Paul Smith
I filed https://github.com/openssl/openssl/issues/7941 about this FYI.

Cheers!


On Wed, 2018-12-19 at 01:56 -0500, Paul Smith wrote:

> As I understand it, it's legal to provide the exact same input and
> output buffer to EVP_EncryptUpdate and EVP_DecryptUpdate, but it's not
> legal to provide pointers into different parts of the same buffer.
> That's a good check.
>
> However, my implementation is getting triggered by this code in
> EVP_DecryptUpdate():
>
>     if (ctx->final_used) {
>         /* see comment about PTRDIFF_T comparison above */
> =>      if (((PTRDIFF_T)out == (PTRDIFF_T)in)
>             || is_partially_overlapping(out, in, b)) {
>             EVPerr(EVP_F_EVP_DECRYPTUPDATE, EVP_R_PARTIALLY_OVERLAPPING);
>             return 0;
>         }
>
> Can someone explain why, only in this specific situation where we're
> decrypting the final block, we require that OUT and IN not be the same
> buffer?  Everywhere else we check is_partially_overlapping() only,
> without equality.
>
> I read the comment about PTRDIFF_T but I didn't come up with a reason
> for the equality check.  This check was added back in 2016 in SHA
> 5fc77684f1 FWIW.

--
openssl-users mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-users