Question about SSL_key_update

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Question about SSL_key_update

Andreas Müller
Hi,

I "inherited" our project to support/use TLSv1.3 from a late  
colleague. We have a server written in C++ (Windows, Linux)
and clients (Windows, Linux, also written in C++ and also a Java  
client). With Java, we use the native SSLSocket implementation, in  
Windows we use Schannel and in Linux we use OpenSSL 1.1.1g. It seems  
to work and even interoperability
did not show issues on some initial testing.

I was curious about SSL_key_update. I read that other implementations  
use it automatically, but with OpenSSL I have to
do it manually. So I added a

         int rc = SSL_key_update(ssl, SSL_KEY_UPDATE_REQUESTED);

after 1000 I/O operations and looked what happened.

I started with the Java client and it gets wrong application data in  
such a situation.

I tested with our Windows implementation (I know it may have  
interoperability issues) and here I can see the
data I get from the server side and it is the same that I see on  
server side in the callback, but the Windows
DecryptMessage function fails with SEC_E_INVALID_TOKEN. (I had  
expected something like SEC_I_RENEGOTIATE to
get the information to put this record into InitializeSecurityContext.)

The Linux client also aborts the connection, but here I have not yet  
more details, but either the decryption fails
or the decrypted application data is wrong. Hopefully I can dig in  
there next week.

Here is what happens on server side:

1. I call SSL_key_update (see above)
2. I call SSL_write with application data
3. The write callback is invoked with this data:
    DATA: 17 03 03 00 16 b6 02 b1 06 87 f2 30 0d 77 35 31 7a 5b 6d 3c  
c2 aa 11 2c 32 95 a9
4. The write callback is invoked again with application data provided  
to SSL_write:
    DATA: 17 03 03 00 45 12 24 e5 66 36 2f 28 ea 62 2e 17 4c 62 f0 38  
07 7f 72 70 87 25 ba 45 ff cf f7 9f 0d 7d 48 ...

I see these data arrive at the client side (Windows):
    DATA: 17 03 03 00 16 b6 02 b1 06 87 f2 30 0d 77 35 31 7a 5b 6d 3c  
c2 aa 11 2c 32 95 a9
but get the error described above. In Java I get wrong application  
data, so it seems to decrypt this as application
data?!

I saw an additional call to SSL_do_handshake in the apps/s_server.c  
and tried this, but it did not change anything.

Is there anything else (except calling SSL_key_update) I have to take care of?

Did anyone use this successfully with a Java SSLSocket? (We currently  
use Azul 8 which has a backport of TLSv1.3, and Azul 11).

Has anyone experience with Windows Schannel against OpenSSL?

I still believe that I do something wrong, because the Linux client  
also fails.

Any hints are appreciated.


Regards,
Andreas

Reply | Threaded
Open this post in threaded view
|

Re: Question about SSL_key_update

OpenSSL - User mailing list
On Thu, Jul 09, 2020 at 06:07:41PM +0000, Andreas Müller wrote:

> Hi,
>
> I "inherited" our project to support/use TLSv1.3 from a late colleague. We
> have a server written in C++ (Windows, Linux)
> and clients (Windows, Linux, also written in C++ and also a Java client).
> With Java, we use the native SSLSocket implementation, in Windows we use
> Schannel and in Linux we use OpenSSL 1.1.1g. It seems to work and even
> interoperability
> did not show issues on some initial testing.
>
> I was curious about SSL_key_update. I read that other implementations use it
> automatically, but with OpenSSL I have to
> do it manually. So I added a
>
>         int rc = SSL_key_update(ssl, SSL_KEY_UPDATE_REQUESTED);
>
> after 1000 I/O operations and looked what happened.
>
> I started with the Java client and it gets wrong application data in such a
> situation.
>
> I tested with our Windows implementation (I know it may have
> interoperability issues) and here I can see the
> data I get from the server side and it is the same that I see on server side
> in the callback, but the Windows
> DecryptMessage function fails with SEC_E_INVALID_TOKEN. (I had expected
> something like SEC_I_RENEGOTIATE to
> get the information to put this record into InitializeSecurityContext.)
>
> The Linux client also aborts the connection, but here I have not yet more
> details, but either the decryption fails
> or the decrypted application data is wrong. Hopefully I can dig in there
> next week.
>
> Here is what happens on server side:
>
> 1. I call SSL_key_update (see above)
> 2. I call SSL_write with application data
> 3. The write callback is invoked with this data:
>    DATA: 17 03 03 00 16 b6 02 b1 06 87 f2 30 0d 77 35 31 7a 5b 6d 3c c2 aa
> 11 2c 32 95 a9
> 4. The write callback is invoked again with application data provided to
> SSL_write:
>    DATA: 17 03 03 00 45 12 24 e5 66 36 2f 28 ea 62 2e 17 4c 62 f0 38 07 7f
> 72 70 87 25 ba 45 ff cf f7 9f 0d 7d 48 ...
>
> I see these data arrive at the client side (Windows):
>    DATA: 17 03 03 00 16 b6 02 b1 06 87 f2 30 0d 77 35 31 7a 5b 6d 3c c2 aa
> 11 2c 32 95 a9
> but get the error described above. In Java I get wrong application data, so
> it seems to decrypt this as application
> data?!
>
> I saw an additional call to SSL_do_handshake in the apps/s_server.c and
> tried this, but it did not change anything.
>
> Is there anything else (except calling SSL_key_update) I have to take care of?

Per the documentation (https://www.openssl.org/docs/man1.1.1/man3/SSL_key_update.html)
it sounds like you're doing all you need to be.  (In particular, you don't need
to call SSL_do_handshake().)

Probably the fastest way to track down what's happening is to get a full packet
capture and keylog (e.g., with s_client -keylogfile path/to/log, then point wireshark
at the trace+keys.

-Ben