renegotiation expected to fail when trust configuration has changed.
We have a C++ client application and a C++ server application using
OpenSSL 1.1.0f to encrypt the TCP/IP communication.
We enforce mutual authentication (also the server requests certificates from the clients and verifies if they are issued by a CA it trusts).
We are able to update certificates (Host certificates and trusted CA certificates) on clients and servers without stopping the executables or interrupting the communication. We managed the update of the trusted
CA certificates by using TLS Extension #3 (trusted_ca_keys) specified in RFC6066.
We are able to remove old obsolete certificates (Host certificates and trusted CA certificates) on clients and servers without stopping the executables or interrupting the communication.
New established TCP/IP sessions will always use the newest certificates.
The update of a trusted CA certificate might look like this:
Server and clients are deployed with certificates issued by
CA1, the CA1 certificate is deployed as trusted root certificate.
Certificates issued by
CAnew and the CAnew certificate will be deployed to the server and all clients. Whenever a new TCP/IP session is established the new certificates are used.
CAnew certificates are deployed, the CA1 certificates can be removed from the server and the clients.
We also want to force established TCP sessions to move from obsolete certificates the newest certificates be triggering a renegotiation. We expect the renegotiation to fail,
if the host does not have the right set of certificates.
This works fine, if we trigger the renegotiation at the client, but the real benefit would be to do the renegotiation on the server. So we could remove all already connected
malicious clients which have managed to steal obsolete certificates. But sadly this does not work the way we implemented it.
we implemented the renegotiation by essentially calling
But see our trace for a better illustration what is going on.
We use SSL_CTX_set_info_callback() to get insights into OpenSSL states and set the following essential configurations:
> Here I am missing server_parse_cb() (see Server trace scenario 1)
When a client receives a HelloRequest message it will send a new ClientHello and
attempt to resume the connection. The attempt at resumption does not occur when
a reneg is initiated from the client side unless you call
SSL_renegotiate_abbreviated() instead of SSL_renegotiate().
If the attempt at resumption is successful then custom extension parsing does
not get invoked on the server side. This is by design:
Unfortunately the docs in 1.1.0 are a bit lacking on this point. They have been
significantly revised and updated in 1.1.1 and hopefully this is a little more
obvious. In particular note that the 1.1.1 flag SSL_EXT_IGNORE_ON_RESUMPTION is
automatically set when calling SSL_CTX_add_server_custom_ext(). This is so that
the behaviour is backwards compatible with 1.1.0:
In 1.1.1 we have the new function SSL_CTX_add_custom_ext() which (among other
things) allows you to control whether the callback is called during resumption
or not. See the doc link above.
So, in order to solve your problem, I see 2 possible solutions:
1) Upgrade to 1.1.1 and use the new SSL_CTX_add_custom_ext() API.
2) Invalidate any sessions in the session cache for old connections that were
established using the old certificate. You'll need to some additional custom
code to track that I imagine. You'll also need to avoid session tickets for that
Probably the upgrade to 1.1.1 is your best bet since 1.1.0 is only supported
until September 2019 anyway.