OpenSSL 1.1.0 assertion failure: ssl_free_wbio_buffer()

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

OpenSSL 1.1.0 assertion failure: ssl_free_wbio_buffer()

Marcus.Schafheutle
Hello,
 
we encountered a strange behaviour of OpenSSL 1.1.0 when our test with "sslscan" provokes an unfinished 
handshake. The problem exists since OpenSSL 1.1.0 - there were no problems with OpenSSL 1.0.2x.
 
Our asynchronous communication approach uses memory BIOs in order to read/write data from other 
communication layers into the SSL object. After the read/write operations are done, the BIOs are freed.
 
Our example code is basically as follows:
---
err = SSL_accept (ssl); 
 
BIO *rBIO = BIO_new(BIO_s_mem());
BIO *wBIO = BIO_new(BIO_s_mem());
    
SSL_set_bio(ssl, rBIO, wBIO); 
 
// ... operate with read / write BIOs and SSL_read/SSL_write
 
SSL_set_bio(ssl, NULL, NULL); // free BIOs when finished
 
SSL_free(ssl);
---
 
When calling SSL_free() after a complete handshake, everything is fine, but when calling SSL_free() after an 
unfinished handshake, the assertion "assert(s->wbio != NULL);" in ssl_free_wbio_buffer() fails:
---
void ssl_free_wbio_buffer(SSL *s)
{
    /* callers ensure s is never null */
    if (s->bbio == NULL)
        return;
    s->wbio = BIO_pop(s->wbio);
    assert(s->wbio != NULL);            /* <- this assertion fails! */
    BIO_free(s->bbio);
    s->bbio = NULL;
}
---
 
With a complete handshake the new attribute “bbio” is freed by calling the method tls_finish_handshake(), 
which itself calls ssl_free_wbio_buffer(). When the handshake is not finished successfully, the “bbio” is not freed, and 
therefore when calling SSL_free() the assert in ssl_free_wbio_buffer() fails.
  
Is this a faulty behavior of OpenSSL 1.1.0 or is the fault on my side? 
 
Thank you for your help!
 
Best regards,
Marcus

--
openssl-users mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-users
Reply | Threaded
Open this post in threaded view
|

Re: OpenSSL 1.1.0 assertion failure: ssl_free_wbio_buffer()

Matt Caswell-2


On 16/04/18 09:19, [hidden email] wrote:
> SSL_set_bio(ssl, NULL, NULL); // free BIOs when finished

There should be no reason to do this. The BIO's will get freed
automatically by the SSL_free() call.


> Is this a faulty behavior of OpenSSL 1.1.0 or is the fault on my side? 


This looks like a bug in OpenSSL. It should be able to tolerate passing
NULL for wbio in the SSL_set_bio call, but apparently it can't in the
case of a failed handshake. Please could you open an issue on github for
this?

The workaround is to remove the SSL_set_bio(SSL, NULL, NULL) call. It
should not be necessary.

Matt

--
openssl-users mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-users
Reply | Threaded
Open this post in threaded view
|

Re: OpenSSL 1.1.0 assertion failure: ssl_free_wbio_buffer()

Matt Caswell-2


On 16/04/18 11:38, Matt Caswell wrote:

>
>
> On 16/04/18 09:19, [hidden email] wrote:
>> SSL_set_bio(ssl, NULL, NULL); // free BIOs when finished
>
> There should be no reason to do this. The BIO's will get freed
> automatically by the SSL_free() call.
>
>
>> Is this a faulty behavior of OpenSSL 1.1.0 or is the fault on my side? 
>
>
> This looks like a bug in OpenSSL. It should be able to tolerate passing
> NULL for wbio in the SSL_set_bio call, but apparently it can't in the
> case of a failed handshake. Please could you open an issue on github for
> this?
>
> The workaround is to remove the SSL_set_bio(SSL, NULL, NULL) call. It
> should not be necessary.

I created a fix and some tests here:

https://github.com/openssl/openssl/pull/5966

It turns out the assertion is completely bogus and can simply be removed.

Matt

--
openssl-users mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-users