a possible bug for certain usage of openSSL1.1.0 with TLSv1

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

a possible bug for certain usage of openSSL1.1.0 with TLSv1

Ma chunhui
Hi, Openssl team
After we upgrade openSSL from 1.0.2 to 1.1.0f, we met an error which might be a bug.

1. our usage
We're using java application, and using JNI call to call OpenSSL API.(In fact, we're using an opensource project named wildfly-openssl)

After context and ssl is created, for one record, the decryption step is like this:
call BIO_write to put encrypted data into OpenSSL
call SSL_read(ssl, buf=0x0, len=0) to read 0 byte to an empty address(Note, this is a 0-length byte read)
use SSL_pending to check if there is any decrypted data in the result buffer
if there is data, use SSL_read to read pending data into dest buffer.

2. The usage works fine for openSSL 1.0.1 and 1.0.2 with all protocols. But not work for OpenSSL 1.1.0 with TLSv1.

The reason is: TLSv1 needs two records for application data,  and decrypting the first record will get a 0 length result buffer, and decrypting the second record will get the final decrypted data in the result buffer.

 In OpenSSL 1.0.1 & 1.0.2,  the first 0 length result buffer doesn't matter but  but in OpenSSL 1.1.0  the first 0 length result buffer will stop the next decryptions.

The process in OpenSSL1.1.0 is like this:
For the first record: after BIO_write , SSL_read(ssl, buf=0x0, length=0) is called, the first record is decrypted, tls1_enc is executed and get a 0 length result buffer, and then in ssl3_get_record, numrpipes is set to 1. Then after the execution return to ssl3_read_bytes, it found the user want to read 0 length, so it directly returns. leaving the state of the result buffer to un-read(don't have a chance to set the buffer to have been read).
Then for the second record, before ssl3_get_record, it first check numrpipes, and it found the numrpipes is not 0, and last buffer is not read, so it won't process the second record.  and This cause fails.

and in OpenSSL 1.0.2, the condition of ssl3_get_record is if (rr->length == 0 || xxx), so ssl3_get_record can process the second record.

Of course, I can change our usage to let it work for OpenSSL 1.1.0, for example, we can call SSL_read even we found SSL_pending is 0. But that means we may need another JNI call, which may affect our performance (we don't want to remove the 0 length SSL_read)

A possible fix might like this:
+++b/ssl/record/rec_layer_s3.c
@@ -1132,6 +1132,12@@int ssl3_read_bytes(SSL *s, int type, int *recvd_type, unsigned char *buf)
 
         if (recvd_type != NULL)
             *recvd_type = SSL3_RECORD_get_type(rr);
+        if (type == SSL3_RT_APPLICATION_DATA) {
+             /* mark any zero length record as consumed */
+             if (SSL3_RECORD_get_length(rr) == 0)
                  SSL3_RECORD_set_read(rr);
 
         if (len <= 0)
            return (len);

Thanks.



--
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
Reply | Threaded
Open this post in threaded view
|

Re: a possible bug for certain usage of openSSL1.1.0 with TLSv1

Matt Caswell-2


On 02/11/17 13:01, Ma chunhui wrote:
> Hi, Openssl team
> After we upgrade openSSL from 1.0.2 to 1.1.0f, we met an error which
> might be a bug.


Does this fix it:
https://github.com/openssl/openssl/pull/4685 (master)
https://github.com/openssl/openssl/pull/4686 (1.1.0)

Matt

--
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev