Memory issues with ssl handshake

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

Memory issues with ssl handshake

Amnu
Hi,

Valgrind gives me a *lot* of messages like this

==19021== 2,056 bytes in 2 blocks are indirectly lost in loss record 186
of 190
==19021==    at 0x68EAC8B: malloc (in
/usr/lib/valgrind/vgpreload_memcheck-x86-linux.so)
==19021==    by 0x6C472DB: default_malloc_ex (mem.c:79)
==19021==    by 0x6C4795F: CRYPTO_malloc (mem.c:306)
==19021==    by 0x6C73940: bn_expand_internal (bn_lib.c:336)
==19021==    by 0x6C73AE0: bn_expand2 (bn_lib.c:451)
==19021==    by 0x6C73BB2: BN_set_bit (bn_lib.c:730)
==19021==    by 0x6C7E16E: BN_MONT_CTX_set (bn_mont.c:514)
==19021==    by 0x6C7E402: BN_MONT_CTX_set_locked (bn_mont.c:552)
==19021==    by 0x6C97053: RSA_eay_public_decrypt (rsa_eay.c:693)
==19021==    by 0x6C97F1F: RSA_public_decrypt (rsa_lib.c:309)
==19021==    by 0x6C98BEB: int_rsa_verify (rsa_sign.c:182)
==19021==    by 0x6C98F57: RSA_verify (rsa_sign.c:284)
==19021==    by 0x6949E16: ssl3_get_key_exchange (s3_clnt.c:1562)
==19021==    by 0x694BF5F: ssl3_connect (s3_clnt.c:335)
==19021==    by 0x6964AB9: SSL_connect (ssl_lib.c:933)
==19021==    by 0x6953FDD: ssl23_connect (s23_clnt.c:693)
==19021==    by 0x6963FBC: SSL_do_handshake (ssl_lib.c:2368)
==19021==    by 0x8060D14: ssl_connect (ssl.c:1078)

Compared to the above they vary only in one line, e.g.

==19021== 1,024 bytes in 2 blocks are indirectly lost in loss record 181
of 190
==19021==    at 0x68EAC8B: malloc (in
/usr/lib/valgrind/vgpreload_memcheck-x86-linux.so)
==19021==    by 0x6C472DB: default_malloc_ex (mem.c:79)
==19021==    by 0x6C4795F: CRYPTO_malloc (mem.c:306)
==19021==    by 0x6C73940: bn_expand_internal (bn_lib.c:336)
==19021==    by 0x6C73AE0: bn_expand2 (bn_lib.c:451)
==19021==    by 0x6C73D53: BN_copy (bn_lib.c:506)
==19021==    by 0x6C7DFCA: BN_MONT_CTX_set (bn_mont.c:421)
==19021==    by 0x6C7E402: BN_MONT_CTX_set_locked (bn_mont.c:552)
==19021==    by 0x6C97053: RSA_eay_public_decrypt (rsa_eay.c:693)
==19021==    by 0x6C97F1F: RSA_public_decrypt (rsa_lib.c:309)
==19021==    by 0x6C98BEB: int_rsa_verify (rsa_sign.c:182)
==19021==    by 0x6C98F57: RSA_verify (rsa_sign.c:284)
==19021==    by 0x6949E16: ssl3_get_key_exchange (s3_clnt.c:1562)
==19021==    by 0x694BF5F: ssl3_connect (s3_clnt.c:335)
==19021==    by 0x6964AB9: SSL_connect (ssl_lib.c:933)
==19021==    by 0x6953FDD: ssl23_connect (s23_clnt.c:693)
==19021==    by 0x6963FBC: SSL_do_handshake (ssl_lib.c:2368)
==19021==    by 0x8060D14: ssl_connect (ssl.c:1078)

The application in question is a http proxy and is leaking memory
heavily. Actually, Valgrind also reports many entries as "still
reachable" instead of "indirectly lost" but even those regions are
mentioned by Valgrind like the above two blocks.

I doubt OpenSSL has such blatant memory leaks but I have been searching
for the leaks for some time now and just cannot find anything - also due
to the fact that Valgrind reports all the leaks like above, which is not
really helpful. I do realize that for detailed help I would need to give
some code on how the connections are set up but that code is pretty
large. So best would be some hint as to where to look for or what
regions of code to supply.

Looking for hints as to what is going wrong I went through some of the
OpenSSL functions mentioned above and after "leaving my code" and
entering OpenSSL code the only thing being passed around the functions
is the actual SSL object - and no buffers of any kind. So I suppose the
source of the problem lies in setting up the object, it's context or
what ever is stuck to it.

The first thing I checked on was how the SSL connections are torn down
in my application:
    ERR_remove_state(0);
    SSL_CTX_free(ssl->ctx);
    SSL_free(ssl);
The first line was added only after starting the search for the leaks
and I'm still not sure if that line doesn't break anything.

I did find some minor leaks in my code along the way (e.g. a forgotten
ASN1_INTEGER_free()) but they were neither easy to find - because
Valgrind also reported them like the above two blocks, meaning in the
wrong place! - nor did they change much.

Does anyone have some tips for me ?
______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
User Support Mailing List                    [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

RE: Memory issues with ssl handshake

Charles Mills
Not sure if it help at all but FWIW I am using Windows pre-built OpenSSL
dll's with server & client certificates, CAs, DH keys, and a CRL, and I am
seeing no memory leaks whatsoever in Visual Studio 2010.

Charles

-----Original Message-----
From: [hidden email]
[mailto:[hidden email]] On Behalf Of Thomas Eckert
Sent: Wednesday, September 12, 2012 7:59 AM
To: [hidden email]
Subject: Memory issues with ssl handshake

Hi,

Valgrind gives me a *lot* of messages like this

==19021== 2,056 bytes in 2 blocks are indirectly lost in loss record 186 of
190
==19021==    at 0x68EAC8B: malloc (in
/usr/lib/valgrind/vgpreload_memcheck-x86-linux.so)
==19021==    by 0x6C472DB: default_malloc_ex (mem.c:79)
==19021==    by 0x6C4795F: CRYPTO_malloc (mem.c:306)
==19021==    by 0x6C73940: bn_expand_internal (bn_lib.c:336)
==19021==    by 0x6C73AE0: bn_expand2 (bn_lib.c:451)
==19021==    by 0x6C73BB2: BN_set_bit (bn_lib.c:730)
==19021==    by 0x6C7E16E: BN_MONT_CTX_set (bn_mont.c:514)
==19021==    by 0x6C7E402: BN_MONT_CTX_set_locked (bn_mont.c:552)
==19021==    by 0x6C97053: RSA_eay_public_decrypt (rsa_eay.c:693)
==19021==    by 0x6C97F1F: RSA_public_decrypt (rsa_lib.c:309)
==19021==    by 0x6C98BEB: int_rsa_verify (rsa_sign.c:182)
==19021==    by 0x6C98F57: RSA_verify (rsa_sign.c:284)
==19021==    by 0x6949E16: ssl3_get_key_exchange (s3_clnt.c:1562)
==19021==    by 0x694BF5F: ssl3_connect (s3_clnt.c:335)
==19021==    by 0x6964AB9: SSL_connect (ssl_lib.c:933)
==19021==    by 0x6953FDD: ssl23_connect (s23_clnt.c:693)
==19021==    by 0x6963FBC: SSL_do_handshake (ssl_lib.c:2368)
==19021==    by 0x8060D14: ssl_connect (ssl.c:1078)

Compared to the above they vary only in one line, e.g.

==19021== 1,024 bytes in 2 blocks are indirectly lost in loss record 181 of
190
==19021==    at 0x68EAC8B: malloc (in
/usr/lib/valgrind/vgpreload_memcheck-x86-linux.so)
==19021==    by 0x6C472DB: default_malloc_ex (mem.c:79)
==19021==    by 0x6C4795F: CRYPTO_malloc (mem.c:306)
==19021==    by 0x6C73940: bn_expand_internal (bn_lib.c:336)
==19021==    by 0x6C73AE0: bn_expand2 (bn_lib.c:451)
==19021==    by 0x6C73D53: BN_copy (bn_lib.c:506)
==19021==    by 0x6C7DFCA: BN_MONT_CTX_set (bn_mont.c:421)
==19021==    by 0x6C7E402: BN_MONT_CTX_set_locked (bn_mont.c:552)
==19021==    by 0x6C97053: RSA_eay_public_decrypt (rsa_eay.c:693)
==19021==    by 0x6C97F1F: RSA_public_decrypt (rsa_lib.c:309)
==19021==    by 0x6C98BEB: int_rsa_verify (rsa_sign.c:182)
==19021==    by 0x6C98F57: RSA_verify (rsa_sign.c:284)
==19021==    by 0x6949E16: ssl3_get_key_exchange (s3_clnt.c:1562)
==19021==    by 0x694BF5F: ssl3_connect (s3_clnt.c:335)
==19021==    by 0x6964AB9: SSL_connect (ssl_lib.c:933)
==19021==    by 0x6953FDD: ssl23_connect (s23_clnt.c:693)
==19021==    by 0x6963FBC: SSL_do_handshake (ssl_lib.c:2368)
==19021==    by 0x8060D14: ssl_connect (ssl.c:1078)

The application in question is a http proxy and is leaking memory heavily.
Actually, Valgrind also reports many entries as "still reachable" instead of
"indirectly lost" but even those regions are mentioned by Valgrind like the
above two blocks.

I doubt OpenSSL has such blatant memory leaks but I have been searching for
the leaks for some time now and just cannot find anything - also due to the
fact that Valgrind reports all the leaks like above, which is not really
helpful. I do realize that for detailed help I would need to give some code
on how the connections are set up but that code is pretty large. So best
would be some hint as to where to look for or what regions of code to
supply.

Looking for hints as to what is going wrong I went through some of the
OpenSSL functions mentioned above and after "leaving my code" and entering
OpenSSL code the only thing being passed around the functions is the actual
SSL object - and no buffers of any kind. So I suppose the source of the
problem lies in setting up the object, it's context or what ever is stuck to
it.

The first thing I checked on was how the SSL connections are torn down in my
application:
    ERR_remove_state(0);
    SSL_CTX_free(ssl->ctx);
    SSL_free(ssl);
The first line was added only after starting the search for the leaks and
I'm still not sure if that line doesn't break anything.

I did find some minor leaks in my code along the way (e.g. a forgotten
ASN1_INTEGER_free()) but they were neither easy to find - because Valgrind
also reported them like the above two blocks, meaning in the wrong place! -
nor did they change much.

Does anyone have some tips for me ?

______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
User Support Mailing List                    [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: Memory issues with ssl handshake

michel-60
In reply to this post by Amnu
Hi Thomas,

I don't know if it makes a real difference concerning memory leaks,
but I would have freed the TLS session *BEFORE* freing the context :
    SSL_free(ssl);
    ...
    SSL_CTX_free(ssl->ctx);
Does your app setup and free a context each time a client is connecting ?

Le 12/09/2012 16:58, Thomas Eckert a écrit :

> Hi,
>
> Valgrind gives me a *lot* of messages like this
>
> ==19021== 2,056 bytes in 2 blocks are indirectly lost in loss record
> 186 of 190
> ==19021==    at 0x68EAC8B: malloc (in
> /usr/lib/valgrind/vgpreload_memcheck-x86-linux.so)
> ==19021==    by 0x6C472DB: default_malloc_ex (mem.c:79)
> ==19021==    by 0x6C4795F: CRYPTO_malloc (mem.c:306)
> ==19021==    by 0x6C73940: bn_expand_internal (bn_lib.c:336)
> ==19021==    by 0x6C73AE0: bn_expand2 (bn_lib.c:451)
> ==19021==    by 0x6C73BB2: BN_set_bit (bn_lib.c:730)
> ==19021==    by 0x6C7E16E: BN_MONT_CTX_set (bn_mont.c:514)
> ==19021==    by 0x6C7E402: BN_MONT_CTX_set_locked (bn_mont.c:552)
> ==19021==    by 0x6C97053: RSA_eay_public_decrypt (rsa_eay.c:693)
> ==19021==    by 0x6C97F1F: RSA_public_decrypt (rsa_lib.c:309)
> ==19021==    by 0x6C98BEB: int_rsa_verify (rsa_sign.c:182)
> ==19021==    by 0x6C98F57: RSA_verify (rsa_sign.c:284)
> ==19021==    by 0x6949E16: ssl3_get_key_exchange (s3_clnt.c:1562)
> ==19021==    by 0x694BF5F: ssl3_connect (s3_clnt.c:335)
> ==19021==    by 0x6964AB9: SSL_connect (ssl_lib.c:933)
> ==19021==    by 0x6953FDD: ssl23_connect (s23_clnt.c:693)
> ==19021==    by 0x6963FBC: SSL_do_handshake (ssl_lib.c:2368)
> ==19021==    by 0x8060D14: ssl_connect (ssl.c:1078)
>
> Compared to the above they vary only in one line, e.g.
>
> ==19021== 1,024 bytes in 2 blocks are indirectly lost in loss record
> 181 of 190
> ==19021==    at 0x68EAC8B: malloc (in
> /usr/lib/valgrind/vgpreload_memcheck-x86-linux.so)
> ==19021==    by 0x6C472DB: default_malloc_ex (mem.c:79)
> ==19021==    by 0x6C4795F: CRYPTO_malloc (mem.c:306)
> ==19021==    by 0x6C73940: bn_expand_internal (bn_lib.c:336)
> ==19021==    by 0x6C73AE0: bn_expand2 (bn_lib.c:451)
> ==19021==    by 0x6C73D53: BN_copy (bn_lib.c:506)
> ==19021==    by 0x6C7DFCA: BN_MONT_CTX_set (bn_mont.c:421)
> ==19021==    by 0x6C7E402: BN_MONT_CTX_set_locked (bn_mont.c:552)
> ==19021==    by 0x6C97053: RSA_eay_public_decrypt (rsa_eay.c:693)
> ==19021==    by 0x6C97F1F: RSA_public_decrypt (rsa_lib.c:309)
> ==19021==    by 0x6C98BEB: int_rsa_verify (rsa_sign.c:182)
> ==19021==    by 0x6C98F57: RSA_verify (rsa_sign.c:284)
> ==19021==    by 0x6949E16: ssl3_get_key_exchange (s3_clnt.c:1562)
> ==19021==    by 0x694BF5F: ssl3_connect (s3_clnt.c:335)
> ==19021==    by 0x6964AB9: SSL_connect (ssl_lib.c:933)
> ==19021==    by 0x6953FDD: ssl23_connect (s23_clnt.c:693)
> ==19021==    by 0x6963FBC: SSL_do_handshake (ssl_lib.c:2368)
> ==19021==    by 0x8060D14: ssl_connect (ssl.c:1078)
>
> The application in question is a http proxy and is leaking memory
> heavily. Actually, Valgrind also reports many entries as "still
> reachable" instead of "indirectly lost" but even those regions are
> mentioned by Valgrind like the above two blocks.
>
> I doubt OpenSSL has such blatant memory leaks but I have been
> searching for the leaks for some time now and just cannot find
> anything - also due to the fact that Valgrind reports all the leaks
> like above, which is not really helpful. I do realize that for
> detailed help I would need to give some code on how the connections
> are set up but that code is pretty large. So best would be some hint
> as to where to look for or what regions of code to supply.
>
> Looking for hints as to what is going wrong I went through some of the
> OpenSSL functions mentioned above and after "leaving my code" and
> entering OpenSSL code the only thing being passed around the functions
> is the actual SSL object - and no buffers of any kind. So I suppose
> the source of the problem lies in setting up the object, it's context
> or what ever is stuck to it.
>
> The first thing I checked on was how the SSL connections are torn down
> in my application:
>    ERR_remove_state(0);
>    SSL_CTX_free(ssl->ctx);
>    SSL_free(ssl);
> The first line was added only after starting the search for the leaks
> and I'm still not sure if that line doesn't break anything.
>
> I did find some minor leaks in my code along the way (e.g. a forgotten
> ASN1_INTEGER_free()) but they were neither easy to find - because
> Valgrind also reported them like the above two blocks, meaning in the
> wrong place! - nor did they change much.
>
> Does anyone have some tips for me ?
> ______________________________________________________________________
> OpenSSL Project http://www.openssl.org
> User Support Mailing List [hidden email]
> Automated List Manager [hidden email]
>
>

______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
User Support Mailing List                    [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: Memory issues with ssl handshake

Amnu
Hi Michel,

Thanks for trying to help, I really appreciate it :-)

"Does your app setup and free a context each time a client is connecting ?"

The context is created only when a client requests a HTTPS connection
and is destroyed together with the SSL session once the connection goes
down. It is rather related to connections then to clients since one
client can open several connections but I think you implied one
connection per client and then the answer is 'yes'.

I will try freeing the session before the context and come back with the
results.

Regards,
  Thomas


On 09/13/2012 10:09 AM, Michel wrote:

> Hi Thomas,
>
> I don't know if it makes a real difference concerning memory leaks,
> but I would have freed the TLS session *BEFORE* freing the context :
>    SSL_free(ssl);
>    ...
>    SSL_CTX_free(ssl->ctx);
> Does your app setup and free a context each time a client is connecting ?
>
> Le 12/09/2012 16:58, Thomas a écrit :
>> Hi,
>>
>> Valgrind gives me a *lot* of messages like this
>>
>> ==19021== 2,056 bytes in 2 blocks are indirectly lost in loss record
>> 186 of 190
>> ==19021==    at 0x68EAC8B: malloc (in
>> /usr/lib/valgrind/vgpreload_memcheck-x86-linux.so)
>> ==19021==    by 0x6C472DB: default_malloc_ex (mem.c:79)
>> ==19021==    by 0x6C4795F: CRYPTO_malloc (mem.c:306)
>> ==19021==    by 0x6C73940: bn_expand_internal (bn_lib.c:336)
>> ==19021==    by 0x6C73AE0: bn_expand2 (bn_lib.c:451)
>> ==19021==    by 0x6C73BB2: BN_set_bit (bn_lib.c:730)
>> ==19021==    by 0x6C7E16E: BN_MONT_CTX_set (bn_mont.c:514)
>> ==19021==    by 0x6C7E402: BN_MONT_CTX_set_locked (bn_mont.c:552)
>> ==19021==    by 0x6C97053: RSA_eay_public_decrypt (rsa_eay.c:693)
>> ==19021==    by 0x6C97F1F: RSA_public_decrypt (rsa_lib.c:309)
>> ==19021==    by 0x6C98BEB: int_rsa_verify (rsa_sign.c:182)
>> ==19021==    by 0x6C98F57: RSA_verify (rsa_sign.c:284)
>> ==19021==    by 0x6949E16: ssl3_get_key_exchange (s3_clnt.c:1562)
>> ==19021==    by 0x694BF5F: ssl3_connect (s3_clnt.c:335)
>> ==19021==    by 0x6964AB9: SSL_connect (ssl_lib.c:933)
>> ==19021==    by 0x6953FDD: ssl23_connect (s23_clnt.c:693)
>> ==19021==    by 0x6963FBC: SSL_do_handshake (ssl_lib.c:2368)
>> ==19021==    by 0x8060D14: ssl_connect (ssl.c:1078)
>>
>> Compared to the above they vary only in one line, e.g.
>>
>> ==19021== 1,024 bytes in 2 blocks are indirectly lost in loss record
>> 181 of 190
>> ==19021==    at 0x68EAC8B: malloc (in
>> /usr/lib/valgrind/vgpreload_memcheck-x86-linux.so)
>> ==19021==    by 0x6C472DB: default_malloc_ex (mem.c:79)
>> ==19021==    by 0x6C4795F: CRYPTO_malloc (mem.c:306)
>> ==19021==    by 0x6C73940: bn_expand_internal (bn_lib.c:336)
>> ==19021==    by 0x6C73AE0: bn_expand2 (bn_lib.c:451)
>> ==19021==    by 0x6C73D53: BN_copy (bn_lib.c:506)
>> ==19021==    by 0x6C7DFCA: BN_MONT_CTX_set (bn_mont.c:421)
>> ==19021==    by 0x6C7E402: BN_MONT_CTX_set_locked (bn_mont.c:552)
>> ==19021==    by 0x6C97053: RSA_eay_public_decrypt (rsa_eay.c:693)
>> ==19021==    by 0x6C97F1F: RSA_public_decrypt (rsa_lib.c:309)
>> ==19021==    by 0x6C98BEB: int_rsa_verify (rsa_sign.c:182)
>> ==19021==    by 0x6C98F57: RSA_verify (rsa_sign.c:284)
>> ==19021==    by 0x6949E16: ssl3_get_key_exchange (s3_clnt.c:1562)
>> ==19021==    by 0x694BF5F: ssl3_connect (s3_clnt.c:335)
>> ==19021==    by 0x6964AB9: SSL_connect (ssl_lib.c:933)
>> ==19021==    by 0x6953FDD: ssl23_connect (s23_clnt.c:693)
>> ==19021==    by 0x6963FBC: SSL_do_handshake (ssl_lib.c:2368)
>> ==19021==    by 0x8060D14: ssl_connect (ssl.c:1078)
>>
>> The application in question is a http proxy and is leaking memory
>> heavily. Actually, Valgrind also reports many entries as "still
>> reachable" instead of "indirectly lost" but even those regions are
>> mentioned by Valgrind like the above two blocks.
>>
>> I doubt OpenSSL has such blatant memory leaks but I have been
>> searching for the leaks for some time now and just cannot find
>> anything - also due to the fact that Valgrind reports all the leaks
>> like above, which is not really helpful. I do realize that for
>> detailed help I would need to give some code on how the connections
>> are set up but that code is pretty large. So best would be some hint
>> as to where to look for or what regions of code to supply.
>>
>> Looking for hints as to what is going wrong I went through some of
>> the OpenSSL functions mentioned above and after "leaving my code" and
>> entering OpenSSL code the only thing being passed around the
>> functions is the actual SSL object - and no buffers of any kind. So I
>> suppose the source of the problem lies in setting up the object, it's
>> context or what ever is stuck to it.
>>
>> The first thing I checked on was how the SSL connections are torn
>> down in my application:
>>    ERR_remove_state(0);
>>    SSL_CTX_free(ssl->ctx);
>>    SSL_free(ssl);
>> The first line was added only after starting the search for the leaks
>> and I'm still not sure if that line doesn't break anything.
>>
>> I did find some minor leaks in my code along the way (e.g. a
>> forgotten ASN1_INTEGER_free()) but they were neither easy to find -
>> because Valgrind also reported them like the above two blocks,
>> meaning in the wrong place! - nor did they change much.
>>
>> Does anyone have some tips for me ?
>> ______________________________________________________________________
>> OpenSSL Project http://www.openssl.org
>> User Support Mailing List [hidden email]
>> Automated List Manager [hidden email]
>>
>>
>
______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
User Support Mailing List                    [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: Memory issues with ssl handshake

michel-60
In reply to this post by michel-60
Hi again Thomas,

Do you really need to free your context each time you free your TLS
session ?
I believe it is not needed and at least not usual.
If you need several *DIFFERENT* contexts, implying different TLS
configurations/setup, wich, I think, is not so common,
you can keep them 'alive' during all your app 'run', even in
multi-threaded programs.
It would allow you to access some activity informations like the ones
documented in :
http://www.openssl.org/docs/ssl/SSL_CTX_sess_number.html

Hope this helps,
Regards

Le 13/09/2012 10:39, Thomas a écrit :

> Hi Michel,
>
> Thanks for trying to help, I really appreciate it :-)
>
> "Does your app setup and free a context each time a client is
> connecting ?"
>
> The context is created only when a client requests a HTTPS connection
> and is destroyed together with the SSL session once the connection
> goes down. It is rather related to connections then to clients since
> one client can open several connections but I think you implied one
> connection per client and then the answer is 'yes'.
>
> I will try freeing the session before the context and come back with
> the results.
>
> Regards,
>  Thomas
>

______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
User Support Mailing List                    [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: Memory issues with ssl handshake

Dominik Oepen-2
In reply to this post by Amnu
On 12.09.2012 16:58, Thomas Eckert wrote:
> Hi,
>
> Valgrind gives me a *lot* of messages like this
[snip]
> Does anyone have some tips for me ?

Have you seen the FAQ entry regarding valgrind:
http://www.openssl.org/support/faq.html#PROG14 ?

Regards,
Dominik
______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
User Support Mailing List                    [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: Memory issues with ssl handshake

Amnu
In reply to this post by michel-60
Hi again,

I've changed the code to reuse the SSL contexts but in terms of memory
consumption/release it did not change much - if anything at all. By the
way, is there a way to "unload" a certificate once it has been loaded
into a SSL context via SSL_CTX_use_certificate() ? I didn't find
anything in the docs and simply specifying NULL as cert parameter caused
a crash in OpenSSL.

The only places left that cause memory leaks are reported inside OpenSSL
as in

at 0x68EAC8B: malloc (in /usr/lib/valgrind/vgpreload_memcheck-x86-linux.so)
==27041==    by 0x6C472DB: default_malloc_ex (mem.c:79)
==27041==    by 0x6C4795F: CRYPTO_malloc (mem.c:306)
==27041==    by 0x6C73940: bn_expand_internal (bn_lib.c:336)
==27041==    by 0x6C73AE0: bn_expand2 (bn_lib.c:451)
==27041==    by 0x6C73BB2: BN_set_bit (bn_lib.c:730)
==27041==    by 0x6C7E16E: BN_MONT_CTX_set (bn_mont.c:514)
==27041==    by 0x6C7E402: BN_MONT_CTX_set_locked (bn_mont.c:552)
==27041==    by 0x6C95B56: RSA_eay_mod_exp (rsa_eay.c:782)
==27041==    by 0x6C96422: RSA_eay_private_decrypt (rsa_eay.c:565)
==27041==    by 0x6C97EDF: RSA_private_decrypt (rsa_lib.c:303)
==27041==    by 0x6942918: ssl3_get_client_key_exchange (s3_srvr.c:2038)
==27041==    by 0x6946693: ssl3_accept (s3_srvr.c:529)
==27041==    by 0x69513CA: ssl3_read_bytes (s3_pkt.c:941)
==27041==    by 0x694C688: ssl3_read_internal (s3_lib.c:3274)
==27041==    by 0x69642E8: SSL_read (ssl_lib.c:954)

Sometimes these are flagged "still reachable" and sometimes "indirectly
lost", usually both types are reported as I get a large amount of these
traces. One thing I noticed is that all goes well if I cause the code to
run sequentially (e.g. cause requests to come one ater another). Yet it
starts eating up memory like crazy if I cause several (HTTPS) requests
to come at once.

I'm at a loss here. Valgrind insists the leaks happen in OpenSSL code.
I'll be happy to supply more information if anyone has an idea of how to
approach this.

Regards,
  Thomas

On 09/13/2012 12:30 PM, Michel wrote:

> Hi again Thomas,
>
> Do you really need to free your context each time you free your TLS
> session ?
> I believe it is not needed and at least not usual.
> If you need several *DIFFERENT* contexts, implying different TLS
> configurations/setup, wich, I think, is not so common,
> you can keep them 'alive' during all your app 'run', even in
> multi-threaded programs.
> It would allow you to access some activity informations like the ones
> documented in :
> http://www.openssl.org/docs/ssl/SSL_CTX_sess_number.html
>
> Hope this helps,
> Regards
>
> Le 13/09/2012 10:39, Thomas a écrit :
>> Hi Michel,
>>
>> Thanks for trying to help, I really appreciate it :-)
>>
>> "Does your app setup and free a context each time a client is
>> connecting ?"
>>
>> The context is created only when a client requests a HTTPS connection
>> and is destroyed together with the SSL session once the connection
>> goes down. It is rather related to connections then to clients since
>> one client can open several connections but I think you implied one
>> connection per client and then the answer is 'yes'.
>>
>> I will try freeing the session before the context and come back with
>> the results.
>>
>> Regards,
>>  Thomas
>>
>
______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
User Support Mailing List                    [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

RE: Memory issues with ssl handshake

Charles Mills
Try plugging your code to exit after the first OpenSSL function, then after
the second, and so forth, and see if you can get down to the simplest case.

Charles

-----Original Message-----
From: [hidden email]
[mailto:[hidden email]] On Behalf Of Thomas
Sent: Monday, September 17, 2012 11:36 AM
To: Michel
Cc: [hidden email]
Subject: Re: Memory issues with ssl handshake

Hi again,

I've changed the code to reuse the SSL contexts but in terms of memory
consumption/release it did not change much - if anything at all. By the way,
is there a way to "unload" a certificate once it has been loaded into a SSL
context via SSL_CTX_use_certificate() ? I didn't find anything in the docs
and simply specifying NULL as cert parameter caused a crash in OpenSSL.

The only places left that cause memory leaks are reported inside OpenSSL as
in

at 0x68EAC8B: malloc (in /usr/lib/valgrind/vgpreload_memcheck-x86-linux.so)
==27041==    by 0x6C472DB: default_malloc_ex (mem.c:79)
==27041==    by 0x6C4795F: CRYPTO_malloc (mem.c:306)
==27041==    by 0x6C73940: bn_expand_internal (bn_lib.c:336)
==27041==    by 0x6C73AE0: bn_expand2 (bn_lib.c:451)
==27041==    by 0x6C73BB2: BN_set_bit (bn_lib.c:730)
==27041==    by 0x6C7E16E: BN_MONT_CTX_set (bn_mont.c:514)
==27041==    by 0x6C7E402: BN_MONT_CTX_set_locked (bn_mont.c:552)
==27041==    by 0x6C95B56: RSA_eay_mod_exp (rsa_eay.c:782)
==27041==    by 0x6C96422: RSA_eay_private_decrypt (rsa_eay.c:565)
==27041==    by 0x6C97EDF: RSA_private_decrypt (rsa_lib.c:303)
==27041==    by 0x6942918: ssl3_get_client_key_exchange (s3_srvr.c:2038)
==27041==    by 0x6946693: ssl3_accept (s3_srvr.c:529)
==27041==    by 0x69513CA: ssl3_read_bytes (s3_pkt.c:941)
==27041==    by 0x694C688: ssl3_read_internal (s3_lib.c:3274)
==27041==    by 0x69642E8: SSL_read (ssl_lib.c:954)

Sometimes these are flagged "still reachable" and sometimes "indirectly
lost", usually both types are reported as I get a large amount of these
traces. One thing I noticed is that all goes well if I cause the code to run
sequentially (e.g. cause requests to come one ater another). Yet it starts
eating up memory like crazy if I cause several (HTTPS) requests to come at
once.

I'm at a loss here. Valgrind insists the leaks happen in OpenSSL code.
I'll be happy to supply more information if anyone has an idea of how to
approach this.

Regards,
  Thomas

On 09/13/2012 12:30 PM, Michel wrote:

> Hi again Thomas,
>
> Do you really need to free your context each time you free your TLS
> session ?
> I believe it is not needed and at least not usual.
> If you need several *DIFFERENT* contexts, implying different TLS
> configurations/setup, wich, I think, is not so common, you can keep
> them 'alive' during all your app 'run', even in multi-threaded
> programs.
> It would allow you to access some activity informations like the ones
> documented in :
> http://www.openssl.org/docs/ssl/SSL_CTX_sess_number.html
>
> Hope this helps,
> Regards
>
> Le 13/09/2012 10:39, Thomas a écrit :
>> Hi Michel,
>>
>> Thanks for trying to help, I really appreciate it :-)
>>
>> "Does your app setup and free a context each time a client is
>> connecting ?"
>>
>> The context is created only when a client requests a HTTPS connection
>> and is destroyed together with the SSL session once the connection
>> goes down. It is rather related to connections then to clients since
>> one client can open several connections but I think you implied one
>> connection per client and then the answer is 'yes'.
>>
>> I will try freeing the session before the context and come back with
>> the results.
>>
>> Regards,
>>  Thomas
>>
>
______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
User Support Mailing List                    [hidden email]
Automated List Manager                           [hidden email]

______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
User Support Mailing List                    [hidden email]
Automated List Manager                           [hidden email]