[openssl.org #2891] deadlock in X509_PUBKEY_get without recursive mutexes

[Rich Salz via RT]

I am using openssl-1.0.1c, and found a deadlock when using the library.

In function X509_PUBKEY_get (xpubkey.c:175) it locks
CRYPTO_LOCK_EVP_PKEY.  Three lines later (so xpubkey.c:178), it calls
EVP_PKEY_free which also locks CRYPTO_LOCK_EVP_PKEY (p_lib.c: 393).

This behavior is fine if the user is providing OpenSSL with recursive
mutexes, but I gather from the example code mttest.c that recursive
mutexes are not required, as it creates non-recursive pthread mutexes.

Here is a callstack from a deadlocked thread at this point
(codulus::SSLLockingCallback is my user provided locking callback):

#5  0x000000000044b1eb in codulus::SSLLockingCallback (mode=9,
type=10, file=0x68eead "p_lib.c", line=393)
    at ../..//util/ssl.cc:19
#6  0x000000000045f198 in CRYPTO_add_lock ()
#7  0x00000000004d0687 in EVP_PKEY_free ()
#8  0x00000000005ff020 in X509_PUBKEY_get ()
#9  0x00000000004e5c01 in internal_verify ()
#10 0x00000000004e65bf in X509_verify_cert ()
#11 0x0000000000472720 in ssl_verify_cert_chain ()
#12 0x0000000000507d7b in ssl3_get_server_certificate ()
#13 0x000000000050c184 in ssl3_connect ()
#14 0x0000000000465d87 in ssl23_connect ()
#15 0x0000000000466741 in ssl23_write ()

I hope this is useful.

Also, OpenSSL is great.

Cheers,
Ben Hendrickson

______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
Development Mailing List                       [hidden email]
Automated List Manager                           [hidden email]

[Quanah Gibson-Mount-3]

--On Wednesday, October 03, 2012 10:41 AM +0200 Ben Hendrickson via RT
<[hidden email]> wrote:

> I am using openssl-1.0.1c, and found a deadlock when using the library.
>
> In function X509_PUBKEY_get (xpubkey.c:175) it locks
> CRYPTO_LOCK_EVP_PKEY.  Three lines later (so xpubkey.c:178), it calls
> EVP_PKEY_free which also locks CRYPTO_LOCK_EVP_PKEY (p_lib.c: 393).
>
> This behavior is fine if the user is providing OpenSSL with recursive
> mutexes, but I gather from the example code mttest.c that recursive
> mutexes are not required, as it creates non-recursive pthread mutexes.
>
> Here is a callstack from a deadlocked thread at this point
> (codulus::SSLLockingCallback is my user provided locking callback):
>
># 5  0x000000000044b1eb in codulus::SSLLockingCallback (mode=9,
> type=10, file=0x68eead "p_lib.c", line=393)
>     at ../..//util/ssl.cc:19
># 6  0x000000000045f198 in CRYPTO_add_lock ()
># 7  0x00000000004d0687 in EVP_PKEY_free ()
># 8  0x00000000005ff020 in X509_PUBKEY_get ()
># 9  0x00000000004e5c01 in internal_verify ()
># 10 0x00000000004e65bf in X509_verify_cert ()
># 11 0x0000000000472720 in ssl_verify_cert_chain ()
># 12 0x0000000000507d7b in ssl3_get_server_certificate ()
># 13 0x000000000050c184 in ssl3_connect ()
># 14 0x0000000000465d87 in ssl23_connect ()
># 15 0x0000000000466741 in ssl23_write ()

Dupe of 2866?

--Quanah


--

Quanah Gibson-Mount
Sr. Member of Technical Staff
Zimbra, Inc
A Division of VMware, Inc.
--------------------
Zimbra ::  the leader in open source messaging and collaboration
______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
Development Mailing List                       [hidden email]
Automated List Manager                           [hidden email]