SSL_accept blocks forever

[Henry Yip (hyip)]

Hi,

I have a SSL server program that accepts SSL connections. But after receiving around 150 connections confirmed with the "netstat -a  | grep ESTAB" command and many SYN_RECV half opened connections, the client is terminated. This forces the sockets to close.  The number of ESTAB connections goes to zero.  The SYN_RECV half opened connection still lingers. The SSL server continue to process these SYN_RECV half opened connections and soon even though the client is terminated, some half opened connection went int other ESTAB state.   At this point, the SSL server can no longer accept any new connections.

This happens on all Redhat Linux platform that I tested with kernel 2.4.20, 2.4.18, and 2.6.x.   Solaris 2.8 does not have this problem.

Below is the stack trace on where the code is blocking in.

[Switching to Thread -1210186048 (LWP 21173)]
0x0012e7a2 in _dl_sysinfo_int80 () from /lib/ld-linux.so.2
(gdb) where
#0  0x0012e7a2 in _dl_sysinfo_int80 () from /lib/ld-linux.so.2
#1  0x00bd33fb in __read_nocancel () from /lib/tls/libpthread.so.0
#2  0x00608d24 in BIO_sock_should_retry () from /lib/libcrypto.so.4
#3  0x00606d8c in BIO_read () from /lib/libcrypto.so.4
#4  0x007b21b0 in ssl23_read_bytes () from /lib/libssl.so.4
#5  0x007b0d31 in ssl23_get_client_hello () from /lib/libssl.so.4
#6  0x007b1309 in ssl23_accept () from /lib/libssl.so.4
#7  0x007b60ab in SSL_accept () from /lib/libssl.so.4
#8  0xb7e093bc in ACE_SSL_SOCK_Acceptor::ssl_accept (this=0xbffff6f0,
    [hidden email], timeout=0x0) at SSL_SOCK_Acceptor.cpp:74
#9  0xb7e097d5 in ACE_SSL_SOCK_Acceptor::accept (this=0xbffff6f0,
    [hidden email], remote_addr=0x0, timeout=0x0, restart=1,
    reset_new_handle=0) at SSL_SOCK_Acceptor.cpp:195
#10 0x0804aace in main ()

Does anyone has any idea on what is going on?

Thanks in advance.

-Henry

[Girish Venkatachalam]

The problem you are talking about seems to be
occurring at the TCP layer even before things come
into the hands of SSL_accept().

So I suggest you test this without SSL first.

You might hit some SSL session reuse limit and get a
similar problem but yours doesnt seem to be that.

And 150 is a very small number for you to run into
this sort of problem. I suggest you try some other
approach for attacking this problem.

Hope this helps.

regards,
Girish

--- "Henry Yip (hyip)" <[hidden email]> wrote:

> Hi,
>
> I have a SSL server program that accepts SSL
> connections. But after
> receiving around 150 connections confirmed with the
> "netstat -a  | grep
> ESTAB" command and many SYN_RECV half opened
> connections, the client is
> terminated. This forces the sockets to close.  The
> number of ESTAB
> connections goes to zero.  The SYN_RECV half opened
> connection still
> lingers. The SSL server continue to process these
> SYN_RECV half opened
> connections and soon even though the client is
> terminated, some half
> opened connection went int other ESTAB state.   At
> this point, the SSL
> server can no longer accept any new connections.
>
> This happens on all Redhat Linux platform that I
> tested with kernel
> 2.4.20, 2.4.18, and 2.6.x.   Solaris 2.8 does not
> have this problem.
>
> Below is the stack trace on where the code is
> blocking in.
>
> [Switching to Thread -1210186048 (LWP 21173)]
> 0x0012e7a2 in _dl_sysinfo_int80 () from
> /lib/ld-linux.so.2
> (gdb) where
> #0  0x0012e7a2 in _dl_sysinfo_int80 () from
> /lib/ld-linux.so.2
> #1  0x00bd33fb in __read_nocancel () from
> /lib/tls/libpthread.so.0
> #2  0x00608d24 in BIO_sock_should_retry () from
> /lib/libcrypto.so.4
> #3  0x00606d8c in BIO_read () from
> /lib/libcrypto.so.4
> #4  0x007b21b0 in ssl23_read_bytes () from
> /lib/libssl.so.4
> #5  0x007b0d31 in ssl23_get_client_hello () from
> /lib/libssl.so.4
> #6  0x007b1309 in ssl23_accept () from
> /lib/libssl.so.4
> #7  0x007b60ab in SSL_accept () from
> /lib/libssl.so.4
> #8  0xb7e093bc in ACE_SSL_SOCK_Acceptor::ssl_accept
> (this=0xbffff6f0,
>     new_stream=@0x59040198, timeout=0x0) at
> SSL_SOCK_Acceptor.cpp:74
> #9  0xb7e097d5 in ACE_SSL_SOCK_Acceptor::accept
> (this=0xbffff6f0,
>     new_stream=@0x59040198, remote_addr=0x0,
> timeout=0x0, restart=1,
>     reset_new_handle=0) at SSL_SOCK_Acceptor.cpp:195
> #10 0x0804aace in main ()
>
> Does anyone has any idea on what is going on?
>
> Thanks in advance.
>
> -Henry
>
>

__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around
http://mail.yahoo.com 

[Gayathri Sundar-2]

Have you set the the underlying FD to nonblocking (FIONBIO?)..I dont see why if the socket non blocking flag is set, you should have this problem..it doesnt make sense..

 

Thanks

--G3

-----Original Message-----
From: [hidden email] [mailto:[hidden email]]On Behalf Of Henry Yip (hyip)
Sent: Friday, March 10, 2006 10:28 PM
To: [hidden email]
Cc: Henry Yip (hyip)
Subject: SSL_accept blocks forever

Hi,

I have a SSL server program that accepts SSL connections. But after receiving around 150 connections confirmed with the "netstat -a  | grep ESTAB" command and many SYN_RECV half opened connections, the client is terminated. This forces the sockets to close.  The number of ESTAB connections goes to zero.  The SYN_RECV half opened connection still lingers. The SSL server continue to process these SYN_RECV half opened connections and soon even though the client is terminated, some half opened connection went int other ESTAB state.   At this point, the SSL server can no longer accept any new connections.

This happens on all Redhat Linux platform that I tested with kernel 2.4.20, 2.4.18, and 2.6.x.   Solaris 2.8 does not have this problem.

Below is the stack trace on where the code is blocking in.

[Switching to Thread -1210186048 (LWP 21173)]
0x0012e7a2 in _dl_sysinfo_int80 () from /lib/ld-linux.so.2
(gdb) where
#0  0x0012e7a2 in _dl_sysinfo_int80 () from /lib/ld-linux.so.2
#1  0x00bd33fb in __read_nocancel () from /lib/tls/libpthread.so.0
#2  0x00608d24 in BIO_sock_should_retry () from /lib/libcrypto.so.4
#3  0x00606d8c in BIO_read () from /lib/libcrypto.so.4
#4  0x007b21b0 in ssl23_read_bytes () from /lib/libssl.so.4
#5  0x007b0d31 in ssl23_get_client_hello () from /lib/libssl.so.4
#6  0x007b1309 in ssl23_accept () from /lib/libssl.so.4
#7  0x007b60ab in SSL_accept () from /lib/libssl.so.4
#8  0xb7e093bc in ACE_SSL_SOCK_Acceptor::ssl_accept (this=0xbffff6f0,
    [hidden email], timeout=0x0) at SSL_SOCK_Acceptor.cpp:74
#9  0xb7e097d5 in ACE_SSL_SOCK_Acceptor::accept (this=0xbffff6f0,
    [hidden email], remote_addr=0x0, timeout=0x0, restart=1,
    reset_new_handle=0) at SSL_SOCK_Acceptor.cpp:195
#10 0x0804aace in main ()

Does anyone has any idea on what is going on?

Thanks in advance.

-Henry