Server not accepting connections after too many client connections
I am running the "UDP: DTLS Echo Server and Client" from http://sctp.fh-muenster.de/dtls-samples.html The only difference is my code uses pre-shared key.
The issue I am facing is after the server accepting couple of hundred connections (to few hundred depending on timeouts set) the server stops accepting anymore connects (which I think is fine) but it continues to be in that state even if all the old connections get closed. The client prints out messages "SSL_connect: Resource temporarily unavailable".
We plan to use DTLS in our code so I was trying to figure out how scalable or what could be the upper bound on number of active connections? To test this I am forking 2000 client processes where each process attempt (almost at the same time) to make connection to the server.
Has anyone attempted to find out the upper limit for maximum open/active DTLS connections?
I was not able to find if this is a known issue. Is anyone aware of it?
---- I tried to debug this issue a bit to see what exactly was happening.
It seems to me the server gets into an infinite loop:
again: i = dtls1_get_message_fragment(s, st1, stn, max, ok); if ( i == DTLS1_HM_BAD_FRAGMENT || i == DTLS1_HM_FRAGMENT_RETRY) /* bad fragment received */ goto again;
The function dtls1_get_message_fragment( ) always returns DTLS1_HM_FRAGMENT_RETRY. The reason I could see was "s->d1->handshake_read_seq" does not match the incoming message's seq number i.e. "msg_hdr.seq". This happens when the server sends "Hello Verify Request" to one of the clients that is attempting to connect; after that the server starts expect the next message's seq number (msg_hdr.seq) to be 1. In my scenario either the server was too late to respond back with the HelloVerify or the client just died. But the servers is still expecting the response to its HelloVerify from that particular client and keeps discarding any further incoming handshake messages in the above loop.