I am looking for advice for designing the transmit/receive network module of my application using openSSL.
I originally designed the application using two threads, a transmit thread and a receive thread. These threads relied on a single BIO that was set for blocking more that would connect to the server and remain connected as long as possible. If a disconnection was detected, it would reconnect. However, this design has some drawbacks. From what I've read, despite having implemented the locking mechanism for the library, I now realize that a single BIO within an application is not thread safe. This may be an underlying cause of communication failures that crops up after long periods of run-time. It's hard to predict when a failure will occur, but it seems as though neither thread makes progress when it happens.
My receiver thread was responsible for keeping the connection and reading data from the BIO. The transmitter would continue to try and transmit data received in a posix mqueue when the socket while connected.
Is there a better pattern to use?
From what I have read, I could switch to a single thread and create an underlying non-blocking socket that is bound to an SSL. I could also read transmit data from the existing queue, but set it for non-blocking.
However, the details of how the library works in non-blocking more are still a bit fuzzy.
When WANT_READ is returned, does this mean one should select on write and perform whichever operation (read or write) caused the error? If WANT_WRITE is returned, should I select on read and perform the previous operation?
Can someone please outline clearly what steps to take for each return value for both read/write operations. Perhaps there is an opportunity to improve the OpenSSL documentation since I see questions like this in the mailing list frequently.