Replacement of functions that operate with sockets

classic Classic list List threaded Threaded
12 messages Options
Reply | Threaded
Open this post in threaded view
|

Replacement of functions that operate with sockets

Vladimir Belov
Hello.
By default OpenSSL itself works with sockets. I would want to implement operation with sockets without admitting it to OpenSSL. I.e. for example, when OpenSSL wants to write down something in a socket, it should cause my function and I will transfer data to the socket. And it is exact also obtaining the data from a socket I cause a function of OpenSSL, transferring to it the data accepted from a socket.
I.e. I implement function WriteSocket. When OpenSSL wants to write something in a socket, it causes WriteSocket and nothing more it should disturb. When the data from a socket has come I send notification message OpenSSL about that the data has come, actually I cause function of OpenSSL ReadSocket, transferring it the buffer with the come data. I.e. any operations of waiting of reading OpenSSL is not necessary to do, it has sent the data and can easy smoke a bamboo, expecting when I will cause ReadSocket.
 
Whether is it possible to implement? I have read in the documentation about BIO-functions, and could not understand  is it possible to implement or not.
Thanks for any help or suggestions.
 
Vladimir.
Reply | Threaded
Open this post in threaded view
|

Re: Replacement of functions that operate with sockets

Wim Lewis-3

On 15 Jun 2011, at 11:57 AM, <[hidden email]> wrote:
> Whether is it possible to implement? I have read in the documentation about BIO-functions, and could not understand  is it possible to implement or not.
> Thanks for any help or suggestions.

Yes, this is possible. It is asked on this mailing list every now and then, I guess there should be better documentation or an example of using OpenSSL in this way.


______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
User Support Mailing List                    [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: Replacement of functions that operate with sockets

Michael S. Zick-4
On Wed June 15 2011, Wim Lewis wrote:
>
> On 15 Jun 2011, at 11:57 AM, <[hidden email]> wrote:
> > Whether is it possible to implement? I have read in the documentation about BIO-functions, and could not understand  is it possible to implement or not.
> > Thanks for any help or suggestions.
>
> Yes, this is possible. It is asked on this mailing list every now and then, I guess there should be better documentation or an example of using OpenSSL in this way.
>

Yes - search for "BIO pairs"

A recent exchange:
http://old.nabble.com/SSL-Communication-using-BIO-td31678026.html

A bit old but may be helpful:
http://www.ibm.com/developerworks/linux/library/l-openssl/index.html

Mike
>
> ______________________________________________________________________
> OpenSSL Project                                 http://www.openssl.org
> User Support Mailing List                    [hidden email]
> Automated List Manager                           [hidden email]
>
>


______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
User Support Mailing List                    [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: Replacement of functions that operate with sockets

JoelKatz
In reply to this post by Vladimir Belov
On 6/15/2011 11:57 AM, [hidden email] wrote:

> Hello.
> By default OpenSSL itself works with sockets. I would want to implement
> operation with sockets without admitting it to OpenSSL. I.e. for
> example, when OpenSSL wants to write down something in a socket, it
> should cause my function and I will transfer data to the socket. And it
> is exact also obtaining the data from a socket I cause a function of
> OpenSSL, transferring to it the data accepted from a socket.
> I.e. I implement function WriteSocket. When OpenSSL wants to write
> something in a socket, it causes WriteSocket and nothing more it should
> disturb.

Well that wouldn't work as stated. How would OpenSSL know when it was
time to call WriteSocket? You will have to call into OpenSSL when you
want to see if has any data it needs to write to the socket.

In fact, you will have to manage *four* I/O streams to and from OpenSSL.
When you receive encrypted data from the socket, you will have to hand
it to OpenSSL. When you know it is safe to write to the socket, you will
need to check if OpenSSL has any encrypted data to send and if so, read
it from OpenSSL and send it to the other side. When anything changes,
you will also need to check if OpenSSL has any decrypted plaintext to
deliver to your application. And you will have to pass any plaintext
your application wish to send to OpenSSL.

 > When the data from a socket has come I send notification
> message OpenSSL about that the data has come, actually I cause function
> of OpenSSL ReadSocket, transferring it the buffer with the come data.
> I.e. any operations of waiting of reading OpenSSL is not necessary to
> do, it has sent the data and can easy smoke a bamboo, expecting when I
> will cause ReadSocket.
> Whether is it possible to implement? I have read in the documentation
> about BIO-functions, and could not understand is it possible to
> implement or not.

Look at the example code that uses BIO pairs.

DS

______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
User Support Mailing List                    [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: Replacement of functions that operate with sockets

Jim Segrave-3
On Mon 20 Jun 2011 (11:00 -0700), David Schwartz wrote:

> On 6/15/2011 11:57 AM, [hidden email] wrote:
>> Hello.
>> By default OpenSSL itself works with sockets. I would want to implement
>> operation with sockets without admitting it to OpenSSL. I.e. for
>> example, when OpenSSL wants to write down something in a socket, it
>> should cause my function and I will transfer data to the socket. And it
>> is exact also obtaining the data from a socket I cause a function of
>> OpenSSL, transferring to it the data accepted from a socket.
>> I.e. I implement function WriteSocket. When OpenSSL wants to write
>> something in a socket, it causes WriteSocket and nothing more it should
>> disturb.
>
> Well that wouldn't work as stated. How would OpenSSL know when it was time
> to call WriteSocket? You will have to call into OpenSSL when you want to
> see if has any data it needs to write to the socket.
>
> In fact, you will have to manage *four* I/O streams to and from OpenSSL.
> When you receive encrypted data from the socket, you will have to hand it
> to OpenSSL. When you know it is safe to write to the socket, you will need
> to check if OpenSSL has any encrypted data to send and if so, read it from
> OpenSSL and send it to the other side. When anything changes, you will also
> need to check if OpenSSL has any decrypted plaintext to deliver to your
> application. And you will have to pass any plaintext your application wish
> to send to OpenSSL.

Why not simply create a second socket - your app reads from that, does
whatever is needed and writes to the openssl socket. When openssl
writes to its socket, you read it, do whatever, then write it out your
socket?

--
Jim Segrave           [hidden email]


______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
User Support Mailing List                    [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: Replacement of functions that operate with sockets

Vladimir Belov
In reply to this post by JoelKatz
David Schwartz wrote:
> Well that wouldn't work as stated. How would OpenSSL know when it was time
> to call WriteSocket? You will have to call into OpenSSL when you want to
> see if has any data it needs to write to the socket.

> In fact, you will have to manage *four* I/O streams to and from OpenSSL.
> When you receive encrypted data from the socket, you will have to hand it
> to OpenSSL. When you know it is safe to write to the socket, you will need
> to check if OpenSSL has any encrypted data to send and if so, read it from
> OpenSSL and send it to the other side. When anything changes, you will
> also need to check if OpenSSL has any decrypted plaintext to deliver to
> your application. And you will have to pass any plaintext your application
> wish to send to OpenSSL.


Ok. I'll try to explain my idea in details. Generally we'll need 4 main and
3 additional functions. One of 4 main functions is callback function that is
calling by OpenSSL library itself. There are two types of "socket IO". The
first is that OpenSSL Library making by itself and programmer don't know
about this IO, and must not to know, it's like a blackbox - SSL-TLS protocol
specific IO. For this IO we need 2 functions SysWriteToSocket and
SSL_ReadArrivedFromSocketData. SysWriteSocket is implemented by programmer,
it is an asynchronous function and may be defined so:

    int SysWriteToSocket(char *buf,int buf_len);

and incorporates with OpenSSL by means of the following function:

    typedef int (*FSysWriteToSocket)(char *,int);
    void SSL_SetSysWriteToSocketFunction(FSysWriteToSocket fun);

When OpenSSL want to write some data to the socket it calls asynchronous
SysWriteToSocket function.

When I(programmer) receive any data from socket I must call
SSL_ReadArrivedFromSocketData and OpenSSL can process the arrived data. It
may be defined so:

    int SSL_ReadArrivedFromSocketData(char *buf, int buf_len)

One of the returning values of this function may be the flag that indicates
about existing decrypted application data that could be read.
After that and also at any time I can call SSL_GetApplicationData to read
application data or determine if any application data is available for
reading:

    int SSL_GetApplicationDataSize();
    int SSL_GetApplicationData(char *buf, int buf_len,int *need_buf_len);

The result of these functions is number of bytes of available and written
application data accordingly, if success.
[need_buf_len] - the necessary size of the buffer if buf_len is not enough
to contain all data

The fourth function is SSL_EncryptUserData, which encrypt our own
application data before we can send their to secure channel:

    int SSL_EncryptApplicationData(char *buf_in, int buf_in_len, char
buf_out, int buf_out_len, int *need_buf_out_len);

The result of this function is number of bytes written to the buf_out
buffer, if success.
[need_buf_out_len] - the necessary size of the output buffer if buf_out_len
is not enough to contain all data

When I(programmer) need to send any data to the secure socket I am calling
SSL_EncryptUserData and after this I send encrypted data from buf_out to the
socket.

Also we need to define one status function SSL_GetCurrentSocketIOState:

    int SSL_GetCurrentIOState();

This function would return such values as:
SSL_SYS_DATA_WRITING
SSL_SYS_WAITING_DATA_READING
SSL_SYS_DATA_READING
SSL_SYS_OPERATION_TIMEOUT
SSL_APPLICATION_DATA_READING
SSL_READY_TO_TRANSFER_APPLICATION_DATA
... and so on, all what we need.


These functions let us manipulate with sockets ourself, as we want.


David Schwartz wrote:
> Look at the example code that uses BIO pairs.

Please say where can I find this example code? What is the name of c-file?



______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
User Support Mailing List                    [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: Replacement of functions that operate with sockets

Vladimir Belov
In reply to this post by Jim Segrave-3
Jim Segrave wrote:

> Why not simply create a second socket - your app reads from that, does
> whatever is needed and writes to the openssl socket. When openssl
> writes to its socket, you read it, do whatever, then write it out your
> socket?

Jim, for me  the main goal to replace functions that operate with sockets is
performance. I want to use OpenSSL with Windows IO Completion ports. The
method that you suggest is very interesting but the main is not achieved -
OpenSSL is still writing to the socket. Besides we got so-called "double
buffering" and also more memory usage because of 2 sockets.

______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
User Support Mailing List                    [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: Replacement of functions that operate with sockets

Vladimir Belov
In reply to this post by Vladimir Belov
Some corrections of SSL_EncryptApplicationData function. By SSL/TLS protocol data is send in packets not more than defined value(as i know 16Kb). That's why we must define this function differently and add one function for freeing buffers:
 
    typedef struct TBUFS
    {
     char *buf;
     int buf_len;
    };
    int SSL_EncryptApplicationData(char *buf_in, int buf_in_len, TBUFS *bufs_out, int bufs_out_count);
    void SSL_FreeEncryptBuffes(TBUFS *buf, int bufs_out_count);
 
So this function must allocate memory for output buffers and fill them with encrypted data. After that I must only send these buffers over net through the socket.
Reply | Threaded
Open this post in threaded view
|

Re: Replacement of functions that operate with sockets

JoelKatz
In reply to this post by Vladimir Belov
On 6/21/2011 2:40 AM, [hidden email] wrote:

> The fourth function is SSL_EncryptUserData, which encrypt our own
> application data before we can send their to secure channel:
>
> int SSL_EncryptApplicationData(char *buf_in, int buf_in_len, char
> buf_out, int buf_out_len, int *need_buf_out_len);
>
> The result of this function is number of bytes written to the buf_out
> buffer, if success.
> [need_buf_out_len] - the necessary size of the output buffer if
> buf_out_len is not enough to contain all data
>
> When I(programmer) need to send any data to the secure socket I am
> calling SSL_EncryptUserData and after this I send encrypted data from
> buf_out to the socket.

No, that can't possibly work. Any mechanism involving trying to "look
through" the SSL state machine is doomed to fail. Completely erase from
your mind any notion that you can map particular bits of encrypted data
to particular bits of decrypted data or vice versa.

The SSL engine is a black box with four hooks. What goes on inside it
is, as far as your application should be concerned, unimportant.

DS

______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
User Support Mailing List                    [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: Replacement of functions that operate with sockets

JoelKatz
In reply to this post by Vladimir Belov
On 6/21/2011 2:53 AM, [hidden email] wrote:

> Jim, for me the main goal to replace functions that operate with sockets
> is performance. I want to use OpenSSL with Windows IO Completion ports.
> The method that you suggest is very interesting but the main is not
> achieved - OpenSSL is still writing to the socket. Besides we got
> so-called "double buffering" and also more memory usage because of 2
> sockets.

I do exactly this using BIO pairs. I manage all four data streams. When
the application wants to send data to the other side, I hand it to
OpenSSL. When I receive data on the socket, I hand it to OpenSSL. When I
can send data on the socket, I get it from OpenSSL and send it to the
socket. When OpenSSL has decrypted data, I get it from OpenSSL and send
it to the upper application layers.

Just remember that you have four I/O streams you have to handle --
encrypted in, encrypted out, plaintext in, plaintext out. Make no
attempt to 'associate' these streams. Treat them as completely logically
independent.

DS

______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
User Support Mailing List                    [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: Replacement of functions that operate with sockets

Vladimir Belov
David Schwartz(Wednesday, June 22, 2011 12:04 PM) :
> I do exactly this using BIO pairs. I manage all four data streams. When
> the application wants to send data to the other side, I hand it to
> OpenSSL. When I receive data on the socket, I hand it to OpenSSL. When I
> can send data on the socket, I get it from OpenSSL and send it to the
> socket. When OpenSSL has decrypted data, I get it from OpenSSL and send it
> to the upper application layers.

> Just remember that you have four I/O streams you have to handle --
> encrypted in, encrypted out, plaintext in, plaintext out. Make no attempt
> to 'associate' these streams. Treat them as completely logically
> independent.

Where can I find this example with BIO pairs? I can't understand only with
openssl's documentation how to work with BIO pairs.

I will be grateful for the help.

______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
User Support Mailing List                    [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: Replacement of functions that operate with sockets

JoelKatz
On 6/22/2011 3:20 AM, [hidden email] wrote:

> Where can I find this example with BIO pairs? I can't understand only
> with openssl's documentation how to work with BIO pairs.
>
> I will be grateful for the help.

Look in ssl/ssltest.c, in the doit_biopair function.

DS

______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
User Support Mailing List                    [hidden email]
Automated List Manager                           [hidden email]