Performance related queries for SSL based client server model

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

Performance related queries for SSL based client server model

alok sharma-2
Hi,
   I am writing one sample ssl based client server model which uses SSL_Read & SSL_Write API provided by openssl. But I found that my application is very slow it takes around 40 mins to copy 700MB file. While same file using scp finishes in 10 mins.
   So my query is that is there  an alternative way to use open ssl read or write to improve performance. I searched in scp code and found it does not use SSL_read/SSL_write. So if there is another set of APIs which I can use or any idea how I can meet the same performance as scp.
Regards,
Alok
Reply | Threaded
Open this post in threaded view
|

RE: Performance related queries for SSL based client server model

Dave Thompson-5

This is not a –dev question, and there’s no need to send three times.

 

scp uses the SSH protocol. OpenSSL does not implement SSH.

OpenSSH, which is a different product from a different source, implements

SSH, although in their design the scp program doesn’t do any comms at all,

it just pipes to the ssh program which does.

 

What kind of network(s) are you transiting, and what are your endpoints?

On my dev LAN, which is one uncongested reliable 100Mbps switch, I get

plain TCP at nearly the hardware limit 8sec per 100MB, and within 10% of

that for SCP/SSH or trivial-app/SSL. These do 700MB in barely a minute.

 

SSL and SSH differ significantly in connection setup/handshake, and slightly

in multiplexing the data, but once actually sending application data they use

mostly the same range of ciphers and MAC, with openssh actually calling

libcrypto, and use TCP pretty much the same way, so unless you’re doing or

(perhaps unintentionally) invoking something wrong, you should get roughly

the same speed for both.

 

Try netcat to measure only the network (and disk) with almost no CPU;

that gives you an upper bound on any protocol – except one that can and does

compress well: I believe openssh can and openssl definitely can depending

on how it’s built, but many people disable it post-CRIME, and it certainly

depends very much on your data. You might try gzip on your data and

if that makes much difference send the gzipped form.

 

 

From: [hidden email] [mailto:[hidden email]] On Behalf Of Alok Sharma
Sent: Sunday, September 07, 2014 03:30
To: [hidden email]; [hidden email]
Subject: Performance related queries for SSL based client server model

 

Hi,

   I am writing one sample ssl based client server model which uses SSL_Read & SSL_Write API provided by openssl. But I found that my application is very slow it takes around 40 mins to copy 700MB file. While same file using scp finishes in 10 mins.

   So my query is that is there  an alternative way to use open ssl read or write to improve performance. I searched in scp code and found it does not use SSL_read/SSL_write. So if there is another set of APIs which I can use or any idea how I can meet the same performance as scp.

Regards,
Alok

Reply | Threaded
Open this post in threaded view
|

Re: Performance related queries for SSL based client server model

Viktor Dukhovni
In reply to this post by alok sharma-2
On Sun, Sep 07, 2014 at 01:00:17PM +0530, Alok Sharma wrote:

>    I am writing one sample ssl based client server model which uses
> SSL_Read & SSL_Write API provided by openssl.

If you transfering each block of data as an RPC, with a round-trip
acknowledgement before sending the next block, and the blocks are
small enough, you're going to severely limit throughput.

In bulk data transfer applications that stream data, TLS typically
outperforms SSH, but a lot depends on the details.

--
        Viktor.
______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
User Support Mailing List                    [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: Performance related queries for SSL based client server model

alok sharma-2
Hi,
      I am trying with simple client server model where it is reading datat from a file and calling SSL_write and similar thing is happening on other end i.e. reading from SSL using SSL_read and writing to file. Itis taking 40 mins of time to copy file of size 600 MB and if I run a non-ssl client server (i.e. pure TCP socket application ) it finishes in 5-6 mins and also on same network scp is taking 10 mins. One thing I observerd by looking into scp code that it does not use SSL provided APIs (i.e.SSL_Read or SSL_Write) but they use it differenly i.e. might be directly calling encryption APIs and writing data to sockets. But I don't have much understanding what SSL_Write or SSL_read does internally . So wanted to understand if there is any way to improve performance of SSL_Read or SSL_write to achive high performance. Following are my client server programmes. Here  client writes file on server machine in hardcoded location and name.

Client
----------------------------

//SSL-Client.c
#include <stdio.h>
#include <errno.h>
#include <unistd.h>
#include <malloc.h>
#include <string.h>
#include <sys/socket.h>
#include <resolv.h>
#include <netdb.h>
#include <openssl/ssl.h>
#include <openssl/err.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <netinet/tcp.h>
#include <netinet/ip.h>

#define FAIL    -1

int OpenConnection(const char *hostname, int port)
{   int sd;
    struct hostent *host;
    struct sockaddr_in addr;

    if ( (host = gethostbyname(hostname)) == NULL )
    {
        perror(hostname);
        abort();
    }
    sd = socket(PF_INET, SOCK_STREAM, 0);
    bzero(&addr, sizeof(addr));
    addr.sin_family = AF_INET;
    addr.sin_port = htons(port);
    addr.sin_addr.s_addr = *(long*)(host->h_addr);
    if ( connect(sd, (struct sockaddr*)&addr, sizeof(addr)) != 0 )
    {
        close(sd);
        perror(hostname);
        abort();
    }
    return sd;
}

SSL_CTX* InitCTX(void)
{   SSL_METHOD *method;
    SSL_CTX *ctx;

    OpenSSL_add_all_algorithms();  /* Load cryptos, et.al. */
    SSL_load_error_strings();   /* Bring in and register error messages */
    method = SSLv2_client_method();  /* Create new client-method instance */
    ctx = SSL_CTX_new(method);   /* Create new context */
    if ( ctx == NULL )
    {
        ERR_print_errors_fp(stderr);
        abort();
    }
    return ctx;
}

void ShowCerts(SSL* ssl)
{   X509 *cert;
    char *line;

    cert = SSL_get_peer_certificate(ssl); /* get the server's certificate */
    if ( cert != NULL )
    {
        printf("Server certificates:\n");
        line = X509_NAME_oneline(X509_get_subject_name(cert), 0, 0);
        printf("Subject: %s\n", line);
        free(line);       /* free the malloc'ed string */
        line = X509_NAME_oneline(X509_get_issuer_name(cert), 0, 0);
        printf("Issuer: %s\n", line);
        free(line);       /* free the malloc'ed string */
        X509_free(cert);     /* free the malloc'ed certificate copy */
    }
    else
        printf("No certificates.\n");
}

int main(int count, char *strings[])
{   SSL_CTX *ctx;
    int server;
    SSL *ssl;
    char buf[1024];
    int bytes;
    char *hostname, *portnum;
    char recvBuff[16*1024];
    int fd=0;
    int mode,i,n,len;
    int total=0;
    if ( count != 4 )
    {
        printf("usage: %s <hostname> <portnum><FileName>\n", strings[0]);
        exit(0);
    }
    SSL_library_init();
    hostname=strings[1];
    portnum=strings[2];

    ctx = InitCTX();
    server = OpenConnection(hostname, atoi(portnum));
    ssl = SSL_new(ctx);      /* create new SSL connection state */
    SSL_set_fd(ssl, server);    /* attach the socket descriptor */
    if ( SSL_connect(ssl) == FAIL )   /* perform the connection */
        ERR_print_errors_fp(stderr);
    else
    {   char *msg = "Hello???";

        printf("Connected with %s encryption\n", SSL_get_cipher(ssl));
        ShowCerts(ssl);        /* get any certs */

    fd=open(strings[3],O_RDWR);

    while ( (n = read(fd, recvBuff, sizeof(recvBuff)-1)) > 0)
    {
        recvBuff[n] = 0;
        mode=n;
        i=0;
            while(mode>0)
            {
             len=SSL_write(ssl, recvBuff,mode );   /* encrypt & send message */
             total+=len;
             mode=mode-len;
            }
    }


        bytes = SSL_read(ssl, buf, sizeof(buf)); /* get reply & decrypt */
        buf[bytes] = 0;
        printf("Received: \"%s\"\n", buf);
        SSL_free(ssl);        /* release connection state */
    }
    close(server);         /* close socket */
    SSL_CTX_free(ctx);        /* release context */
    return 0;
}


----------------------------------------------------

Server Programme
-------------------------------------------------


//SSL-Server.c
#include <errno.h>
#include <unistd.h>
#include <malloc.h>
#include <string.h>
#include <arpa/inet.h>
#include <sys/socket.h>
#include <sys/types.h>
#include <netinet/in.h>
#include <resolv.h>
#include "openssl/ssl.h"
#include "openssl/err.h"
#include <sys/stat.h>
#include <fcntl.h>
#include <netinet/tcp.h>
#include <netinet/ip.h>

#define FAIL    -1

int OpenListener(int port)
{   int sd;
    struct sockaddr_in addr;

    sd = socket(PF_INET, SOCK_STREAM, 0);
    bzero(&addr, sizeof(addr));
    addr.sin_family = AF_INET;
    addr.sin_port = htons(port);
    addr.sin_addr.s_addr = INADDR_ANY;
    if ( bind(sd, (struct sockaddr*)&addr, sizeof(addr)) != 0 )
    {
        perror("can't bind port");
        abort();
    }
    if ( listen(sd, 10) != 0 )
    {
        perror("Can't configure listening port");
        abort();
    }
    return sd;
}

SSL_CTX* InitServerCTX(void)
{   SSL_METHOD *method;
    SSL_CTX *ctx;

    OpenSSL_add_all_algorithms();  /* load & register all cryptos, etc. */
    SSL_load_error_strings();   /* load all error messages */
    method = SSLv2_server_method();  /* create new server-method instance */
    ctx = SSL_CTX_new(method);   /* create new context from method */
    if ( ctx == NULL )
    {
        ERR_print_errors_fp(stderr);
        abort();
    }
    return ctx;
}

void LoadCertificates(SSL_CTX* ctx, char* CertFile, char* KeyFile)
{
 /* set the local certificate from CertFile */
    if ( SSL_CTX_use_certificate_file(ctx, CertFile, SSL_FILETYPE_PEM) <= 0 )
    {
        ERR_print_errors_fp(stderr);
        abort();
    }
    /* set the private key from KeyFile (may be the same as CertFile) */
    if ( SSL_CTX_use_PrivateKey_file(ctx, KeyFile, SSL_FILETYPE_PEM) <= 0 )
    {
        ERR_print_errors_fp(stderr);
        abort();
    }
    /* verify private key */
    if ( !SSL_CTX_check_private_key(ctx) )
    {
        fprintf(stderr, "Private key does not match the public certificate\n");
        abort();
    }
}

void ShowCerts(SSL* ssl)
{   X509 *cert;
    char *line;

    cert = SSL_get_peer_certificate(ssl); /* Get certificates (if available) */
    if ( cert != NULL )
    {
        printf("Server certificates:\n");
        line = X509_NAME_oneline(X509_get_subject_name(cert), 0, 0);
        printf("Subject: %s\n", line);
        free(line);
        line = X509_NAME_oneline(X509_get_issuer_name(cert), 0, 0);
        printf("Issuer: %s\n", line);
        free(line);
        X509_free(cert);
    }
    else
        printf("No certificates.\n");
}

void Servlet(SSL* ssl) /* Serve the connection -- threadable */
{   char buf[1024*16];
    char reply[1024];
    int sd, bytes;
    int fd=0;
     int mode,len,len1;
    const char* HTMLecho="<html><body><pre>%s</pre></body></html>\n\n";

    if ( SSL_accept(ssl) == FAIL )     /* do SSL-protocol accept */
        ERR_print_errors_fp(stderr);
    else
    {
        ShowCerts(ssl);        /* get any certificates */
         fd=open("/tmp/test1",O_WRONLY | O_CREAT, 0644);
        while(1)
        {
        bytes = SSL_read(ssl, buf, sizeof(buf)); /* get request */
        if ( bytes > 0 )
        {
           // printf("Client msg: \"%s\"\n", buf);
            //sprintf(reply, HTMLecho, buf);   /* construct reply */
            //SSL_write(ssl, reply, strlen(reply)); /* send reply */
        len=bytes;
        mode=len;
        while(mode>0)
        {
                  len1=write(fd,buf,len);
                  mode=mode-len1;
                  len-=len1;
        }

        }
        else {
            ERR_print_errors_fp(stderr);
            break;
        }

      }
    }
    sd = SSL_get_fd(ssl);       /* get socket connection */
    SSL_free(ssl);         /* release SSL state */
    close(sd);          /* close connection */
}

int main(int count, char *strings[])
{   SSL_CTX *ctx;
    int server;
    char *portnum;

    if ( count != 2 )
    {
        printf("Usage: %s <portnum>\n", strings[0]);
        exit(0);
    }
    SSL_library_init();

    portnum = strings[1];
   ctx = InitServerCTX();        /* initialize SSL */
    LoadCertificates(ctx, "mycert.pem", "mycert.pem"); /* load certs */
    server = OpenListener(atoi(portnum));    /* create server socket */
    while (1)
    {   struct sockaddr_in addr;
        socklen_t len = sizeof(addr);
        SSL *ssl;

        int client = accept(server, (struct sockaddr*)&addr, &len);  /* accept connection as usual */
        printf("Connection: %s:%d\n",inet_ntoa(addr.sin_addr), ntohs(addr.sin_port));
        ssl = SSL_new(ctx);              /* get new SSL state with context */
        SSL_set_fd(ssl, client);      /* set connection socket to SSL state */
        Servlet(ssl);         /* service connection */
    }
    close(server);          /* close server socket */
    SSL_CTX_free(ctx);         /* release context */
}






On Sun, Sep 7, 2014 at 8:11 PM, Viktor Dukhovni <[hidden email]> wrote:
On Sun, Sep 07, 2014 at 01:00:17PM +0530, Alok Sharma wrote:

>    I am writing one sample ssl based client server model which uses
> SSL_Read & SSL_Write API provided by openssl.

If you transfering each block of data as an RPC, with a round-trip
acknowledgement before sending the next block, and the blocks are
small enough, you're going to severely limit throughput.

In bulk data transfer applications that stream data, TLS typically
outperforms SSH, but a lot depends on the details.

--
        Viktor.
______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
User Support Mailing List                    [hidden email]
Automated List Manager                           [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Performance related queries for SSL based client server model

Viktor Dukhovni
On Mon, Sep 08, 2014 at 11:16:46PM +0530, Alok Sharma wrote:

>     while ( (n = read(fd, recvBuff, sizeof(recvBuff)-1)) > 0)
>     {
>         recvBuff[n] = 0;
>         mode=n;
>         i=0;
>             while(mode>0)
>             {
>              len=SSL_write(ssl, recvBuff, mode);
>              total+=len;
>              mode=mode-len;
>             }
>     }

The inner loop is wrong unless all the data is always written on
each SSL_write() call.

>         bytes = SSL_read(ssl, buf, sizeof(buf)); /* get reply & decrypt */
>         buf[bytes] = 0;
>         printf("Received: \"%s\"\n", buf);
>         SSL_free(ssl);        /* release connection state */

Why do you expect data back from the server?  It is surely still
waiting for more data from the client, since there is no explicit
message framing to indicate to the server that all the data is
sent, and it should reply.  You're freeing the SSL state without
a gracefull SSL_shutdown() (often called twice, see the docs).

>         while(1)
>         {
>         bytes = SSL_read(ssl, buf, sizeof(buf)); /* get request */
>         if ( bytes > 0 )
>         {
>            // printf("Client msg: \"%s\"\n", buf);
>             //sprintf(reply, HTMLecho, buf);   /* construct reply */
>             //SSL_write(ssl, reply, strlen(reply)); /* send reply */
>         len=bytes;
>         mode=len;
>         while(mode>0)
>         {
>                   len1=write(fd,buf,len);
>                   mode=mode-len1;
>                   len-=len1;
>         }
>
>         }
>         else {
>             ERR_print_errors_fp(stderr);
>             break;
>         }
>
>       }

THe inner loop is wrong unless all the data is written on each
write(2) call.  The server is in an infinite read loop, deadlocked
with the client.

>     }
>     sd = SSL_get_fd(ssl);       /* get socket connection */
>     SSL_free(ssl);         /* release SSL state */
>     close(sd);          /* close connection */

And sends no reply.  This code is broken, and should block forever
with SSL_read()/SSL_write() replaced with read()/write().  Since
you're reporting finite completion times, you're not posting the
code you're testing, which wastes everyone's time.

--
        Viktor.
______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
User Support Mailing List                    [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: Performance related queries for SSL based client server model

Iñaki Baz Castillo
In reply to this post by alok sharma-2
2014-09-08 19:46 GMT+02:00 Alok Sharma <[hidden email]>:
> One thing I observerd by looking into scp
> code that it does not use SSL provided APIs (i.e.SSL_Read or SSL_Write) but
> they use it differenly i.e. might be directly calling encryption APIs and
> writing data to sockets. But I don't have much understanding what SSL_Write
> or SSL_read does internally.

It has been already replied above. SSH is not SSL so don't look for
SSL_xxxx methods on openssh. Said that, AFAIK openssh uses the crypto
library from openssl, but that is not SSL/TLS at all.


> So wanted to understand if there is any way to
> improve performance of SSL_Read or SSL_write to achive high performance.
> Following are my client server programmes. Here  client writes file on
> server machine in hardcoded location and name.

You have lot of errors in your program. I suggest that you first
properly learn openssl, then measure your code if you need.



--
Iñaki Baz Castillo
<[hidden email]>
______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
User Support Mailing List                    [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: Performance related queries for SSL based client server model

alok sharma-2
Hi,
   Thanks all for your update. But functionality wise it is working
fine. I can remove the inner loop but that will require packet size to
be of 1K. I tried with that also but did not find any improvement in
performance. In my setup there are 8 routers between source &
destination. Can anyone share me sample SSL based client server
programme which I could use to measure performance.
Regards,
Alok

On Tue, Sep 9, 2014 at 3:54 AM, Iñaki Baz Castillo <[hidden email]> wrote:

>
> 2014-09-08 19:46 GMT+02:00 Alok Sharma <[hidden email]>:
> > One thing I observerd by looking into scp
> > code that it does not use SSL provided APIs (i.e.SSL_Read or SSL_Write) but
> > they use it differenly i.e. might be directly calling encryption APIs and
> > writing data to sockets. But I don't have much understanding what SSL_Write
> > or SSL_read does internally.
>
> It has been already replied above. SSH is not SSL so don't look for
> SSL_xxxx methods on openssh. Said that, AFAIK openssh uses the crypto
> library from openssl, but that is not SSL/TLS at all.
>
>
> > So wanted to understand if there is any way to
> > improve performance of SSL_Read or SSL_write to achive high performance.
> > Following are my client server programmes. Here  client writes file on
> > server machine in hardcoded location and name.
>
> You have lot of errors in your program. I suggest that you first
> properly learn openssl, then measure your code if you need.
>
>
>
> --
> Iñaki Baz Castillo
> <[hidden email]>
> ______________________________________________________________________
> OpenSSL Project                                 http://www.openssl.org
> User Support Mailing List                    [hidden email]
> Automated List Manager                           [hidden email]
______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
User Support Mailing List                    [hidden email]
Automated List Manager                           [hidden email]