The smallest minimal example of an HTTPS GET request with openssl

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

The smallest minimal example of an HTTPS GET request with openssl

Ivan Medoedov
Hello,

I've only managed to find this, but it seems to do too much for what I need:


Basically I need something like Go's http.Get("https://mysite.com/version") to just get a one line of text from the server.

I can't use libcurl, I have to use pure openssl.

Thanks


Reply | Threaded
Open this post in threaded view
|

Re: The smallest minimal example of an HTTPS GET request with openssl

Viktor Dukhovni
On Sat, Mar 30, 2019 at 04:02:55PM +0100, Ivan Medoedov wrote:

> I've only managed to find this, but it seems to do too much for what I need:
>
> https://wiki.openssl.org/index.php/SSL/TLS_Client
>
> Basically I need something like Go's http.Get("https://mysite.com/version")
> to just get a one line of text from the server.
>
> I can't use libcurl, I have to use pure openssl.

OpenSSL is NOT an HTTPS (client or server) library.  OpenSSL can
establish a TLS connection, but does not implement anything like
"http.Get".  That's what libcurl and the like are for.

If you're willing to roll your own HTTP client code, then you
can use OpenSSL for the TLS part of the HTTPS transaction.

HTTP clients handle all sorts of complications over and above TLS:

    * Proxies
    * Redirects
    * Client requests with bodies
    * Authentication
    * Chunked transfer encoding
    * Content encoding (e.g. gzip)
    * Cookies
    ...

None of these are in scope for OpenSSL.

--
        Viktor.
Reply | Threaded
Open this post in threaded view
|

Re: The smallest minimal example of an HTTPS GET request with openssl

Ivan Medoedov
You are right of course. I handle HTTP myself. A TLS connection example will suffice.

On Sat, Mar 30, 2019 at 7:53 PM Viktor Dukhovni <[hidden email]> wrote:
On Sat, Mar 30, 2019 at 04:02:55PM +0100, Ivan Medoedov wrote:

> I've only managed to find this, but it seems to do too much for what I need:
>
> https://wiki.openssl.org/index.php/SSL/TLS_Client
>
> Basically I need something like Go's http.Get("https://mysite.com/version")
> to just get a one line of text from the server.
>
> I can't use libcurl, I have to use pure openssl.

OpenSSL is NOT an HTTPS (client or server) library.  OpenSSL can
establish a TLS connection, but does not implement anything like
"http.Get".  That's what libcurl and the like are for.

If you're willing to roll your own HTTP client code, then you
can use OpenSSL for the TLS part of the HTTPS transaction.

HTTP clients handle all sorts of complications over and above TLS:

    * Proxies
    * Redirects
    * Client requests with bodies
    * Authentication
    * Chunked transfer encoding
    * Content encoding (e.g. gzip)
    * Cookies
    ...

None of these are in scope for OpenSSL.

--
        Viktor.
Reply | Threaded
Open this post in threaded view
|

Re: The smallest minimal example of an HTTPS GET request with openssl

Viktor Dukhovni
On Sat, Mar 30, 2019 at 08:09:01PM +0100, Ivan Medoedov wrote:

> You are right of course. I handle HTTP myself. A TLS connection example
> will suffice.
>
> > > https://wiki.openssl.org/index.php/SSL/TLS_Client

That example can be simplified, but OpenSSL does not presently
provide a built-in high-level function to make a verified connection
to a given host:port.

With library initialization now implicit in 1.1.0 and later, and
thread-safety no longer requiring application callbacks, this would
now be easier to provide, but:

    * SSL_CTX creation is moderately expensive, and one should
      generally create and use the same SSL_CTX for multiple SSL
      connecitons in any applications that uses more than one SSL
      connection.  So typically, there would still be a one-time
      initialization step to create the SSL_CTX (application context).

    * Some applications will want to do post-handshake I/O via the
      socket API, and prefer to hand OpenSSL an already connected
      socket on which to perform the TLS handshake.  Others may
      want OpenSSL to establish the connection and may prefer to
      use BIO interface to interact with the peer.

    * While many systems have a usable default "trust store",
      some do not, or the default "trust store" is overly inclusive.
      Applications should generally allow the user to specify the
      set of trusted CAs.

    ...

Through in enough similar qualifiers, and you end up with the rather
complex example.  It could perhaps be refactored as (hypothetical
interface):

        /*
         * Context with default trust store, protocols, ciphers, ...
         * and peer verification enabled.
         */
        SSL_CTX *app_ctx = SSL_default_ssl_ctx(TLS_method());

        /*
         * Prepare verified (with name checks) SSL handle for the
         * requested host:port (with SNI).
         */
        SSL *con = SSL_default_ssl(app_ctx, host, port);

Then depending on whether you want to give OpenSSL an already
connected socket, or have OpenSSL make the connection for you:

        int err = SSL_connect_socket(con, fd);

or (something along the lines of):

        BIO *bp = SSL_connect_hostport(con, host, port, &err);

At this point a stock error handler may be required, but if all
went well, you now need an I/O loop.  And so would need to either
use the connected file descriptor with your own buffering, ...
or use OpenSSL BIOs, or some other buffering I/O layer that
supports custom read/write wrappers.

On FreeBSD/NetBSD it might be nice to have stock "fittings" to
wrap stdio around OpenSSL connections via:

     #include <stdio.h>

     FILE *
     funopen(const void *cookie,
             int (*readfn)  (void *, char *, int),
             int (*writefn) (void *, const char *, int),
             fpos_t (*seekfn)  (void *, fpos_t, int),
             int (*closefn) (void *));

Providing higher level interfaces to the core TLS library would
make a good project for someone good at API design, who is familiar
with OpenSSL, but more interested in the API than the internals.

--
        Viktor.
Reply | Threaded
Open this post in threaded view
|

Re: The smallest minimal example of an HTTPS GET request with openssl

Ivan Medoedov
Thanks, Viktor.

On Sat, Mar 30, 2019 at 8:59 PM Viktor Dukhovni <[hidden email]> wrote:
On Sat, Mar 30, 2019 at 08:09:01PM +0100, Ivan Medoedov wrote:

> You are right of course. I handle HTTP myself. A TLS connection example
> will suffice.
>
> > > https://wiki.openssl.org/index.php/SSL/TLS_Client

That example can be simplified, but OpenSSL does not presently
provide a built-in high-level function to make a verified connection
to a given host:port.

With library initialization now implicit in 1.1.0 and later, and
thread-safety no longer requiring application callbacks, this would
now be easier to provide, but:

    * SSL_CTX creation is moderately expensive, and one should
      generally create and use the same SSL_CTX for multiple SSL
      connecitons in any applications that uses more than one SSL
      connection.  So typically, there would still be a one-time
      initialization step to create the SSL_CTX (application context).

    * Some applications will want to do post-handshake I/O via the
      socket API, and prefer to hand OpenSSL an already connected
      socket on which to perform the TLS handshake.  Others may
      want OpenSSL to establish the connection and may prefer to
      use BIO interface to interact with the peer.

    * While many systems have a usable default "trust store",
      some do not, or the default "trust store" is overly inclusive.
      Applications should generally allow the user to specify the
      set of trusted CAs.

    ...

Through in enough similar qualifiers, and you end up with the rather
complex example.  It could perhaps be refactored as (hypothetical
interface):

        /*
         * Context with default trust store, protocols, ciphers, ...
         * and peer verification enabled.
         */
        SSL_CTX *app_ctx = SSL_default_ssl_ctx(TLS_method());

        /*
         * Prepare verified (with name checks) SSL handle for the
         * requested host:port (with SNI).
         */
        SSL *con = SSL_default_ssl(app_ctx, host, port);

Then depending on whether you want to give OpenSSL an already
connected socket, or have OpenSSL make the connection for you:

        int err = SSL_connect_socket(con, fd);

or (something along the lines of):

        BIO *bp = SSL_connect_hostport(con, host, port, &err);

At this point a stock error handler may be required, but if all
went well, you now need an I/O loop.  And so would need to either
use the connected file descriptor with your own buffering, ...
or use OpenSSL BIOs, or some other buffering I/O layer that
supports custom read/write wrappers.

On FreeBSD/NetBSD it might be nice to have stock "fittings" to
wrap stdio around OpenSSL connections via:

     #include <stdio.h>

     FILE *
     funopen(const void *cookie,
             int        (*readfn)  (void *, char *, int),
             int        (*writefn) (void *, const char *, int),
             fpos_t     (*seekfn)  (void *, fpos_t, int),
             int        (*closefn) (void *));

Providing higher level interfaces to the core TLS library would
make a good project for someone good at API design, who is familiar
with OpenSSL, but more interested in the API than the internals.

--
        Viktor.
Reply | Threaded
Open this post in threaded view
|

Re: The smallest minimal example of an HTTPS GET request with openssl

Viktor Dukhovni


> On Mar 30, 2019, at 4:28 PM, Ivan Medoedov <[hidden email]> wrote:
>
> Thanks, Viktor.

You're welcome.  One important note about the example on the Wiki.
Since OpenSSL 1.0.2, there is internal support for certificate
name checks.  You should not roll your own.

The SSL_set1_host(3) interface is present since OpenSSL 1.1.0.
In OpenSSL 1.0.2 you can use SSL_CTX_get0_param(3) and
X509_VERIFY_PARAM_set1_host(3):

        X509_VERIFY_PARAM *vpm = SSL_get0_param(ssl);
        X509_VERIFY_PARAM_set1_host(vpm, "www.foo.com", 0);

Either of the above needs to happen before the handshake starts and
then the checks are made automatically as part of the handshake,
resulting in a certificate verification failure if the name checks
fail.

Alternatively, you can call  X509_check_host(3) after the handshake
completes.  This might also then need to happen after session resumption,
because the cached certificate validity would only cover the trust path,
and not the name checks.  But if you never resume sessions that failed
name checks previously, and never re-use sessions across different
host names (for the same IP e.g.) then you might be safe without, some
care is recommended.

--
        Viktor.

Reply | Threaded
Open this post in threaded view
|

RE: The smallest minimal example of an HTTPS GET request with openssl

Porter, Andrew
In reply to this post by Ivan Medoedov

You would have to ignore some initial certificate info output but some variation on

 

echo "GET /version" | openssl s_client -connect test.example.com -crlf -ign_eof -quiet

 

would be as pure OpenSSL as you can get…

 

Andrew

 

From: openssl-users [mailto:[hidden email]] On Behalf Of Ivan Medoedov
Sent: Saturday, March 30, 2019 08:03
To: [hidden email]
Subject: [EXTERNAL] The smallest minimal example of an HTTPS GET request with openssl

 

Hello,

 

I've only managed to find this, but it seems to do too much for what I need:

 

 

Basically I need something like Go's http.Get("https://mysite.com/version") to just get a one line of text from the server.

 

I can't use libcurl, I have to use pure openssl.

 

Thanks