Deleted client certificate trust expectations

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Deleted client certificate trust expectations

Dan Freed

Hello,

 

I have a question/issue about how OpenSSL should handle a deleted client certificate. It appears that once a trusted certificate is read from the filesystem, it remains trusted throughout the lifespan of the server process.

 

I wrote a small SSL web service that reproduces the issue I’m having with my application.

 

Pardon the Perl syntax – I’ve not rewritten this in C but I think the intent is clear.  This code reproduces the scenario:

 

use Socket;

use Net::SSLeay qw(die_now die_if_ssl_error);

Net::SSLeay::load_error_strings();

Net::SSLeay::SSLeay_add_ssl_algorithms();

Net::SSLeay::randomize();

$our_ip = "\0\0\0\0";

$port = 1235;

$sockaddr_template = 'S n a4 x8';

$our_serv_params = pack ($sockaddr_template, &AF_INET, $port, $our_ip);

socket (S, &AF_INET, &SOCK_STREAM, 0)  or die "socket: $!";

bind (S, $our_serv_params)             or die "bind:   $!";

listen (S, 5);

$ctx = Net::SSLeay::CTX_new ();

$key = "client.key";

$cert = "client.crt";

$trust_dir = "/client_trusted_certificates";

Net::SSLeay::CTX_use_RSAPrivateKey_file($ctx, $key, Net::SSLeay::FILETYPE_PEM());

Net::SSLeay::CTX_use_certificate_file($ctx, $cert, Net::SSLeay::FILETYPE_PEM());

Net::SSLeay::CTX_set_session_id_context($ctx,'sessiontest',length('sessiontest'));

Net::SSLeay::CTX_load_verify_locations($ctx,"",$trust_dir);

Net::SSLeay::CTX_set_verify($ctx,&Net::SSLeay::VERIFY_PEER, \&verify_client_cert);

while (1)

{

   $addr = accept (NS, S);

   select (NS);

   $| = 1;

   select (STDOUT);

   $ssl = Net::SSLeay::new($ctx);

   Net::SSLeay::set_fd($ssl, fileno(NS));

   $err = Net::SSLeay::accept($ssl);

   $got = Net::SSLeay::read($ssl);

   print $got."\n";

   Net::SSLeay::write ($ssl, uc ($got));

   Net::SSLeay::free ($ssl);

   close NS;

}

 

sub verify_client_cert

{

    my ($pre_verify,$x509_store) = @_;

 

    print "Pre-verify: $pre_verify\n";

    print "ctx error: ".Net::SSLeay::X509_STORE_CTX_get_error($x509_store)."\n";

    return $pre_verify;

}

 

This all works as it should, and verify_client_cert() is called appropriately when the client cert is provided.

 

The issue I’m having is how the verify process works when a certificate is removed from the trusted directory while this service is running.  If a client connects with a client cert and the service verifies that certificate, then the trusted client cert is removed from /trusted_clients, then the client connects again – the client cert will still verify.  The client cert will continue to  verify until I restart the server. 

 

An strace of the process confirms that it only opens the trusted directory once, subsequent connections using this client cert do not re-open or look for the file in the trust directory.

 

My understanding of how this should work was that it should read the contents of that directory at the time the verify takes place, not when CTX_set_verify() is called, but that doesn’t seem to be what is happening. 

 

Another interesting bit is that the inverse is not true.  If I add a cert to the trusted directory, it immediately uses it without having to restart the process.

 

I assume that if I used a certificate revocation list and revoked the client cert this wouldn’t be an issue, but why are the directory contents cached? Is this for performance reasons?

 

Thanks

Dan Freed

 

 

Reply | Threaded
Open this post in threaded view
|

Re: Deleted client certificate trust expectations

Dan Freed

Sorry I realized I didn’t include the OpenSSL version I was using.

 

This is with OpenSSL 1.1.1d  10 Sep 2019.

 

-Dan

 

From: openssl-users <[hidden email]>
Date: Wednesday, November 11, 2020 at 10:29 AM
To: [hidden email] <[hidden email]>
Subject: Deleted client certificate trust expectations

External Mail. Careful of links / attachments. Submit Helpdesk if unsure.

 

Hello,

 

I have a question/issue about how OpenSSL should handle a deleted client certificate. It appears that once a trusted certificate is read from the filesystem, it remains trusted throughout the lifespan of the server process.

 

I wrote a small SSL web service that reproduces the issue I’m having with my application.

 

Pardon the Perl syntax – I’ve not rewritten this in C but I think the intent is clear.  This code reproduces the scenario:

 

use Socket;

use Net::SSLeay qw(die_now die_if_ssl_error);

Net::SSLeay::load_error_strings();

Net::SSLeay::SSLeay_add_ssl_algorithms();

Net::SSLeay::randomize();

$our_ip = "\0\0\0\0";

$port = 1235;

$sockaddr_template = 'S n a4 x8';

$our_serv_params = pack ($sockaddr_template, &AF_INET, $port, $our_ip);

socket (S, &AF_INET, &SOCK_STREAM, 0)  or die "socket: $!";

bind (S, $our_serv_params)             or die "bind:   $!";

listen (S, 5);

$ctx = Net::SSLeay::CTX_new ();

$key = "client.key";

$cert = "client.crt";

$trust_dir = "/client_trusted_certificates";

Net::SSLeay::CTX_use_RSAPrivateKey_file($ctx, $key, Net::SSLeay::FILETYPE_PEM());

Net::SSLeay::CTX_use_certificate_file($ctx, $cert, Net::SSLeay::FILETYPE_PEM());

Net::SSLeay::CTX_set_session_id_context($ctx,'sessiontest',length('sessiontest'));

Net::SSLeay::CTX_load_verify_locations($ctx,"",$trust_dir);

Net::SSLeay::CTX_set_verify($ctx,&Net::SSLeay::VERIFY_PEER, \&verify_client_cert);

while (1)

{

   $addr = accept (NS, S);

   select (NS);

   $| = 1;

   select (STDOUT);

   $ssl = Net::SSLeay::new($ctx);

   Net::SSLeay::set_fd($ssl, fileno(NS));

   $err = Net::SSLeay::accept($ssl);

   $got = Net::SSLeay::read($ssl);

   print $got."\n";

   Net::SSLeay::write ($ssl, uc ($got));

   Net::SSLeay::free ($ssl);

   close NS;

}

 

sub verify_client_cert

{

    my ($pre_verify,$x509_store) = @_;

 

    print "Pre-verify: $pre_verify\n";

    print "ctx error: ".Net::SSLeay::X509_STORE_CTX_get_error($x509_store)."\n";

    return $pre_verify;

}

 

This all works as it should, and verify_client_cert() is called appropriately when the client cert is provided.

 

The issue I’m having is how the verify process works when a certificate is removed from the trusted directory while this service is running.  If a client connects with a client cert and the service verifies that certificate, then the trusted client cert is removed from /trusted_clients, then the client connects again – the client cert will still verify.  The client cert will continue to  verify until I restart the server. 

 

An strace of the process confirms that it only opens the trusted directory once, subsequent connections using this client cert do not re-open or look for the file in the trust directory.

 

My understanding of how this should work was that it should read the contents of that directory at the time the verify takes place, not when CTX_set_verify() is called, but that doesn’t seem to be what is happening. 

 

Another interesting bit is that the inverse is not true.  If I add a cert to the trusted directory, it immediately uses it without having to restart the process.

 

I assume that if I used a certificate revocation list and revoked the client cert this wouldn’t be an issue, but why are the directory contents cached? Is this for performance reasons?

 

Thanks

Dan Freed

 

 

Reply | Threaded
Open this post in threaded view
|

Re: Deleted client certificate trust expectations

JordanBrown
In reply to this post by Dan Freed
What you observe is indeed reality; we ran into it too.  (Though we ran into it in the context of a long-running client verifying server certificates.)

My assumption is that it's for performance, and that's sensible, but it would sure be nice to figure out how to detect those changes.  If a stat() on each verification is considered too expensive, maybe there could be a timeout, that if the file hasn't been checked in the last ten minutes then check it.

-- 
Jordan Brown, Oracle ZFS Storage Appliance, Oracle Solaris
Reply | Threaded
Open this post in threaded view
|

Re: Deleted client certificate trust expectations

Viktor Dukhovni
In reply to this post by Dan Freed
On Wed, Nov 11, 2020 at 04:28:40PM +0000, Dan Freed wrote:

> I have a question/issue about how OpenSSL should handle a deleted
> client certificate. It appears that once a trusted certificate is read
> from the filesystem, it remains trusted throughout the lifespan of the
> server process.

The built-in trust stores (code behind CAfile and CApath) are caching
stores.  They use an in memory cache of trusted certificates that is
pre-loaded in the case of CAfile, and demand-loaded on a cache miss in
the case of CApath.  Once a certificate is loaded, it remains in the
cache.  The cache is part of the X509_STORE object that is associated
with the SSL_CTX.

Though I don't see it exposed in the Perl API, it is possible to flush
the X509_STORE cache by calling:

    SSL_CTX *ctx;
    X509_STORE *store;
    STACK_OF(X509) *objs;
    X509 *x;

    ...
    store = SSL_CTX_get_cert_store(ctx);
    X509_STORE_lock(store);
    st = X509_STORE_get0_objects(store);
    while ((x = sk_X509_pop(st)) != NULL)
        X509_free(x);
    X509_STORE_unlock(store);
    ....

An application that uses only CApath and does not wish to cache trusted
certificates indefinitely, can use this to flush the cache.  Note that
this does not work well with CAfile, since the file is read just once,
so you'd need to explicitly reload the CAfile:

    lookup = X509_STORE_add_lookup(ctx, X509_LOOKUP_file());
    if (lookup == NULL)
        return 0;
    if (X509_LOOKUP_load_file(lookup, file, X509_FILETYPE_PEM) != 1)
        return 0;

But keep in mind that X509_LOOKUP_load_file is not atomic, it adds
certificates to the store one at a time.  Therefore flushing and
reloading the store should happen in the same thread and should not
happen concurrently in multiple threads.

A sufficiently sophisticated user can of course add a custom store
that uses no cache, or a more sophisticated cache with expiration
times, ...

> My understanding of how this should work was that it should read the
> contents of that directory at the time the verify takes place, not
> when CTX_set_verify() is called, but that doesn't seem to be what is
> happening.

The directory content is (partly) cached, with the cache growing
incrementally as additional certificates are loaded.

--
    Viktor.
Reply | Threaded
Open this post in threaded view
|

Re: Deleted client certificate trust expectations

Dan Freed

Thanks for the help. This got me on the right track.

 

-Dan

 

From: openssl-users <[hidden email]>
Date: Wednesday, November 11, 2020 at 12:02 PM
To: [hidden email] <[hidden email]>
Subject: Re: Deleted client certificate trust expectations

External Mail. Careful of links / attachments. Submit Helpdesk if unsure.


On Wed, Nov 11, 2020 at 04:28:40PM +0000, Dan Freed wrote:

> I have a question/issue about how OpenSSL should handle a deleted
> client certificate. It appears that once a trusted certificate is read
> from the filesystem, it remains trusted throughout the lifespan of the
> server process.

The built-in trust stores (code behind CAfile and CApath) are caching
stores.  They use an in memory cache of trusted certificates that is
pre-loaded in the case of CAfile, and demand-loaded on a cache miss in
the case of CApath.  Once a certificate is loaded, it remains in the
cache.  The cache is part of the X509_STORE object that is associated
with the SSL_CTX.

Though I don't see it exposed in the Perl API, it is possible to flush
the X509_STORE cache by calling:

    SSL_CTX *ctx;
    X509_STORE *store;
    STACK_OF(X509) *objs;
    X509 *x;

    ...
    store = SSL_CTX_get_cert_store(ctx);
    X509_STORE_lock(store);
    st = X509_STORE_get0_objects(store);
    while ((x = sk_X509_pop(st)) != NULL)
        X509_free(x);
    X509_STORE_unlock(store);
    ....

An application that uses only CApath and does not wish to cache trusted
certificates indefinitely, can use this to flush the cache.  Note that
this does not work well with CAfile, since the file is read just once,
so you'd need to explicitly reload the CAfile:

    lookup = X509_STORE_add_lookup(ctx, X509_LOOKUP_file());
    if (lookup == NULL)
        return 0;
    if (X509_LOOKUP_load_file(lookup, file, X509_FILETYPE_PEM) != 1)
        return 0;

But keep in mind that X509_LOOKUP_load_file is not atomic, it adds
certificates to the store one at a time.  Therefore flushing and
reloading the store should happen in the same thread and should not
happen concurrently in multiple threads.

A sufficiently sophisticated user can of course add a custom store
that uses no cache, or a more sophisticated cache with expiration
times, ...

> My understanding of how this should work was that it should read the
> contents of that directory at the time the verify takes place, not
> when CTX_set_verify() is called, but that doesn't seem to be what is
> happening.

The directory content is (partly) cached, with the cache growing
incrementally as additional certificates are loaded.

--
    Viktor.