Peer certificate verification in verify_callback

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

Peer certificate verification in verify_callback

Jason Schultz

I have some questions about my application’s verify_callback() function and how I handle some of the OpenSSL errors.

 

For example, if my client application is presented a self-signed certificate in the handshake, verify_callback() is called with an error, for which X509_STORE_CTX_get_error() returns 18/X509_V_ERR_DEPTH_ZERO_SELF_SIGNED_CERT. In this case, my application searches its trusted store for this certificate, and if it finds a match, the error is cleared and the handshake is allow to proceed.

 

Other examples are cases where my client application is presented with a certificate chain. Let’s say the chain looks like root -> intermediate -> end-entity, but the server is configured to not send the root, so my client gets: intermediate -> end-entity in the handshake.

 

One case is where my client is presented these  certificates and has the end-entity certificate in its trusted store. In this case, the verify_callback() gets error 20/ X509_V_ERR_UNABLE_TO_GET_ISSUER_CERT_LOCALLY. For this error, my application will search its trusted store for the end-entity certificate, and when a match is found the error is cleared and the handshake is allowed to proceed.

 

A slightly different case is when the client has only the intermediate certificate in its trusted store, while the server presents the intermediate -> end-entity chain. In this case, verify_callback() is called with an error, and X509_STORE_CTX_get_error() returns 2/ X509_V_ERR_UNABLE_TO_GET_ISSUER_CERT.

 

These last two cases seem very similar but get slightly different errors. Right now my application does not look for a match in the case of X509_V_ERR_UNABLE_TO_GET_ISSUER_CERT. My plan is to add that error to the cases where the trusted store is searched for a match. Are there more subtle differences between these two errors that I’m missing? Or does my plan to have the application do the addition checking for this error make sense?


Reply | Threaded
Open this post in threaded view
|

Re: Peer certificate verification in verify_callback

Jason Schultz
Just wanted to bring this up again as I didn't get any responses initially. Has anyone dealt with this or similar issues with OpenSSL 1.1.1?




From: openssl-users <[hidden email]> on behalf of Jason Schultz <[hidden email]>
Sent: Thursday, March 5, 2020 2:04 PM
To: [hidden email] <[hidden email]>
Subject: Peer certificate verification in verify_callback
 

I have some questions about my application’s verify_callback() function and how I handle some of the OpenSSL errors.

 

For example, if my client application is presented a self-signed certificate in the handshake, verify_callback() is called with an error, for which X509_STORE_CTX_get_error() returns 18/X509_V_ERR_DEPTH_ZERO_SELF_SIGNED_CERT. In this case, my application searches its trusted store for this certificate, and if it finds a match, the error is cleared and the handshake is allow to proceed.

 

Other examples are cases where my client application is presented with a certificate chain. Let’s say the chain looks like root -> intermediate -> end-entity, but the server is configured to not send the root, so my client gets: intermediate -> end-entity in the handshake.

 

One case is where my client is presented these  certificates and has the end-entity certificate in its trusted store. In this case, the verify_callback() gets error 20/ X509_V_ERR_UNABLE_TO_GET_ISSUER_CERT_LOCALLY. For this error, my application will search its trusted store for the end-entity certificate, and when a match is found the error is cleared and the handshake is allowed to proceed.

 

A slightly different case is when the client has only the intermediate certificate in its trusted store, while the server presents the intermediate -> end-entity chain. In this case, verify_callback() is called with an error, and X509_STORE_CTX_get_error() returns 2/ X509_V_ERR_UNABLE_TO_GET_ISSUER_CERT.

 

These last two cases seem very similar but get slightly different errors. Right now my application does not look for a match in the case of X509_V_ERR_UNABLE_TO_GET_ISSUER_CERT. My plan is to add that error to the cases where the trusted store is searched for a match. Are there more subtle differences between these two errors that I’m missing? Or does my plan to have the application do the addition checking for this error make sense?


Reply | Threaded
Open this post in threaded view
|

Re: Peer certificate verification in verify_callback

Jeremy Harris
On 30/03/2020 17:01, Jason Schultz wrote:
> For example, if my client application is presented a self-signed certificate in the handshake, verify_callback() is called with an error, for which X509_STORE_CTX_get_error() returns 18/X509_V_ERR_DEPTH_ZERO_SELF_SIGNED_CERT. In this case, my application searches its trusted store for this certificate, and if it finds a match, the error is cleared and the handshake is allow to proceed.

If you don't care that the cert is verifiable, tell OpenSSL that before
connection?

SSL_CTX_set_verify  !SSL_VERIFY_FAIL_IF_NO_PEER_CERT

> Other examples are cases where my client application is presented with a certificate chain. Let’s say the chain looks like root -> intermediate -> end-entity, but the server is configured to not send the root, so my client gets: intermediate -> end-entity in the handshake.
>
>
>
> One case is where my client is presented these  certificates and has the end-entity certificate in its trusted store. In this case, the verify_callback() gets error 20/ X509_V_ERR_UNABLE_TO_GET_ISSUER_CERT_LOCALLY. For this error, my application will search its trusted store for the end-entity certificate, and when a match is found the error is cleared and the handshake is allowed to proceed.

Tell OpenSSL the root(s) you trust before connecting

--
Cheers,
  Jeremy
Reply | Threaded
Open this post in threaded view
|

Re: Peer certificate verification in verify_callback

Viktor Dukhovni
In reply to this post by Jason Schultz
On Thu, Mar 05, 2020 at 02:04:27PM +0000, Jason Schultz wrote:

> I have some questions about my application’s verify_callback() function and how I handle some of the OpenSSL errors.

You're going about this the wrong way.  Instead of tryign (likely
insecurely) to patch up verification errors in a verify callback, if you
have a certificate store that is not directly supported by OpenSSL, you
need to implement your own custom X509_STORE type, associate that store
with the SSL_CTX and have OpenSSL's built-in certificate verification
search that store for you.

If you also want to directly trust intermediate certificates that are
not self-signed roots, you can either set the "partial chain" flag,
or load into your store intermediate certificates with auxiliary
trust settings (aka "TRUSTED CERTIFICATES"), which will then be
trusted without chaining to a root, but simpler to just add the
roots to the store.

--
    Viktor.
Reply | Threaded
Open this post in threaded view
|

Re: Peer certificate verification in verify_callback

Jason Schultz
Victor, Jeremy-

Thanks for your responses. It sounds like I should maybe take a step back and describe what I'm doing and how. I'm possibly doing things fundamentally wrong, maybe because the way I'm doing them is based originally on OpenSSL 0.9.8. I'm currently moving from 1.0.2 to 1.1.1, which is why the questions are coming up.

I won't get into the details of my application as it's complex, but it can act as a client or a server. The case we are worried about is obviously when it's acting as a client. I thought the standard way of dealing with these type of errors was with setting a verify_callback() function, which is part of the description below.

I set up an X509_STORE object and then cycle through all of the certificate files in /etc/ssl/certs/, open them, and call PEM_read_X509() to get an X509 (certificate) object and then call X509_STORE_add_cert(x509_stor, certificate) to read the certificates into  my trusted store, X509_store object. Then when I create an SSL CTX, I call SSL_CTX_set_cert_store() to set the X509_store object for the CTX. I also call SSL_set_verify( ,SSL_VERIFY_PEER,verify_callback) with the SSL object created from that CTX to set the verify_callback() function for each user that will act as a client. 

If the user of this CTX is acting as a client and the server presents a certificate chain, and my trusted store has the root, the connection will work, as the chain is verified and trusted. If the server presents a self-signed certificate, the verify_callback() function is invoked with the error
18/X509_V_ERR_DEPTH_ZERO_SELF_SIGNED_CERT, but if I search for and find the self-signed cert in my trusted store, I clear the error and let the 
connection proceed. The scenario is very similar for the case I asked about, the only difference is I'm presented a 2 certs in a chain, and I have the 
intermediate cert in my store.  My understanding was the verify_callback() is the correct place to handle these cases. 

From Victor's response, I don't know what a custom X509_STORE type is or how to set one up. Likewise, I don't know how to use the "partial chain" flag 
and what API I need to load intermediate certificates into my trusted store(other than what I describe above). Because of the way certificates are 
distributed, I can't always rely on having the root in the trusted store, so I'll need to trust some intermediate certs, provided they are valid, actually signed
the end-entity cert, etc. 

Let me know if I am off track, and if so, what APIs I should be looking at, especially if there are been changes in this area.

Thanks.



From: openssl-users <[hidden email]> on behalf of Viktor Dukhovni <[hidden email]>
Sent: Monday, March 30, 2020 6:17 PM
To: [hidden email] <[hidden email]>
Subject: Re: Peer certificate verification in verify_callback
 
On Thu, Mar 05, 2020 at 02:04:27PM +0000, Jason Schultz wrote:

> I have some questions about my application’s verify_callback() function and how I handle some of the OpenSSL errors.

You're going about this the wrong way.  Instead of tryign (likely
insecurely) to patch up verification errors in a verify callback, if you
have a certificate store that is not directly supported by OpenSSL, you
need to implement your own custom X509_STORE type, associate that store
with the SSL_CTX and have OpenSSL's built-in certificate verification
search that store for you.

If you also want to directly trust intermediate certificates that are
not self-signed roots, you can either set the "partial chain" flag,
or load into your store intermediate certificates with auxiliary
trust settings (aka "TRUSTED CERTIFICATES"), which will then be
trusted without chaining to a root, but simpler to just add the
roots to the store.

--
    Viktor.
Reply | Threaded
Open this post in threaded view
|

Re: Peer certificate verification in verify_callback

Viktor Dukhovni
On Mon, Mar 30, 2020 at 09:02:47PM +0000, Jason Schultz wrote:

> I won't get into the details of my application as it's complex, but it
> can act as a client or a server. The case we are worried about is
> obviously when it's acting as a client. I thought the standard way of
> dealing with these type of errors was with setting a verify_callback()
> function, which is part of the description below.

The verify callback is mostly for logging and error reporting.  It is
not intended to supplant the built-in verification logic.  While it
can be used to ignore some errors, that's generally quite risky and
difficult to do correctly.  You should strive to arrange for the
built-in verification to succeed, rather than attempt to work around
the resulting errors when it does not.

> I set up an X509_STORE object and then cycle through all of the
> certificate files in /etc/ssl/certs/, open them, and call
> PEM_read_X509() to get an X509 (certificate) object and then call
> X509_STORE_add_cert(x509_stor, certificate) to read the certificates
> into  my trusted store, X509_store object.

It would be far simpler to concatenate them into a single CAfile, or use
"c_rehash" to create the symlinks need to make the directory into a
workable CApath.  You should not have to manually load them into your
own store, although doing the latter potentially gives you the
opportunity to decorate them with auxiliary trust EKUs.


> If the user of this CTX is acting as a client and the server presents
> a certificate chain, and my trusted store has the root, the connection
> will work, as the chain is verified and trusted.

With the partial chain flag it can also work when the EE cert is present
(verbatim) in the store, or an intermediate CA is present in the store.

> If the server presents a self-signed certificate, the
> verify_callback() function is invoked with the error
>
> 18/X509_V_ERR_DEPTH_ZERO_SELF_SIGNED_CERT, but if I search for and find the self-signed cert in my trusted store, I clear the error and let the
> connection proceed.

With 1.0.2 and later, the partial chain flag makes this work-around
unnecessary.

> The scenario is very similar for the case I asked about, the only difference is I'm presented a 2 certs in a chain, and I have the
> intermediate cert in my store.  My understanding was the verify_callback() is the correct place to handle these cases.

No, it is not.

> From Victor's response, I don't know what a custom X509_STORE type is
> or how to set one up.

You're already populating a custom store (though not a store type, which
it does not look like you need since your store is just a directory full
of PEM files).

> Likewise, I don't know how to use the "partial chain" flag

See X509_VERIFY_PARAM_set_flags(3).

> and what API I need to load intermediate certificates into my trusted
> store(other than what I describe above). Because of the way
> certificates are distributed, I can't always rely on having the root
> in the trusted store, so I'll need to trust some intermediate certs,
> provided they are valid, actually signed the end-entity cert, etc.

You just need to add them to the store, simplest is a CAfile or
a hashed CApath.

--
    Viktor.
Reply | Threaded
Open this post in threaded view
|

Re: Peer certificate verification in verify_callback

Jason Schultz
Thanks Victor.

I need to look at X509_VERIFY_PARAM_set_flags() a little closer, but I think I understand what I need to do. 

I also can't concatenate all my trusted certificates into a single file, there are dozens of certificates in the trusted store. Our users can also manipulate the trusted store, so the trusted certificates will always be in PEM files in /etc/ssl/certs/. 

It sounds like that's not going to hold me back from accomplishing what I need to do though, but I'll pursue this and let the list know if I run into any other issues. 

Thanks again,

Jason



From: openssl-users <[hidden email]> on behalf of Viktor Dukhovni <[hidden email]>
Sent: Monday, March 30, 2020 9:19 PM
To: [hidden email] <[hidden email]>
Subject: Re: Peer certificate verification in verify_callback
 
On Mon, Mar 30, 2020 at 09:02:47PM +0000, Jason Schultz wrote:

> I won't get into the details of my application as it's complex, but it
> can act as a client or a server. The case we are worried about is
> obviously when it's acting as a client. I thought the standard way of
> dealing with these type of errors was with setting a verify_callback()
> function, which is part of the description below.

The verify callback is mostly for logging and error reporting.  It is
not intended to supplant the built-in verification logic.  While it
can be used to ignore some errors, that's generally quite risky and
difficult to do correctly.  You should strive to arrange for the
built-in verification to succeed, rather than attempt to work around
the resulting errors when it does not.

> I set up an X509_STORE object and then cycle through all of the
> certificate files in /etc/ssl/certs/, open them, and call
> PEM_read_X509() to get an X509 (certificate) object and then call
> X509_STORE_add_cert(x509_stor, certificate) to read the certificates
> into  my trusted store, X509_store object.

It would be far simpler to concatenate them into a single CAfile, or use
"c_rehash" to create the symlinks need to make the directory into a
workable CApath.  You should not have to manually load them into your
own store, although doing the latter potentially gives you the
opportunity to decorate them with auxiliary trust EKUs.


> If the user of this CTX is acting as a client and the server presents
> a certificate chain, and my trusted store has the root, the connection
> will work, as the chain is verified and trusted.

With the partial chain flag it can also work when the EE cert is present
(verbatim) in the store, or an intermediate CA is present in the store.

> If the server presents a self-signed certificate, the
> verify_callback() function is invoked with the error
>
> 18/X509_V_ERR_DEPTH_ZERO_SELF_SIGNED_CERT, but if I search for and find the self-signed cert in my trusted store, I clear the error and let the
> connection proceed.

With 1.0.2 and later, the partial chain flag makes this work-around
unnecessary.

> The scenario is very similar for the case I asked about, the only difference is I'm presented a 2 certs in a chain, and I have the
> intermediate cert in my store.  My understanding was the verify_callback() is the correct place to handle these cases.

No, it is not.

> From Victor's response, I don't know what a custom X509_STORE type is
> or how to set one up.

You're already populating a custom store (though not a store type, which
it does not look like you need since your store is just a directory full
of PEM files).

> Likewise, I don't know how to use the "partial chain" flag

See X509_VERIFY_PARAM_set_flags(3).

> and what API I need to load intermediate certificates into my trusted
> store(other than what I describe above). Because of the way
> certificates are distributed, I can't always rely on having the root
> in the trusted store, so I'll need to trust some intermediate certs,
> provided they are valid, actually signed the end-entity cert, etc.

You just need to add them to the store, simplest is a CAfile or
a hashed CApath.

--
    Viktor.
Reply | Threaded
Open this post in threaded view
|

RE: Peer certificate verification in verify_callback

Michel
In reply to this post by Viktor Dukhovni
Hi Viktor,

Could you please elaborate on "...although doing the latter potentially
gives you the
opportunity to decorate them with auxiliary trust EKUs."

Does it mean "EKUs" "out of" the certificate ?

Is it just about using X509_STORE_set_trust() and the like as mentioned in
X509_STORE_add_cert man page or something else ?

Regards,

Michel

-----Message d'origine-----
De : openssl-users [mailto:[hidden email]] De la part de
Viktor Dukhovni
Envoyé : lundi 30 mars 2020 23:19
À : [hidden email]
Objet : Re: Peer certificate verification in verify_callback

[...]

> I set up an X509_STORE object and then cycle through all of the
> certificate files in /etc/ssl/certs/, open them, and call
> PEM_read_X509() to get an X509 (certificate) object and then call
> X509_STORE_add_cert(x509_stor, certificate) to read the certificates
> into  my trusted store, X509_store object.

It would be far simpler to concatenate them into a single CAfile, or use
"c_rehash" to create the symlinks need to make the directory into a
workable CApath.  You should not have to manually load them into your
own store, although doing the latter potentially gives you the
opportunity to decorate them with auxiliary trust EKUs.


> If the user of this CTX is acting as a client and the server presents
> a certificate chain, and my trusted store has the root, the connection
> will work, as the chain is verified and trusted.


[...]

--
    Viktor.