SSL_CTX_set_cert_verify_callback and certificate access

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
13 messages Options
Reply | Threaded
Open this post in threaded view
|

SSL_CTX_set_cert_verify_callback and certificate access

Corey Minyard
I'm working on an application using openssl, and I would like to set
some things up for verification based upon information in the
certificate.  Unfortunately, from what I can tell, there is no way to do
this.  (Maybe it's not a good idea.  Not sure.)

What I would like to do is pull out some information from the
certificate that is being verified, set/modify the verify store based
upon that information (basically chose the CA based upon something in
the certificate.  What I really need is X509_STORE_CTX_get_cert(), but
that function does not exist, and there's no way to get ctx->cert from
what I can tell.  It's not available with SSL_get_peer_certificate at
the point where the cert verify call is done.

It would also be nice to be able to replace the verify store in the
X509_STORE_CTX, or empty it, but I haven't looked too hard at that.

More details on what I am trying to do follow, in case you are interested...

I am the maintainer for ser2net, a program that allows network
connections to connect to serial ports.  People have asked for login
security, but I refuse to transmit passwords like this over the network
in the clear.  But, in reality, people are logging in over this
interface, and it has bothered me for a while.  So, I've been looking at
adding security.  I have rewritten ser2net to split it into two parts: a
library that does general-purpose stream I/O to handle all the
connections and the serial ports, and the main handling and
configuration.  The library (called gensio) is a layered system, so you
have TCP/UCP/SCTP/stdio/serial/IPMIsol available as low-level
interfaces.  Then you have filter layers on top, like SSL and telnet. 
So you can create an SCTP connection, put SSL on top of that, then put
telnet on top of that, for instance.  I already have basic SSL support
working.

My first inclination for a secure connection was to use ssh. However,
ssh is not as well suited for this as I would have liked, and all the
ssh libraries are tied to a file descriptor in ways that are not easily
fixable, and thus can't be used on top of an abstract connection, which
is what I need.  That was rather disappointing, as it would have been
really nice to for users to just be able to ssh to ser2net.

So now I'm looking at doing something like what ssh does, but with
openssl. Unfortunately, SSL has no concept of a userid and I would like
to have it verify certificates from different stores based upon a
userid.  I've come up with the following options:

 1. Send the userid in a lower layer filter so it is transmitted before
    ssl starts up.  This means the userid is not authenticated in any
    way, which seems like a bad idea.
 2. Set the userid in the certificate and use client authentication to
    authenticate the user logging in.  Set the username in the CN field
    of the certificate so it can't be changed, extract that and set the
    CA before verification.  This is what I'm currently trying to do,
    and I keep running into roadblocks.
 3. Create a filter layer that can sit on top of SSL that will basically
    do what SSL client authentication does, except it can get the userid
    as the first part of the data and then run the authentication from
    there.  This is definitely doable, and then the userid is
    transmitted encrypted (which seems nice) but it's duplicating some
    fairly complex code that would already be done for me by openssl.

I am afraid I am going to be stuck with option 3, which is not terrible,
I suppose.  But does anyone have any ideas here?

Thanks,

-corey

--
openssl-users mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-users
Reply | Threaded
Open this post in threaded view
|

Re: SSL_CTX_set_cert_verify_callback and certificate access

JordanBrown
On 1/9/2019 6:54 PM, Corey Minyard wrote:
2. Set the userid in the certificate and use client authentication to
   authenticate the user logging in.  Set the username in the CN field
   of the certificate so it can't be changed, extract that and set the
   CA before verification.  This is what I'm currently trying to do,
   and I keep running into roadblocks.

Why do you think you need to set the CA?

It seems like you should let OpenSSL verify the certificate against your list of trusted CAs, and if it succeeds then you know that one of those CAs vouches for this user's identity.  Then you look at their subject name to derive the user ID (probably from its CN).  If you want to be really paranoid - if you believe that Verisign can vouch for John and Comodo can vouch for Sam, but not vice versa, factor the issuer name into the process.

-- 
Jordan Brown, Oracle ZFS Storage Appliance, Oracle Solaris

--
openssl-users mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-users
Reply | Threaded
Open this post in threaded view
|

Re: SSL_CTX_set_cert_verify_callback and certificate access

Michael Wojcik
> From: openssl-users [mailto:[hidden email]] On Behalf Of Jordan Brown
> Sent: Thursday, January 10, 2019 11:15

> On 1/9/2019 6:54 PM, Corey Minyard wrote:
>> 2. Set the userid in the certificate and use client authentication to
>>   authenticate the user logging in.  Set the username in the CN field
>>   of the certificate so it can't be changed, extract that and set the
>>   CA before verification.  This is what I'm currently trying to do,
>>   and I keep running into roadblocks.

> Why do you think you need to set the CA?

Agreed. That's an odd requirement.

> Then you look at their subject name to derive the user ID (probably from its CN).

That's one logical option, but some systems (e.g. IBM zOS) instead maintain a database mapping client certificates to system user IDs. That has the advantage of not imposing any particular requirement on the client certificate's subject name format, with the disadvantage of additional administration. (For HTTPS, there's a provision for "automatic registration" where an unrecognized but valid and trusted client certificate will trigger HTTP Basic Authentication, and if those credentials are verified by the OS, the certificate is added to the database.) The database may index certificates by subject DN (for ease of administration, e.g. when certificates are renewed) or by fingerprint (as a defense against impersonation if a CA is compromised).

> If you want to be really paranoid - if you believe that Verisign can vouch for John
> and Comodo can vouch for Sam, but not vice versa, factor the issuer name into the process.

Right. For the original "set the CA" plan to work, you would need to already have some association between client certificates and CAs, so you can simply check that a validated client certificate was issued by the expected CA.

But again this would be an unusual requirement; and even if it exists, the contemporary standard mechanism for guarding against rogue or compromised CAs is Certificate Transparency. Checking CT logs seems like overkill in this case, and it would arguably be more useful to implement revocation checks and other defenses first; but if restricting certificates by issuer is important for some reason, CT might be a better approach than inventing your own mechanism.

Regarding Corey's original note: SSL/TLS does not have a "username" concept because it would be redundant or inconsistent. A certificate is a peer identifier; it takes the place of a username.

--
Michael Wojcik
Distinguished Engineer, Micro Focus

--
openssl-users mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-users
Reply | Threaded
Open this post in threaded view
|

Re: SSL_CTX_set_cert_verify_callback and certificate access

Viktor Dukhovni
In reply to this post by Corey Minyard
On Wed, Jan 09, 2019 at 08:54:30PM -0600, Corey Minyard wrote:


> What I would like to do is pull out some information from the
> certificate that is being verified, set/modify the verify store based
> upon that information (basically chose the CA based upon something in
> the certificate.  What I really need is X509_STORE_CTX_get_cert(), but
> that function does not exist,

It does in OpenSSL 1.1.0 and later:

    See X509_STORE_CTX_get0_cert().

In OpenSSL 1.0.2 the structures are not opaque, so Postfix has forward
compatibility macros:

    https://github.com/vdukhovni/postfix/blob/master/postfix/src/tls/tls.h#L92-L110

--
        Viktor.
--
openssl-users mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-users
Reply | Threaded
Open this post in threaded view
|

Re: SSL_CTX_set_cert_verify_callback and certificate access

OpenSSL - User mailing list
In reply to this post by Michael Wojcik
On 10/01/2019 18:00, Michael Wojcik wrote:

>> From: openssl-users [mailto:[hidden email]] On Behalf Of Jordan Brown
>> Sent: Thursday, January 10, 2019 11:15
>> On 1/9/2019 6:54 PM, Corey Minyard wrote:
>>> 2. Set the userid in the certificate and use client authentication to
>>>    authenticate the user logging in.  Set the username in the CN field
>>>    of the certificate so it can't be changed, extract that and set the
>>>    CA before verification.  This is what I'm currently trying to do,
>>>    and I keep running into roadblocks.
>> Why do you think you need to set the CA?
> Agreed. That's an odd requirement.
>
>> Then you look at their subject name to derive the user ID (probably from its CN).
> That's one logical option, but some systems (e.g. IBM zOS) instead maintain a database mapping client certificates to system user IDs. That has the advantage of not imposing any particular requirement on the client certificate's subject name format, with the disadvantage of additional administration. (For HTTPS, there's a provision for "automatic registration" where an unrecognized but valid and trusted client certificate will trigger HTTP Basic Authentication, and if those credentials are verified by the OS, the certificate is added to the database.) The database may index certificates by subject DN (for ease of administration, e.g. when certificates are renewed) or by fingerprint (as a defense against impersonation if a CA is compromised).
>
>> If you want to be really paranoid - if you believe that Verisign can vouch for John
>> and Comodo can vouch for Sam, but not vice versa, factor the issuer name into the process.
> Right. For the original "set the CA" plan to work, you would need to already have some association between client certificates and CAs, so you can simply check that a validated client certificate was issued by the expected CA.
>
> But again this would be an unusual requirement; and even if it exists, the contemporary standard mechanism for guarding against rogue or compromised CAs is Certificate Transparency. Checking CT logs seems like overkill in this case, and it would arguably be more useful to implement revocation checks and other defenses first; but if restricting certificates by issuer is important for some reason, CT might be a better approach than inventing your own mechanism.
CT does nothing of the sort, it just provides a means to ensure all
certificates are potentially subject to 3rd party audits.

And CT cannot be legally used for user certificates anyway because
CT publishes unredacted certificate content to the world.

One really should be careful with security ideas from a company whose
main source of income is to spy on the world population for profit.

> Regarding Corey's original note: SSL/TLS does not have a "username" concept because it would be redundant or inconsistent. A certificate is a peer identifier; it takes the place of a username.

Enjoy

Jakob
--
Jakob Bohm, CIO, Partner, WiseMo A/S.  https://www.wisemo.com
Transformervej 29, 2860 Søborg, Denmark.  Direct +45 31 13 16 10
This public discussion message is non-binding and may contain errors.
WiseMo - Remote Service Management for PCs, Phones and Embedded

--
openssl-users mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-users
Reply | Threaded
Open this post in threaded view
|

Re: SSL_CTX_set_cert_verify_callback and certificate access

Corey Minyard
In reply to this post by Michael Wojcik
On 1/10/19 11:00 AM, Michael Wojcik wrote:

>> From: openssl-users [mailto:[hidden email]] On Behalf Of Jordan Brown
>> Sent: Thursday, January 10, 2019 11:15
>> On 1/9/2019 6:54 PM, Corey Minyard wrote:
>>> 2. Set the userid in the certificate and use client authentication to
>>>    authenticate the user logging in.  Set the username in the CN field
>>>    of the certificate so it can't be changed, extract that and set the
>>>    CA before verification.  This is what I'm currently trying to do,
>>>    and I keep running into roadblocks.
>> Why do you think you need to set the CA?
> Agreed. That's an odd requirement.

Thanks for the responses.

It is unusual, perhaps, but I'm trying to implement something like ssh
does.  I can't expect users of ser2net to obtain certificates from a
real certificate authority, that's too high a barrier for entry.  I want
them to be able to generate a key pair, put the public key on the server
in their account, and authenticate against that.

It's a balance of getting reasonable security that people will actually use.

-corey

--
openssl-users mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-users
Reply | Threaded
Open this post in threaded view
|

Re: SSL_CTX_set_cert_verify_callback and certificate access

Corey Minyard
In reply to this post by Viktor Dukhovni
On 1/10/19 11:17 AM, Viktor Dukhovni wrote:

> On Wed, Jan 09, 2019 at 08:54:30PM -0600, Corey Minyard wrote:
>
>
>> What I would like to do is pull out some information from the
>> certificate that is being verified, set/modify the verify store based
>> upon that information (basically chose the CA based upon something in
>> the certificate.  What I really need is X509_STORE_CTX_get_cert(), but
>> that function does not exist,
> It does in OpenSSL 1.1.0 and later:
>
>      See X509_STORE_CTX_get0_cert().
>
> In OpenSSL 1.0.2 the structures are not opaque, so Postfix has forward
> compatibility macros:
>
>      https://github.com/vdukhovni/postfix/blob/master/postfix/src/tls/tls.h#L92-L110
>
That's useful, thanks.  The macros are just what I needed.

-corey

--
openssl-users mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-users
Reply | Threaded
Open this post in threaded view
|

Re: SSL_CTX_set_cert_verify_callback and certificate access

OpenSSL - User mailing list
In reply to this post by Corey Minyard
On 10/01/2019 19:55, Corey Minyard wrote:

> On 1/10/19 11:00 AM, Michael Wojcik wrote:
>>> From: openssl-users [mailto:[hidden email]] On
>>> Behalf Of Jordan Brown
>>> Sent: Thursday, January 10, 2019 11:15
>>> On 1/9/2019 6:54 PM, Corey Minyard wrote:
>>>> 2. Set the userid in the certificate and use client authentication to
>>>>    authenticate the user logging in.  Set the username in the CN field
>>>>    of the certificate so it can't be changed, extract that and set the
>>>>    CA before verification.  This is what I'm currently trying to do,
>>>>    and I keep running into roadblocks.
>>> Why do you think you need to set the CA?
>> Agreed. That's an odd requirement.
>
> Thanks for the responses.
>
> It is unusual, perhaps, but I'm trying to implement something like ssh
> does.  I can't expect users of ser2net to obtain certificates from a
> real certificate authority, that's too high a barrier for entry.  I
> want them to be able to generate a key pair, put the public key on the
> server in their account, and authenticate against that.
>
> It's a balance of getting reasonable security that people will
> actually use.
>
A simpler solution is to report "Any client CA accepted", then
compare the returned certificate fingerprint (strong hash of
DER-encoded end cert) against a user database, listing the
user name for each cert.

Validation then only checks if the certificate is revoked or
technically invalid (expired, claims to be signed by a CA
that didn't, syntactically invalid, wrong Extended-Key-Usage
etc.).  But signed by a completely unknown CA, or self-signed
is A-OK as long as the end cert is listed as belonging to that
user (similar to an SSH public key).

By the way, I do think ssh can be made to work by handing the
SSH library the actual serial port handles once the user has
been authenticated.  Some SSH libraries may even be able to
do things like BREAK via standard SSH mechanisms.




Enjoy

Jakob
--
Jakob Bohm, CIO, Partner, WiseMo A/S.  https://www.wisemo.com
Transformervej 29, 2860 Søborg, Denmark.  Direct +45 31 13 16 10
This public discussion message is non-binding and may contain errors.
WiseMo - Remote Service Management for PCs, Phones and Embedded

--
openssl-users mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-users
Reply | Threaded
Open this post in threaded view
|

Re: SSL_CTX_set_cert_verify_callback and certificate access

JordanBrown
In reply to this post by Corey Minyard
On 1/10/2019 10:55 AM, Corey Minyard wrote:
It is unusual, perhaps, but I'm trying to implement something like ssh does.  I can't expect users of ser2net to obtain certificates from a real certificate authority, that's too high a barrier for entry.  I want them to be able to generate a key pair, put the public key on the server in their account, and authenticate against that.


Nobody said you needed a real certificate authority.  You need a *trusted* certificate authority.

You could put the user's self-signed certificate into their account as a trusted CA.

However... it seems like you're reinventing ssh.  Your replacement for ssh will likely require a custom client, which will be a pain in the neck for your users.  Maybe you should start with an existing ssh library and hack it until it behaves the way you need.

-- 
Jordan Brown, Oracle ZFS Storage Appliance, Oracle Solaris

--
openssl-users mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-users
Reply | Threaded
Open this post in threaded view
|

Re: SSL_CTX_set_cert_verify_callback and certificate access

Sam Roberts
In reply to this post by Corey Minyard
On Wed, Jan 9, 2019 at 6:54 PM Corey Minyard <[hidden email]> wrote:
> My first inclination for a secure connection was to use ssh. However,
> ssh is not as well suited for this as I would have liked, and all the
> ssh libraries are tied to a file descriptor in ways that are not easily
> fixable, and thus can't be used on top of an abstract connection, which
> is what I need.  That was rather disappointing, as it would have been
> really nice to for users to just be able to ssh to ser2net.

Not to second guess your finding that ssh isn't working well for you,
you know your own code best, but for my own interest, I'm curious what
about the fd is a problem? Perhaps the mismatch between X.509+TLS and
the auth model you want are enough to reconsider your abstractions?
Generating certs is pretty annoying and fragile, and using ssh clients
is pretty easy!

It sounds like your are building the abstractions (in C?) inside the
sernet process, but maybe your abstraction can be an fd, and the
"layers" can be child processes that connect fd-to-fd, sortof
qmail-like? Or, ssh should be able to execute an arbitrary command on
the server, and that command should be able to do anything it wants
with the ssh-facing socket descriptors, perhaps sending data to/from
your server which can then move the data through the in-process
abstractions?

Cheers,
Sam
--
openssl-users mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-users
Reply | Threaded
Open this post in threaded view
|

Re: SSL_CTX_set_cert_verify_callback and certificate access

Corey Minyard
In reply to this post by JordanBrown
On 1/11/19 12:14 PM, Jordan Brown wrote:

> On 1/10/2019 10:55 AM, Corey Minyard wrote:
>> It is unusual, perhaps, but I'm trying to implement something like
>> ssh does.  I can't expect users of ser2net to obtain certificates
>> from a real certificate authority, that's too high a barrier for
>> entry.  I want them to be able to generate a key pair, put the public
>> key on the server in their account, and authenticate against that.
>
>
> Nobody said you needed a real certificate authority.  You need a
> *trusted* certificate authority.
>
> You could put the user's self-signed certificate into their account as
> a trusted CA.
>
Well, that's what I meant :)  And the self-signed certificates is what
I've planned.  If I used the default CA, then the user would have to get
a certificate from someplace else, they couldn't generate it themselves.


> However... it seems like you're reinventing ssh.  Your replacement for
> ssh will likely require a custom client, which will be a pain in the
> neck for your users.  Maybe you should start with an existing ssh
> library and hack it until it behaves the way you need.
>
Indeed, ssh is where I started, and I would love to use it.  A custom
client will be a pain, for sure, and I'd like to avoid it if possible.

But openssh only allows one connection per process, which is definitely
not going to work with what I need.

libssh should be able to support multiple connections per process, but I
spent about a week hacking on it, and the code is all wrapped around
itself, not layered, and the changes to fix it would be huge.  And I'm
not going to support a fork of something that complex, and the
maintainer didn't seem too keen on those types of changes.

I didn't find any other server libraries that were suitable.  I'm not
going to write my own ssh library.

Plus no ssh client will do serial port control over the network
connection (like RFC2217), and many ser2net users use that.  So I might
have been stuck with a custom client, anyway.

I don't really like my options, but I'm kind of boxed in.  I'm not sure
why ssh doesn't run on top of ssl; that seems so sensible.  I assume
that's for historical reasons.

Thanks,

-corey

--
openssl-users mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-users
Reply | Threaded
Open this post in threaded view
|

Re: SSL_CTX_set_cert_verify_callback and certificate access

Corey Minyard
In reply to this post by Sam Roberts
On 1/11/19 12:42 PM, Sam Roberts wrote:

> On Wed, Jan 9, 2019 at 6:54 PM Corey Minyard <[hidden email]> wrote:
>> My first inclination for a secure connection was to use ssh. However,
>> ssh is not as well suited for this as I would have liked, and all the
>> ssh libraries are tied to a file descriptor in ways that are not easily
>> fixable, and thus can't be used on top of an abstract connection, which
>> is what I need.  That was rather disappointing, as it would have been
>> really nice to for users to just be able to ssh to ser2net.
> Not to second guess your finding that ssh isn't working well for you,
> you know your own code best, but for my own interest, I'm curious what
> about the fd is a problem? Perhaps the mismatch between X.509+TLS and
> the auth model you want are enough to reconsider your abstractions?
> Generating certs is pretty annoying and fragile, and using ssh clients
> is pretty easy!

Generating certs is easy if you do it like ssh does, and openssl is
quite capable of that.

The auth model is not the issue, though.  The problems I'm having are
plugging
in to openssl in the right places to do what I want.  But the help I've
received
here has got me through that, I think.


>
> It sounds like your are building the abstractions (in C?) inside the
> sernet process, but maybe your abstraction can be an fd, and the
> "layers" can be child processes that connect fd-to-fd, sortof
> qmail-like? Or, ssh should be able to execute an arbitrary command on
> the server, and that command should be able to do anything it wants
> with the ssh-facing socket descriptors, perhaps sending data to/from
> your server which can then move the data through the in-process
> abstractions?

The model I have is something like openssl and the BIOs.  You can plug
different
things together in openssl any way you like.  In each piece, you shove
data in
one side and data comes out the other.  You have BIOs at the end for dealing
with sockets or whatnot.  So getting openssl running inside my framework
was quite easy.

Both openssh and libssh are not designed that way.  There is no clean
separation
between dealing with file descriptors (that's what I meant by fd) and
the rest
of the library.  And there were a number of other issues, too.

Thanks,

-corey

--
openssl-users mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-users
Reply | Threaded
Open this post in threaded view
|

Re: SSL_CTX_set_cert_verify_callback and certificate access

Michael Wojcik
In reply to this post by Corey Minyard
> From: openssl-users [mailto:[hidden email]] On Behalf Of
> Corey Minyard
> Sent: Friday, January 11, 2019 17:09
>
> I don't really like my options, but I'm kind of boxed in.  I'm not sure
> why ssh doesn't run on top of ssl; that seems so sensible.  I assume
> that's for historical reasons.

Since SSH and SSL were developed simultaneously (both first relased in 1995), neither could be based on the other.

More importantly, SSH and SSL are completely different protocols, initially designed for different use cases and as cryptographically-secured replacements for different existing mechanisms. In their early versions they had very different approaches to PKI, and as they're commonly used they still do.

There are remote-shell and file-transfer applications built on TLS (Telnet-over-TLS and Telnet-with-STARTTLS[1], and similarly for FTP[2]). There's no real advantage to changing SSH to use the same architecture. It would force users away from the SSH PKI, which is widely abused[3]; but the X.509 PKIX is such a mess that it's hard to claim it's all that much better.

Asking why SSH isn't built on SSL is a bit like asking why TCP/IP isn't based on X.25. Similarity of intent doesn't indicate similarity of fitness for a particular purpose.


[1] Though as always with opportunistic TLS, you want a client which will fail safe by aborting the connection, or at least make it *very* clear to the user, when a secure channel cannot be established.

[2] Sometimes referred to as "SFTP" or "FTPS", but the nomenclature is inconsistent; people also use those for SSH-based file transfer, among other things.

[3] Anecdotal evidence and some surveys suggest many, likely most, SSH users practice poor key hygiene, accepting public keys without checking their provenance.

--
Michael Wojcik
Distinguished Engineer, Micro Focus



--
openssl-users mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-users