(future) STORE vs X509_LOOKUP_METHOD by_dir

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

(future) STORE vs X509_LOOKUP_METHOD by_dir

Richard Levitte - VMS Whacker-2
Hi,

I've some ponderings that I need to bounce a bit with you all.

Some have talked about replace the X509_LOOKUP_METHOD bit with the
STORE module I'm building, and while STORE isn't ready for it yet, I
have some thoughts on how the two can approach each other.  This would
involve one or two hooks / callbacks, that a STORE user could specify
(details later) to pick and choose freely among the objects that the
STORE module finds (be it on file or whatever else that can be
represented as a URI).

The troublesome part would be to try to mimic by_dir...  It highly
depends on the specified paths to really be directories, and that it
should find what it wants by adding very specific file names (a hash
of the subject name with a ".{n}" or ".r{n}" extension for X.509 certs
and for X.509 CRLs).  And sure, that works, but will really only work
with regular files.

What if someone would specify a LDAP URI that can return a bunch of
objects?

So...  my ponderings are going along these lines:

1. Should the directory X509_LOOKUPs be restricted to on disk
   directories, or should "directory" be redefined as "whatever URI
   that returns a collection of objects"?  The latter would mean that
   all those objects get loaded and that a hook / callback would then
   be called to check if it's an object that corresponds to what we
   search for.

2. For on disk directories, should we preserve the rehash file form?
   In other words, if we decide to load everything we can find, shall
   we restrict the loading to files matching the regexp
   [0-9a-f]{8}\.r?[0-9]+  ?  If not, are we about to create a new form
   of key store for ourselves and our users?  Should we?

Quite a lot also depends on what OpenSSL version we aim for.  I would
very much like to see the STORE module itself become part of 1.1.1,
but a new key store to replace our current rehash links will obviously
have to wait 'til 1.2.0.

Cheers,
Richard

--
Richard Levitte         [hidden email]
OpenSSL Project         http://www.openssl.org/~levitte/
--
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
Reply | Threaded
Open this post in threaded view
|

Re: (future) STORE vs X509_LOOKUP_METHOD by_dir

Tomas Mraz-2
On Sun, 2017-02-05 at 16:47 +0100, Richard Levitte wrote:

> Hi,
>
> I've some ponderings that I need to bounce a bit with you all.
>
> Some have talked about replace the X509_LOOKUP_METHOD bit with the
> STORE module I'm building, and while STORE isn't ready for it yet, I
> have some thoughts on how the two can approach each other.  This
> would
> involve one or two hooks / callbacks, that a STORE user could specify
> (details later) to pick and choose freely among the objects that the
> STORE module finds (be it on file or whatever else that can be
> represented as a URI).

Just to add something to your thinking - so there is a p11-kit-trust
PKCS11 module which provides all the CA certificates that should be
trusted on the system via individual PKCS11 certificate objects. Could
it be somehow accommodated with the STORE module approach? Mozilla NSS
and GnuTLS can use this PKCS11 module directly as a trust store, we
would like to add the same functionality to OpenSSL.

--
Tomas Mraz
No matter how far down the wrong road you've gone, turn back.
                                              Turkish proverb
(You'll never know whether the road is wrong though.)

--
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
Reply | Threaded
Open this post in threaded view
|

Re: (future) STORE vs X509_LOOKUP_METHOD by_dir

Roumen Petrov
In reply to this post by Richard Levitte - VMS Whacker-2
Hi Richard,

Richard Levitte wrote:
> Hi,
>
> I've some ponderings that I need to bounce a bit with you all.
>
> Some have talked about replace the X509_LOOKUP_METHOD
X.509 lookup method could return certificate , revocation list or
EVP_KEY (structure x509_object_st).

Unfortunately   functionality of EVP_KEY was never implemented.
Another point is specific names of structures - x509_lookup_method_st ,
x509_lookup_st, x509_object_st.
Third point is quite specific implementation - functions not just to
retrieve objects( X.509 or CRL) but to fill them into "context of X509
store".

Current lookup functionality look like "store" but implementation is
specific to X.509 store.


> bit with the
> STORE module I'm building, and while STORE isn't ready for it yet

I hope that you store functionality will fill gap between load of keys
and load of certificates (+crl).

Loadable module (engine) has interface to load key(private or public)  
but lack load of X.509 certificates or CRL.


> , I
> have some thoughts on how the two can approach each other.  This would
> involve one or two hooks / callbacks, that a STORE user could specify
> (details later) to pick and choose freely among the objects that the
> STORE module finds (be it on file or whatever else that can be
> represented as a URI).
I think that functionality requires three phases :
1) instantiation : at this point store is created
2) specification (optional):  set or check capability of store. For
instance store could return only X.509 certificates or to request store
to return only keys.
3) inquiry: fetch data based on specified criteria.

> The troublesome part would be to try to mimic by_dir...  It highly
> depends on the specified paths to really be directories, and that it
> should find what it wants by adding very specific file names (a hash
> of the subject name with a ".{n}" or ".r{n}" extension for X.509 certs
> and for X.509 CRLs).  And sure, that works, but will really only work
> with regular files.
I'm not sure what is issue.

Lets see X.509 lookup method get_by_subject.

- by_dir
2) specification : set directory(path), limit results to X.509 or CRL
and may be to inform store that questions will be performed by subject.

For instance URI scheme could befile://path?certificate="name"

3) query : from subject calculate hash and then process "{hash}.{n}" or
"{hash}.r{n}" depending from URI

- Ldap
It is similar, URI is described in RFCs - at point 2) set host, port, base distinguished name, attribute (for instance cACertificate), construct filter from specified name.



> What if someone would specify a LDAP URI that can return a bunch of
> objects?
>
> So...  my ponderings are going along these lines:
>
> 1. Should the directory X509_LOOKUPs be restricted to on disk
>     directories, or should "directory" be redefined as "whatever URI
>     that returns a collection of objects"?  The latter would mean that
>     all those objects get loaded and that a hook / callback would then
>     be called to check if it's an object that corresponds to what we
>     search for.
I think that replacement of "by_dir" lookup has to be restricted to file
system operation.
Files could be located on network, memory not only on disk.


> 2. For on disk directories, should we preserve the rehash file form?
>     In other words, if we decide to load everything we can find, shall
>     we restrict the loading to files matching the regexp
>     [0-9a-f]{8}\.r?[0-9]+  ?  If not, are we about to create a new form
>     of key store for ourselves and our users?  Should we?

For hash-dir please keep current file name format.

> Quite a lot also depends on what OpenSSL version we aim for.  I would
> very much like to see the STORE module itself become part of 1.1.1,
> but a new key store to replace our current rehash links will obviously
> have to wait 'til 1.2.0.
>
> Cheers,
> Richard
>
Roumen
--
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
Reply | Threaded
Open this post in threaded view
|

Re: (future) STORE vs X509_LOOKUP_METHOD by_dir

Richard Levitte - VMS Whacker-2
In message <[hidden email]> on Wed, 08 Feb 2017 22:59:45 +0200, Roumen Petrov <[hidden email]> said:

openssl> Hi Richard,
openssl>
openssl> Richard Levitte wrote:
openssl> > Hi,
openssl> >
openssl> > I've some ponderings that I need to bounce a bit with you all.
openssl> >
openssl> > Some have talked about replace the X509_LOOKUP_METHOD
openssl> X.509 lookup method could return certificate , revocation list or
openssl> EVP_KEY (structure x509_object_st).
openssl>
openssl> Unfortunately   functionality of EVP_KEY was never implemented.
openssl> Another point is specific names of structures - x509_lookup_method_st
openssl> , x509_lookup_st, x509_object_st.
openssl> Third point is quite specific implementation - functions not just to
openssl> retrieve objects( X.509 or CRL) but to fill them into "context of X509
openssl> store".
openssl>
openssl> Current lookup functionality look like "store" but implementation is
openssl> specific to X.509 store.
openssl>
openssl>
openssl> > bit with the
openssl> > STORE module I'm building, and while STORE isn't ready for it yet
openssl>
openssl> I hope that you store functionality will fill gap between load of keys
openssl> and load of certificates (+crl).
openssl>
openssl> Loadable module (engine) has interface to load key(private or public)
openssl> but lack load of X.509 certificates or CRL.

So far, key parameters, pkeys, certs and crls are covered...  oh, and
names, if the given URI would return a list of names (file:/foo/bar/
would typically do that, for example).

openssl> > , I
openssl> > have some thoughts on how the two can approach each other.  This would
openssl> > involve one or two hooks / callbacks, that a STORE user could specify
openssl> > (details later) to pick and choose freely among the objects that the
openssl> > STORE module finds (be it on file or whatever else that can be
openssl> > represented as a URI).
openssl> I think that functionality requires three phases :
openssl> 1) instantiation : at this point store is created
openssl> 2) specification (optional): set or check capability of store. For
openssl> instance store could return only X.509 certificates or to request
openssl> store to return only keys.
openssl> 3) inquiry: fetch data based on specified criteria.

Previous attempts at a STORE effort were along those lines of
thinking.  Unfortunately, they got stuck in the exact specification of
attributes and how to combine them (i.e. forming a "language" of
sorts), and then to figure out how to make that pratical, especially
for the engines that would have to intrepret it.

So this time around, I'm trying to make OpenSSL fairly agnostic and
leave the exact spec up to be specified by the user or application in
the URI and letting the diverse engines interpret the URI components
at their leasure.  Objects would then be returned according to those
specs in form of one object at a time, be it parameters, keys, certs,
crls or names, for the application or other parts of OpenSSL to do
whatever they wish with.  Given this, I determined yesterday that
there would really just be one hook / callback that makes sense, and
that's one that can massage or possibly discard returned objects
before they make their way all the way to the calling application.
Something like this.

openssl> > The troublesome part would be to try to mimic by_dir...  It highly
openssl> > depends on the specified paths to really be directories, and that it
openssl> > should find what it wants by adding very specific file names (a hash
openssl> > of the subject name with a ".{n}" or ".r{n}" extension for X.509 certs
openssl> > and for X.509 CRLs).  And sure, that works, but will really only work
openssl> > with regular files.
openssl> I'm not sure what is issue.
openssl>
openssl> Lets see X.509 lookup method get_by_subject.
openssl>
openssl> - by_dir
openssl> 2) specification : set directory(path), limit results to X.509 or CRL
openssl> and may be to inform store that questions will be performed by
openssl> subject.
openssl>
openssl> For instance URI scheme could befile://path?certificate="name"
openssl>
openssl> 3) query : from subject calculate hash and then process "{hash}.{n}"
openssl> or "{hash}.r{n}" depending from URI
openssl>
openssl> - Ldap
openssl> It is similar, URI is described in RFCs - at point 2) set host, port,
openssl> base distinguished name, attribute (for instance cACertificate),
openssl> construct filter from specified name.

The issue is there, glaring back at you.  How will by_dir know exactly
how to massage the URI?  Sure, we could program in the big known ones
plus our own mangling of the file: scheme, but then what?  What about
the vendor who runs his own scheme foo:?  How do we know how to encode
their keystore specs into a URI with their scheme?

There needs to be a solution where OpenSSL can be specification
agnostic.  So far, I've this idea of returned names that form a
"directory listing", and it wouldn't be too hard to imagine a by_dir
implementation that simply get a list of names, then loads each of
them and caches *everything*, or possibly one that uses the hook /
callback I mentioned further up to select the one with the subject
it's looking for.  It is workable, I'm just worried about the sudden
huge "directory listing".

openssl> > What if someone would specify a LDAP URI that can return a bunch of
openssl> > objects?
openssl> >
openssl> > So...  my ponderings are going along these lines:
openssl> >
openssl> > 1. Should the directory X509_LOOKUPs be restricted to on disk
openssl> >     directories, or should "directory" be redefined as "whatever URI
openssl> >     that returns a collection of objects"?  The latter would mean that
openssl> >     all those objects get loaded and that a hook / callback would then
openssl> >     be called to check if it's an object that corresponds to what we
openssl> >     search for.
openssl> I think that replacement of "by_dir" lookup has to be restricted to
openssl> file system operation.

That's workable but sub-optimal...  we've had a recent complaint from
someone who wants to create a X509_LOOKUP_METHOD for his own scheme
(hw protected keys, I think).  PKCS#11 has been mentioned quite a
number of times as well.  But sure, for the time being.

openssl> Files could be located on network, memory not only on disk.

Does that differ from what we have today?

openssl> > 2. For on disk directories, should we preserve the rehash file form?
openssl> >     In other words, if we decide to load everything we can find, shall
openssl> >     we restrict the loading to files matching the regexp
openssl> >     [0-9a-f]{8}\.r?[0-9]+  ?  If not, are we about to create a new form
openssl> >     of key store for ourselves and our users?  Should we?
openssl>
openssl> For hash-dir please keep current file name format.

Like I said, a changed keystore format isn't possible before 1.2.0
anyway, so at least for now, the hash-dir format stays.

What I wonder, generally speaking, is that for a URI that's a
directory spec, show the STORE file: scheme return the names of *all*
the file names in that directory, or just those matching
[0-9a-f]{8}\.r?[0-9]+  ?  Either can be done, I just worry about the
consequences...

openssl> > Quite a lot also depends on what OpenSSL version we aim for.  I would
openssl> > very much like to see the STORE module itself become part of 1.1.1,
openssl> > but a new key store to replace our current rehash links will obviously
openssl> > have to wait 'til 1.2.0.

--
Richard Levitte         [hidden email]
OpenSSL Project         http://www.openssl.org/~levitte/
--
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
Reply | Threaded
Open this post in threaded view
|

Possible wrong restore register order in SEH for ecp_nistz256

Jun Sun

Hi,

I noticed Windows exception handler is added to ecp_nistz256-x86_64.pl. In function full_handler, when restore register contents, rbx is the first one with the smallest offset:


3181         mov     -8(%rax),%rbx

3182         mov     -16(%rax),%rbp


but when do push, rbp is always the first one push to the stack:

 497 .Lmul_mont:

 498         push    %rbp

 499         push    %rbx


That is, the contents of rbx is restored to rbp and the content of rbp is restored to rbx. I think it is probably a bug of the code.



Jun


This email and any attachments are for the sole use of the intended recipients and may be privileged or confidential. Any distribution, printing or other use by anyone else is prohibited. If you are not an intended recipient, please contact the sender immediately, and permanently delete this email and attachments.
--
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
Reply | Threaded
Open this post in threaded view
|

Re: Possible wrong restore register order in SEH for ecp_nistz256

Salz, Rich
Does this fix it?
   https://github.com/openssl/openssl/pull/2582

--
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
Reply | Threaded
Open this post in threaded view
|

Re: Possible wrong restore register order in SEH for ecp_nistz256

Andy Polyakov-2
> Does this fix it?
>    https://github.com/openssl/openssl/pull/2582

It's unrelated issues. Yes, it's typo in full_handler, will be fixed
[tomorrow]...
--
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
Reply | Threaded
Open this post in threaded view
|

Re: Possible wrong restore register order in SEH for ecp_nistz256

Andy Polyakov-2
>> Does this fix it?
>>    https://github.com/openssl/openssl/pull/2582
>
> It's unrelated issues. Yes, it's typo in full_handler, will be fixed
> [tomorrow]...

Fix has been applied. Thank you for report!


--
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev