Considering SSL and Cryto libraries for LSB

classic Classic list List threaded Threaded
20 messages Options
Reply | Threaded
Open this post in threaded view
|

Considering SSL and Cryto libraries for LSB

Banginwar, Rajesh
We recently started looking at some of Desktop specific libraries for
including them as part of next LSB (Linux Standard Base
www.linuxbase.org) release. This is part of the effort in the newly
formed Desktop working group in LSB to focus on the Desktop Linux
applications.
 
As a part of this effort we have started looking at the OpenSSL
libraries and found the thread
http://www.mail-archive.com/openssl-dev@.../msg19158.html
discussing ABI compatibility across different releases.  I am
specifically referring to comments from Richard below.
 
I will like to find out what the Openssl community think about making
the libraries part of next LSB release. Of course, the ABIs will need to
become stable before that. What are the plans to achieve this ABI
stability (including releasing 1.0 version of the package)?
 
Thanks,
 
-Rajesh
 
------------
The biggest change that's needed in OpenSSL is to hide all the
structures and all constants and have them available through functions
(creator, destructors and information functions).  So speaking of
incompatibilites, we've really kept it low compared to what needs to
be done and what could be done.

Our version numbering is admitedly weird.  Basically, we've treated
'0.9.' as a prefix to signal that "this isn't a 1.0 yet, and drastic
changes can be expected", and effectively trated the next digit as a
classic major version.  This is reflected in the soname we give the
shared libraries.  We probably should do some drastic changes in our
version numbering (which is quite a lesson to me personally.  I've
been reluctant to make a move to 1.0 because OpenSSL hasn't felt ready
for that).
-------
______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
Development Mailing List                       [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: Considering SSL and Cryto libraries for LSB

Dr. Stephen Henson
On Wed, Jun 29, 2005, Banginwar, Rajesh wrote:

> We recently started looking at some of Desktop specific libraries for
> including them as part of next LSB (Linux Standard Base
> www.linuxbase.org) release. This is part of the effort in the newly
> formed Desktop working group in LSB to focus on the Desktop Linux
> applications.
>  
> As a part of this effort we have started looking at the OpenSSL
> libraries and found the thread
> http://www.mail-archive.com/openssl-dev@.../msg19158.html
> discussing ABI compatibility across different releases.  I am
> specifically referring to comments from Richard below.
>  
> I will like to find out what the Openssl community think about making
> the libraries part of next LSB release. Of course, the ABIs will need to
> become stable before that. What are the plans to achieve this ABI
> stability (including releasing 1.0 version of the package)?
>  

This "ABI stability" is not something which can be done quickly and not
without some rather large levels of incompatibility with existing applications
itself.

One of many reason for this is that some APIs, which have been around since the
SSLeay days, make use of structures like this:

FOO_CTX foo;
FOO_CTX_something(&foo);

Examples of this are the EVP digest and cipher APIs.

This is a problem if the size of the structure FOO_CTX increases. There are
several reasons why the size of FOO_CTX might change including additional
features being added. There are several features which are needed in the
digest and cipher APIs which would have this effect.

There is a solution which already exists which is instead to do this:

FOO_CTX *foo;
foo = FOO_CTX_new();
FOO_CTX_something(foo);

However, very few applications (at present) use this technique.

This means that changing this in the short term is likely to cause widespread
application breakage which wouldn't be too popular :-(

Steve.
--
Dr Stephen N. Henson. Email, S/MIME and PGP keys: see homepage
OpenSSL project core developer and freelance consultant.
Funding needed! Details on homepage.
Homepage: http://www.drh-consultancy.demon.co.uk
______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
Development Mailing List                       [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

RE: Considering SSL and Cryto libraries for LSB

Banginwar, Rajesh
In reply to this post by Banginwar, Rajesh
As part of LSB standardization process, we look at the interfaces and
corresponding data types and make it part of the specification. If the
data types are expected to change and the interfaces do not hide them,
then that part of the library may not be a candidate for LSB.

Given this, I am just wondering (as I do not know enough details yet) if
it makes sense to partially include the crypto library in LSB. I assume
at least parts of this library may be stable and not expected to change
much. E.g. some of the crypto algorithms.

Is that a possibility? From our current analysis of some of the existing
OSS applications, function sets like EVP*, RSA*, DSA* etc are quite
commonly used. It may be a good idea to identfy some of the "stable"
interfaces from crypto library for inclusion in LSB. Any
thoughts/comments?

Thanks,

-Rajesh

-----Original Message-----
From: [hidden email]
[mailto:[hidden email]] On Behalf Of Dr. Stephen Henson
Sent: Wednesday, June 29, 2005 1:01 PM
To: [hidden email]
Subject: Re: Considering SSL and Cryto libraries for LSB

On Wed, Jun 29, 2005, Banginwar, Rajesh wrote:

> We recently started looking at some of Desktop specific libraries for
> including them as part of next LSB (Linux Standard Base
> www.linuxbase.org) release. This is part of the effort in the newly
> formed Desktop working group in LSB to focus on the Desktop Linux
> applications.
>  
> As a part of this effort we have started looking at the OpenSSL
> libraries and found the thread
> http://www.mail-archive.com/openssl-dev@.../msg19158.html
> discussing ABI compatibility across different releases.  I am
> specifically referring to comments from Richard below.
>  
> I will like to find out what the Openssl community think about making
> the libraries part of next LSB release. Of course, the ABIs will need
to
> become stable before that. What are the plans to achieve this ABI
> stability (including releasing 1.0 version of the package)?
>  

This "ABI stability" is not something which can be done quickly and not
without some rather large levels of incompatibility with existing
applications
itself.

One of many reason for this is that some APIs, which have been around
since the
SSLeay days, make use of structures like this:

FOO_CTX foo;
FOO_CTX_something(&foo);

Examples of this are the EVP digest and cipher APIs.

This is a problem if the size of the structure FOO_CTX increases. There
are
several reasons why the size of FOO_CTX might change including
additional
features being added. There are several features which are needed in the
digest and cipher APIs which would have this effect.

There is a solution which already exists which is instead to do this:

FOO_CTX *foo;
foo = FOO_CTX_new();
FOO_CTX_something(foo);

However, very few applications (at present) use this technique.

This means that changing this in the short term is likely to cause
widespread
application breakage which wouldn't be too popular :-(

Steve.
--
Dr Stephen N. Henson. Email, S/MIME and PGP keys: see homepage
OpenSSL project core developer and freelance consultant.
Funding needed! Details on homepage.
Homepage: http://www.drh-consultancy.demon.co.uk
______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
Development Mailing List                       [hidden email]
Automated List Manager                           [hidden email]
______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
Development Mailing List                       [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: Considering SSL and Cryto libraries for LSB

Dr. Stephen Henson
On Wed, Jun 29, 2005, Banginwar, Rajesh wrote:

> As part of LSB standardization process, we look at the interfaces and
> corresponding data types and make it part of the specification. If the
> data types are expected to change and the interfaces do not hide them,
> then that part of the library may not be a candidate for LSB.
>
> Given this, I am just wondering (as I do not know enough details yet) if
> it makes sense to partially include the crypto library in LSB. I assume
> at least parts of this library may be stable and not expected to change
> much. E.g. some of the crypto algorithms.
>
> Is that a possibility? From our current analysis of some of the existing
> OSS applications, function sets like EVP*, RSA*, DSA* etc are quite
> commonly used. It may be a good idea to identfy some of the "stable"
> interfaces from crypto library for inclusion in LSB. Any
> thoughts/comments?
>

Well some of the low level crypto APIs haven't changed for some time but their
use is discouraged because it ties an application down to a specific algorithm
and a specific implementation.

EVP is the preferred interface for accessing crypto algorithms and digests.
Both these however commonly use the construct I mentioned which causes binary
compatibility issues.

That doesn't mean that EVP can't be used in a way that pretty much guarantees
binary compatibility: its just that existing applications don't often use it
in that way.

So its certainly feasible to say that "if you use EVP like *this*" it wont
cause problems.

Similar comments apply to RSA and DSA: there are ways to use the APIs that
avoid structure access and possible future incompatibilities.

There are however some areas where the API is lacking: for example there isn't
currently an easy way to populate an RSA structure with its key components.

That is there isn't a function:

RSA_set_PublicKey(rsa, n, e);

but that's not hard to fix.

That all applies to applications that use existing algorithms that are part of
OpenSSL (or which have been loaded in via an ENGINE).

Applications which extend OpenSSL by defining new cryptographic algorithms
currently need to access internal structures directly. APIs can of course be
developed that remedy that situation, but again existing applications wont use
them.

Steve.
--
Dr Stephen N. Henson. Email, S/MIME and PGP keys: see homepage
OpenSSL project core developer and freelance consultant.
Funding needed! Details on homepage.
Homepage: http://www.drh-consultancy.demon.co.uk
______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
Development Mailing List                       [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: Considering SSL and Cryto libraries for LSB

Peter Waltenberg

IBM has already done this in creating it's FIPS certified crypto. code which is layered on top of OpenSSL.

In our case we can guarantee that IBM code only uses our restricted subset of the OpenSSL API.
Unfortunately you'll need to support the older API's to support legacy applications and in won't be possible to ensure that everyone writing new code restricts themselves to the sanitized API's.

If you can live with that then the only problem will be getting the necessary changes made to provide a clean API. I'm not claiming that'll be trivial, but we already know it is possible.

Peter




"Dr. Stephen Henson" <[hidden email]>
Sent by: [hidden email]

30/06/2005 08:14 AM

Please respond to
openssl-dev

To
[hidden email]
cc
Subject
Re: Considering SSL and Cryto libraries for LSB





On Wed, Jun 29, 2005, Banginwar, Rajesh wrote:

> As part of LSB standardization process, we look at the interfaces and
> corresponding data types and make it part of the specification. If the
> data types are expected to change and the interfaces do not hide them,
> then that part of the library may not be a candidate for LSB.
>
> Given this, I am just wondering (as I do not know enough details yet) if
> it makes sense to partially include the crypto library in LSB. I assume
> at least parts of this library may be stable and not expected to change
> much. E.g. some of the crypto algorithms.
>
> Is that a possibility? From our current analysis of some of the existing
> OSS applications, function sets like EVP*, RSA*, DSA* etc are quite
> commonly used. It may be a good idea to identfy some of the "stable"
> interfaces from crypto library for inclusion in LSB. Any
> thoughts/comments?
>

Well some of the low level crypto APIs haven't changed for some time but their
use is discouraged because it ties an application down to a specific algorithm
and a specific implementation.

EVP is the preferred interface for accessing crypto algorithms and digests.
Both these however commonly use the construct I mentioned which causes binary
compatibility issues.

That doesn't mean that EVP can't be used in a way that pretty much guarantees
binary compatibility: its just that existing applications don't often use it
in that way.

So its certainly feasible to say that "if you use EVP like *this*" it wont
cause problems.

Similar comments apply to RSA and DSA: there are ways to use the APIs that
avoid structure access and possible future incompatibilities.

There are however some areas where the API is lacking: for example there isn't
currently an easy way to populate an RSA structure with its key components.

That is there isn't a function:

RSA_set_PublicKey(rsa, n, e);

but that's not hard to fix.

That all applies to applications that use existing algorithms that are part of
OpenSSL (or which have been loaded in via an ENGINE).

Applications which extend OpenSSL by defining new cryptographic algorithms
currently need to access internal structures directly. APIs can of course be
developed that remedy that situation, but again existing applications wont use
them.

Steve.
--
Dr Stephen N. Henson. Email, S/MIME and PGP keys: see homepage
OpenSSL project core developer and freelance consultant.
Funding needed! Details on homepage.
Homepage: http://www.drh-consultancy.demon.co.uk
______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
Development Mailing List                       [hidden email]
Automated List Manager                           [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Considering SSL and Cryto libraries for LSB

Dan Nuffer
In reply to this post by Dr. Stephen Henson
Dr. Stephen Henson wrote:
> This means that changing this in the short term is likely to cause widespread
> application breakage which wouldn't be too popular :-(
>

Speaking as an application developer, I would willingly go through a
one-time source code upgrade to achieve binary compatiblity.  That is
more appealing to me than having to repeatedly deal with binary
compatiblity issues, because it's less work in the long run.

--
Dan Nuffer
Vintela, Inc. http://vintela.com/


______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
Development Mailing List                       [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: Considering SSL and Cryto libraries for LSB

Dr. Stephen Henson
In reply to this post by Peter Waltenberg
On Thu, Jun 30, 2005, Peter Waltenberg wrote:

> IBM has already done this in creating it's FIPS certified crypto. code
> which is layered on top of OpenSSL.
>
> In our case we can guarantee that IBM code only uses our restricted subset
> of the OpenSSL API.
> Unfortunately you'll need to support the older API's to support legacy
> applications and in won't be possible to ensure that everyone writing new
> code restricts themselves to the sanitized API's.
>

Well we could provide compilation options which would result in some
applications using non-portable constructs producing warnings or errors.

For example the problematical:

EVP_CIPHER_CTX ctx;

will produce a compilation error if the definition of EVP_CIPHER_CTX is not
public. Wheareas:

EVP_CIPHER_CTX *ctx;

will be OK.

Legacy applications are a problem because some behaviour doesn't even have an
appropriate API at present.

For example so set the public key components you currently have to do:

rsa->n = n;
rsa->e = e;

which is a no-no.

> If you can live with that then the only problem will be getting the
> necessary changes made to provide a clean API. I'm not claiming that'll be
> trivial, but we already know it is possible.
>

Its certainly possible. There is an added complication in that some thought has
to be given to future directions of development.

For example the current RSA API lacks any easy way to pass additional
parameters to some padding types. OAEP and especially PSS really need a way to
do that.
       
That could be neatly solved (and some other parts made more efficient) if RSA
had a per-thread context structure analagous to EVP_CIPHER_CTX but alas it
doesn't.

Steve.
--
Dr Stephen N. Henson. Email, S/MIME and PGP keys: see homepage
OpenSSL project core developer and freelance consultant.
Funding needed! Details on homepage.
Homepage: http://www.drh-consultancy.demon.co.uk
______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
Development Mailing List                       [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: Considering SSL and Cryto libraries for LSB

Dr. Stephen Henson
In reply to this post by Dan Nuffer
On Wed, Jun 29, 2005, Dan Nuffer wrote:

> Dr. Stephen Henson wrote:
> >This means that changing this in the short term is likely to cause
> >widespread
> >application breakage which wouldn't be too popular :-(
> >
>
> Speaking as an application developer, I would willingly go through a
> one-time source code upgrade to achieve binary compatiblity.  That is
> more appealing to me than having to repeatedly deal with binary
> compatiblity issues, because it's less work in the long run.
>

Also speaking as an application developer so would I. However there are a hell
of a lot of applications out there that depend on the existing APIs not
changing (too) much.

Can you give details of some of the binary compatibility issues you've come
across?

Steve.
--
Dr Stephen N. Henson. Email, S/MIME and PGP keys: see homepage
OpenSSL project core developer and freelance consultant.
Funding needed! Details on homepage.
Homepage: http://www.drh-consultancy.demon.co.uk
______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
Development Mailing List                       [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: Considering SSL and Cryto libraries for LSB

Geoff Thorpe-2
In reply to this post by Banginwar, Rajesh
On June 29, 2005 05:50 pm, Banginwar, Rajesh wrote:
> As part of LSB standardization process, we look at the interfaces and
> corresponding data types and make it part of the specification. If the
> data types are expected to change and the interfaces do not hide them,
> then that part of the library may not be a candidate for LSB.

Well there are two things that spring to mind.

1. Most "use" of openssl, at least as might be considered useful from an
LSB-perspective, is of the SSL layer itself (libssl). This requires the
direct use of certain APIs in the underlying libcrypto layer too, but
AFAICT these would be easier to standardise on than the full set of APIs
and are probably more ABI-friendly (ie. loading and initialising RSA/X509
objects, etc). NB, that's a cursory thought, I've not attempted to dig
terribly far on this.

2. This is a refined variation on the issues distributions already face
when distributing openssl in shared-library form and maintaining classes
of applications that have dependencies on these libs. The consensus seems
to be to use versioning to manage this, where a system can have multiple
openssl libraries installed of differing version levels. Eg. 0.9.6,
0.9.7, etc. Bugfix releases to individual versions are expected to
maintain binary compatibility, or distributors at least Q/A this at their
end and patch around any issues as/where necessary. You'd have to check
with the distributions themselves to know more though, that's out of our
hands. Mark, any comments from a Redhat perspective?

The reasons Steve outlined cover why we can't provide assurances about
full ABI compatibility (nor even API compatibility) in a long-term sense.
To do so would take a lot of undesirable but legacy aspects of the
toolkit and fix them in place. However, the unwritten rule is to ensure
that each incremental(/"bugfix") release of a given stable branch does
preserve binary compatibility, and incorporation into LSB definitions
would obviously go a long way turning this into a firmer commitment. Note
also that major releases, which we freely admit can break ABI and/or API
compatibility where required, are made far less frequently - as a perusal
of release history will aptly demonstrate. I should also mention that
we're not making vast overhauls of APIs on a regular basis, nor for the
mere fun of it, so if adaptations are required long-term to allow future
LSB revisions to incorporate newer release levels, this would hardly
involve major surgery. Perhaps that's an avenue until such time as
openssl does get to some kind of 1.0 plateau. Hope springs eternal.

[ Or people wanting LSB compliancy can always statically link any version
of openssl they like, but I guess that's not what you're after :-) ]

Cheers,
Geoff

--
Geoff Thorpe
[hidden email]
http://www.openssl.org/

______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
Development Mailing List                       [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

RE: Considering SSL and Cryto libraries for LSB

Banginwar, Rajesh
In reply to this post by Banginwar, Rajesh
> -----Original Message-----
> From: Geoff Thorpe [mailto:[hidden email]] On Behalf
> Of Geoff Thorpe
> Sent: Wednesday, June 29, 2005 5:45 PM
> To: [hidden email]
> Cc: Banginwar, Rajesh
> Subject: Re: Considering SSL and Cryto libraries for LSB
>
> On June 29, 2005 05:50 pm, Banginwar, Rajesh wrote:
> > As part of LSB standardization process, we look at the
> interfaces and
> > corresponding data types and make it part of the
> specification. If the
> > data types are expected to change and the interfaces do not
> hide them,
> > then that part of the library may not be a candidate for LSB.
>
> Well there are two things that spring to mind.
>
> 1. Most "use" of openssl, at least as might be considered
> useful from an
> LSB-perspective, is of the SSL layer itself (libssl). This
> requires the
> direct use of certain APIs in the underlying libcrypto layer too, but
> AFAICT these would be easier to standardise on than the full
> set of APIs
> and are probably more ABI-friendly (ie. loading and
> initialising RSA/X509
> objects, etc). NB, that's a cursory thought, I've not
> attempted to dig
> terribly far on this.

So far from the preliminary analysis that we have done (by looking at
some of the OSS applications) we see both libssl and libcrypto being
used. E.g. from libcrypto I find functions in EVP, RSA, MD5 and DSA sets
more commonly used than others. Unfortunately this is not an exhaustive
study. Do you or anyone on this project have data suggesting which APIs
are candidates for LSB inclusion both from demand and stability point of
view?

>
> 2. This is a refined variation on the issues distributions
> already face
> when distributing openssl in shared-library form and
> maintaining classes
> of applications that have dependencies on these libs. The
> consensus seems
> to be to use versioning to manage this, where a system can
> have multiple
> openssl libraries installed of differing version levels. Eg. 0.9.6,
> 0.9.7, etc. Bugfix releases to individual versions are expected to
> maintain binary compatibility, or distributors at least Q/A
> this at their
> end and patch around any issues as/where necessary. You'd
> have to check
> with the distributions themselves to know more though, that's
> out of our
> hands. Mark, any comments from a Redhat perspective?
>
> The reasons Steve outlined cover why we can't provide
> assurances about
> full ABI compatibility (nor even API compatibility) in a
> long-term sense.
> To do so would take a lot of undesirable but legacy aspects of the
> toolkit and fix them in place. However, the unwritten rule is
> to ensure
> that each incremental(/"bugfix") release of a given stable
> branch does
> preserve binary compatibility, and incorporation into LSB definitions
> would obviously go a long way turning this into a firmer
> commitment. Note
> also that major releases, which we freely admit can break ABI
> and/or API
> compatibility where required, are made far less frequently -
> as a perusal
> of release history will aptly demonstrate. I should also mention that
> we're not making vast overhauls of APIs on a regular basis,
> nor for the
> mere fun of it, so if adaptations are required long-term to
> allow future
> LSB revisions to incorporate newer release levels, this would hardly
> involve major surgery. Perhaps that's an avenue until such time as
> openssl does get to some kind of 1.0 plateau. Hope springs eternal.
>


I just checked the release history for openssl and it is encouraging to
see the major releases are years apart. I see that with some efforts it
may be possible to hide some of the data structure dependencies (as
Steve pointed out in other emails on this thread) and get some sets of
APIs to a "stable" level and push for LSB inclusion. There was at least
one comment from application developer encouraging this even though it
will require major changes.


> [ Or people wanting LSB compliancy can always statically link
> any version
> of openssl they like, but I guess that's not what you're after :-) ]
>
> Cheers,
> Geoff
>
> --
> Geoff Thorpe
> [hidden email]
> http://www.openssl.org/
>
>
______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
Development Mailing List                       [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: Considering SSL and Cryto libraries for LSB

Geoff Thorpe-2
On June 29, 2005 08:44 pm, Banginwar, Rajesh wrote:
> So far from the preliminary analysis that we have done (by looking at
> some of the OSS applications) we see both libssl and libcrypto being
> used. E.g. from libcrypto I find functions in EVP, RSA, MD5 and DSA
> sets more commonly used than others. Unfortunately this is not an
> exhaustive study. Do you or anyone on this project have data suggesting
> which APIs are candidates for LSB inclusion both from demand and
> stability point of view?

I'm not sure how you would harvest this kind of information. From a build
perspective, it may be quite difficult to disentangle one API from the
next. I guess someone truly masoc^H^H^H^H^Hdedicated could create an
alternative "openssl-lsb.h" header to build against - one that declared
only the subset of functionality that mattered from an LSB perspective
and then see how external apps fare building against that. However
there's a lot of macro-abuse in some areas of the code, and if this
"shimming" got tangled up in trying to reproduce ASN1 definitions then I
don't even want to contemplate where it might lead...

Something to think about I guess. But it certainly seems hairy to attempt
to standardise too much invariance across major releases that we expect
will ... um ... vary.

> I just checked the release history for openssl and it is encouraging to
> see the major releases are years apart. I see that with some efforts it
> may be possible to hide some of the data structure dependencies (as
> Steve pointed out in other emails on this thread) and get some sets of
> APIs to a "stable" level and push for LSB inclusion. There was at least
> one comment from application developer encouraging this even though it
> will require major changes.

It sounds like a fairly labourious process though (ie. I doubt we'll be
dispersing the volunteers with tear-gas or anything).

Cheers,
Geoff

--
Geoff Thorpe
[hidden email]
http://www.openssl.org/

______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
Development Mailing List                       [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: Considering SSL and Cryto libraries for LSB

Richard Levitte - VMS Whacker
In reply to this post by Banginwar, Rajesh
In message <[hidden email]> on Wed, 29 Jun 2005 17:44:38 -0700, "Banginwar, Rajesh" <[hidden email]> said:

rajesh.banginwar> Do you or anyone on this project have data
rajesh.banginwar> suggesting which APIs are candidates for LSB
rajesh.banginwar> inclusion both from demand and stability point of
rajesh.banginwar> view?

Quick answer, solely based on the header files and looking for the
parts that do not expose there structures: EC, ECDH, ECDSA (although
it exposes the signature structure, but I think that one's
standardardised), pqueue, UI.  ENGINE should also be here even though
there are some exposed structures.  Those structures are fairly well
defined and are not subject to change soon, as far as I can predict.

Quite honestly, even though I'm quite an enthusiastic OpenSSL
developer for years and have been for years (since it started,
really), I can't really recommend OpenSSL as an LSB candidate from
that point of view, as it stands today.  Every "major upgrade" (which
we define as a change of x in 0.9.x) has had some kind of
incompatibility with previous versions.

Cheers,
Richard

-----
Please consider sponsoring my work on free software.
See http://www.free.lp.se/sponsoring.html for details.

--
Richard Levitte                         [hidden email]
                                        http://richard.levitte.org/

"When I became a man I put away childish things, including
 the fear of childishness and the desire to be very grown up."
                                                -- C.S. Lewis
______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
Development Mailing List                       [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: Considering SSL and Cryto libraries for LSB

Dan Kegel-2
Richard Levitte - VMS Whacker wrote:
> Quite honestly, even though I'm quite an enthusiastic OpenSSL
> developer for years and have been for years (since it started,
> really), I can't really recommend OpenSSL as an LSB candidate from
> that point of view, as it stands today.  Every "major upgrade" (which
> we define as a change of x in 0.9.x) has had some kind of
> incompatibility with previous versions.

http://www.gnu.org/software/gnutls/
exposes two APIs: the OpenSSL api (I gather?),
and its own.
Has anyone looked at gnutls's api to see if it exposes fewer structures?
If so, perhaps that might provide a way forward:
apps that need a stable interface can use the gnutls
api (which openssl could provide as a wrapper);
everyone else could use the openssl api (which
gnutls seems to provide as a wrapper, unless I
misread the docs).
- Dan

--
Trying to get a job as a c++ developer?  See http://kegel.com/academy/getting-hired.html
______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
Development Mailing List                       [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

RE: Considering SSL and Cryto libraries for LSB

Rich Salz
In reply to this post by Banginwar, Rajesh
What is the benefit of adding parts of OpenSSL to the LSB now?


--
Rich Salz                  Chief Security Architect
DataPower Technology       http://www.datapower.com
XS40 XML Security Gateway  http://www.datapower.com/products/xs40.html

______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
Development Mailing List                       [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: Considering SSL and Cryto libraries for LSB

Richard Levitte - VMS Whacker
In reply to this post by Dan Kegel-2
In message <[hidden email]> on Wed, 29 Jun 2005 22:05:07 -0700, Dan Kegel <[hidden email]> said:

dank> http://www.gnu.org/software/gnutls/
dank> exposes two APIs: the OpenSSL api (I gather?), and its own.

About the OpenSSL API, this page answers part of the question.

  http://www.gnu.org/software/gnutls/reference/gnutls-openssl.html

The rest of the answer is in gnutls/openssl.h.  They expose some
structures to remain compatible with the way OpenSSL currently works,
so it's basically a compatibility that's as stripped down as
possible.

For the rest of GnuTLS, they seem to expose very little, from what I
can gather by looking at the public header files.

dank> If so, perhaps that might provide a way forward: apps that need
dank> a stable interface can use the gnutls api (which openssl could
dank> provide as a wrapper); everyone else could use the openssl api
dank> (which gnutls seems to provide as a wrapper, unless I misread
dank> the docs).

It's a path.  Just a small warning about license politics: According
to http://www.gnu.org/software/gnutls/gnutls.html, the GnuTLS core
library is licensed under the LGPL.  Looking at the header files, it
looks like there's a mix of GPL and LGPL, and among others, their
openssl.h is under the GPL (something I find very interesting).  This
may have changed with later versions...

Cheers,
Richard

-----
Please consider sponsoring my work on free software.
See http://www.free.lp.se/sponsoring.html for details.

--
Richard Levitte                         [hidden email]
                                        http://richard.levitte.org/

"When I became a man I put away childish things, including
 the fear of childishness and the desire to be very grown up."
                                                -- C.S. Lewis
______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
Development Mailing List                       [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: Considering SSL and Cryto libraries for LSB

Dr. Stephen Henson
In reply to this post by Dr. Stephen Henson
On Thu, Jun 30, 2005, Dr. Stephen Henson wrote:

>
> Its certainly possible. There is an added complication in that some thought has
> to be given to future directions of development.
>
> For example the current RSA API lacks any easy way to pass additional
> parameters to some padding types. OAEP and especially PSS really need a way to
> do that.
>
> That could be neatly solved (and some other parts made more efficient) if RSA
> had a per-thread context structure analagous to EVP_CIPHER_CTX but alas it
> doesn't.
>

While I'm at it here's a few other areas which would be "fun" to deal with...

EVP_CIPHER has two assumptions built in which are problematical when
interfacing (via ENGINE for example) with other APIs.

One is the size of the output buffer: there is an assumption that the output
buffer will be at most one block length larger than the input buffer,
applications naturally follow this rule.

Other APIs (for example PKCS#11) make no such assumption and can buffer the
data and supply the result all in one go.

If you have to connect such a beast via EVP that's one hell of a headache...

One the subject of EVP, unlike the asymmetric APIs (RSA, DSA etc) there is an
assumption that secret (i.e. symmetric) keys can be passed in plain text
to the API. This has been a parameter to EVP_CipherInit_ex() and its
equivalents since the SSLeay days. That assumption isn't present in some other
crypto APIs (again including PKCS#11 and even CryptoAPI) they can all treat a
symmetric key as an opaque blob (in a way similar to RSA structures can be
treated).

Again this is a problem if you need to interface with such APIs or for higher
security applications where exposure of secret keys may be prohibited by
security policy.

Don't get me even started on EVP_PKEY_METHOD :-)

Steve.
--
Dr Stephen N. Henson. Email, S/MIME and PGP keys: see homepage
OpenSSL project core developer and freelance consultant.
Funding needed! Details on homepage.
Homepage: http://www.drh-consultancy.demon.co.uk
______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
Development Mailing List                       [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: Considering SSL and Cryto libraries for LSB

Dan Kegel-2
In reply to this post by Richard Levitte - VMS Whacker
Richard Levitte - VMS Whacker wrote:
> dank> If so, perhaps that might provide a way forward: apps that need
> dank> a stable interface can use the gnutls api (which openssl could
> dank> provide as a wrapper); everyone else could use the openssl api
> dank> (which gnutls seems to provide as a wrapper, unless I misread
> dank> the docs).
>
> It's a path.  Just a small warning about license politics: According
> to http://www.gnu.org/software/gnutls/gnutls.html, the GnuTLS core
> library is licensed under the LGPL.

Sure.  So the OpenSSL community could do a cleanish-room
implementation of the gnutls API on top of OpenSSL.
The gnutls people would probably be happy, since that
would be an endorsement of their interface.

So perhaps it wouldn't be too hard to offer a stable interface
for apps that need it, even before OpenSSL hits 1.0.
- Dan

--
Trying to get a job as a c++ developer?  See http://kegel.com/academy/getting-hired.html

______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
Development Mailing List                       [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: Considering SSL and Cryto libraries for LSB

Dan Nuffer
In reply to this post by Dr. Stephen Henson
Dr. Stephen Henson wrote:

> On Wed, Jun 29, 2005, Dan Nuffer wrote:
>
>
>>Dr. Stephen Henson wrote:
>>
>>>This means that changing this in the short term is likely to cause
>>>widespread
>>>application breakage which wouldn't be too popular :-(
>>>
>>
>>Speaking as an application developer, I would willingly go through a
>>one-time source code upgrade to achieve binary compatiblity.  That is
>>more appealing to me than having to repeatedly deal with binary
>>compatiblity issues, because it's less work in the long run.
>>
>
>
> Also speaking as an application developer so would I. However there are a hell
> of a lot of applications out there that depend on the existing APIs not
> changing (too) much.
>

One possibility is to fork 0.9.x into maintenance mode for those apps.
I'm sure there's lots of reasons that would be painful, but I think
sometimes a library has to make this sort of transition in order to make
a break with old APIs.

> Can you give details of some of the binary compatibility issues you've come
> across?
>

I'm not sure whether this is the Linux distros' fault or not, but you
can't build a binary dynamically linked to libcrypto or libssl and have
it work on more than a handful of distros.

Older Redhats set the soname to libcrypto.[0-3]
Newer Redhats sets the soname to libcrypto.4
SUSE sets the soname to libcrypto.0.9.7

so if you build it on redhat you can't run it on SUSE and vice verse.

--
Dan Nuffer
Vintela, Inc. http://vintela.com/


______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
Development Mailing List                       [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

RE: Considering SSL and Cryto libraries for LSB

Banginwar, Rajesh
In reply to this post by Banginwar, Rajesh
> -----Original Message-----
> From: Geoff Thorpe [mailto:[hidden email]] On Behalf
> Of Geoff Thorpe
> Sent: Wednesday, June 29, 2005 8:15 PM
> To: [hidden email]
> Cc: Banginwar, Rajesh
> Subject: Re: Considering SSL and Cryto libraries for LSB
>
> On June 29, 2005 08:44 pm, Banginwar, Rajesh wrote:
> > So far from the preliminary analysis that we have done (by
> looking at
> > some of the OSS applications) we see both libssl and libcrypto being
> > used. E.g. from libcrypto I find functions in EVP, RSA, MD5 and DSA
> > sets more commonly used than others. Unfortunately this is not an
> > exhaustive study. Do you or anyone on this project have
> data suggesting
> > which APIs are candidates for LSB inclusion both from demand and
> > stability point of view?
>
> I'm not sure how you would harvest this kind of information.
> From a build
> perspective, it may be quite difficult to disentangle one API
> from the
> next. I guess someone truly masoc^H^H^H^H^Hdedicated could create an
> alternative "openssl-lsb.h" header to build against - one
> that declared
> only the subset of functionality that mattered from an LSB
> perspective
> and then see how external apps fare building against that. However
> there's a lot of macro-abuse in some areas of the code, and if this
> "shimming" got tangled up in trying to reproduce ASN1
> definitions then I
> don't even want to contemplate where it might lead...
>
> Something to think about I guess. But it certainly seems
> hairy to attempt
> to standardise too much invariance across major releases that
> we expect
> will ... um ... vary.

The intention here is not to identify the exact apis used by apps, but
to get an idea about which APIs are more commonly used and hence the
candidate for LSB inclusion. E.g. if EVP APIs are commonly used, then
all or most of these APIs in this set should be considered for inclusion
to LSB.

By doing this, we may be able to stage the problem for these libraries
(in the first stage just work on certain set of APIs and update the
function definitions as discussed in other emails on this thread). I am
not sure how good a approach that is for OpenSSL.


>
> > I just checked the release history for openssl and it is
> encouraging to
> > see the major releases are years apart. I see that with
> some efforts it
> > may be possible to hide some of the data structure dependencies (as
> > Steve pointed out in other emails on this thread) and get
> some sets of
> > APIs to a "stable" level and push for LSB inclusion. There
> was at least
> > one comment from application developer encouraging this
> even though it
> > will require major changes.
>
> It sounds like a fairly labourious process though (ie. I
> doubt we'll be
> dispersing the volunteers with tear-gas or anything).
>
> Cheers,
> Geoff
>
> --
> Geoff Thorpe
> [hidden email]
> http://www.openssl.org/
>
>
______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
Development Mailing List                       [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

RE: Considering SSL and Cryto libraries for LSB

Banginwar, Rajesh
In reply to this post by Banginwar, Rajesh
Given the increased focus of LSB on desktop applications, it may be
right time to consider this update. It will be really useful to find out
from the application developer community regarding how often they face
this problem of distro incompatibility. If this is a reasonably
wide-spread problem for many applications, this may provide us a good
opportunity to create a version of library with identified stable ABIs.
Of course not being the developer or user of OpenSSL, it is easy for me
to say that.

From my observations libcrypto is quite widely used and hence the
interest from LSB team. If OpenSSL is really not a candidate today as
mentioned by Richard in earlier email, may be it becomes so in near
future. The next major release from LSB is planned for in late 2006 or
early 2007. So we certainly have time to work towards the goal of
getting OpenSSL part of LSB if there is common understanding.

Thanks,

-Rajesh

> -----Original Message-----
> From: [hidden email]
> [mailto:[hidden email]] On Behalf Of Dan Nuffer
> Sent: Thursday, June 30, 2005 12:33 PM
> To: [hidden email]
> Subject: Re: Considering SSL and Cryto libraries for LSB
>
> Dr. Stephen Henson wrote:
> > On Wed, Jun 29, 2005, Dan Nuffer wrote:
> >
> >
> >>Dr. Stephen Henson wrote:
> >>
> >>>This means that changing this in the short term is likely to cause
> >>>widespread
> >>>application breakage which wouldn't be too popular :-(
> >>>
> >>
> >>Speaking as an application developer, I would willingly go
> through a
> >>one-time source code upgrade to achieve binary
> compatiblity.  That is
> >>more appealing to me than having to repeatedly deal with binary
> >>compatiblity issues, because it's less work in the long run.
> >>
> >
> >
> > Also speaking as an application developer so would I.
> However there are a hell
> > of a lot of applications out there that depend on the
> existing APIs not
> > changing (too) much.
> >
>
> One possibility is to fork 0.9.x into maintenance mode for
> those apps.
> I'm sure there's lots of reasons that would be painful, but I think
> sometimes a library has to make this sort of transition in
> order to make
> a break with old APIs.
>
> > Can you give details of some of the binary compatibility
> issues you've come
> > across?
> >
>
> I'm not sure whether this is the Linux distros' fault or not, but you
> can't build a binary dynamically linked to libcrypto or
> libssl and have
> it work on more than a handful of distros.
>
> Older Redhats set the soname to libcrypto.[0-3]
> Newer Redhats sets the soname to libcrypto.4
> SUSE sets the soname to libcrypto.0.9.7
>
> so if you build it on redhat you can't run it on SUSE and vice verse.
>
> --
> Dan Nuffer
> Vintela, Inc. http://vintela.com/
>
>
> ______________________________________________________________________
> OpenSSL Project                                 http://www.openssl.org
> Development Mailing List                       [hidden email]
> Automated List Manager                           [hidden email]
>
______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
Development Mailing List                       [hidden email]
Automated List Manager                           [hidden email]