Re: [openssl-team] Discussion: design issue: async and -lpthread

classic Classic list List threaded Threaded
60 messages Options
123
Reply | Threaded
Open this post in threaded view
|

Re: [openssl-team] Discussion: design issue: async and -lpthread

Dr Paul Dale

The "have-atomics" is intended to test if the callback was installed by the user.  If we're using an atomic library or compiler support, then it isn't required since we know we've got them.

Likewise, the lock argument isn't required if atomics are used everywhere.  However, some code will need fixing since there are places that adjust reference counters directly using arithmetic operators (while holding the appropriate lock).  These will have to be changed to atomics and the locked sections of code checked to see that that doesn't introduce other problems.

All possible of course.


Pauli


On Tue, 8 Dec 2015 10:01:20 PM Nico Williams wrote:

> On Wed, Dec 09, 2015 at 09:27:16AM +1000, Paul Dale wrote:
> > It will be possible to support atomics in such a way that there is no
> > performance penalty for machines without them or for single threaded
> > operation.  My sketcy design is along the lines of adding a new API
> > CRYPTO_add_atomic that takes the same arguments as CRYPTO_add (i.e.
> > reference to counter, value to add and lock to use):
> >
> > CRYPTO_add_atomic(int *addr, int amount, int lock)
> >     if have-atomics then
> >         atomic_add(addr, amount)
> >     else if (lock == have-lock-already)
> >         *addr += amount
> >     else
> >         CRYPTO_add(addr, amount, lock)
>
> "have-atomics" must be known at compile time.
>
> "lock" should not be needed because we should always have atomics, even
> when we don't have true atomics: just use a global lock in a stub
> implementation of atomic_add() and such.  KISS.  Besides, this will add
> pressure to add true atomics wherever they are truly needed.
>
> Nico
> --

--
Oracle
Dr Paul Dale | Cryptographer | Network Security & Encryption
Phone +61 7 3031 7217
Oracle Australia

_______________________________________________
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
Reply | Threaded
Open this post in threaded view
|

Re: [openssl-team] Discussion: design issue: async and -lpthread

Nico Williams
No more installing callbacks to get locking and atomics.  This has to all work out of the box (the user could be allowed tip supply their own implementations if these things at OpenSSL build time, but that's it, not at run-time).

Nico
-- 

_______________________________________________
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
Reply | Threaded
Open this post in threaded view
|

Re: [openssl-team] Discussion: design issue: async and -lpthread

Nico Williams
On Wed, Dec 09, 2015 at 02:33:46AM -0600, Nico Williams wrote:
> No more installing callbacks to get locking and atomics.

I should explain why.

First, lock callbacks are a serious detriment to usability.

Second, they are an admission that OpenSSL is incomplete.

Third, if we have lock callbacks to install, then we have the risk of
racing (by multiple libraries using OpenSSL) to install them.  Unless
there's a single function to install *all* such callbacks, then there's
no way to install callbacks atomically.  But every once in a while we'll
need to add an Nth callback, thus breaking the ABI or atomicity.

So, no, no lock callbacks.  OpenSSL should work thread-safely out of the
box like other libraries.  That means that the default configuration
should be to use pthreads on *nix, for example.  We'll need an atomics
library (e.g., OpenPA, or something new) with safe and sane -if not very
performant- defaults that use global locks for platform/compiler
combinations where there's no built-in atomics intrinsics or system
library.  It should be possible to have a no-threads configuration where
the locks and atomics are non-concurrent-safe implementations.

Nico
--
_______________________________________________
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
Reply | Threaded
Open this post in threaded view
|

Re: [openssl-team] Discussion: design issue: async and -lpthread

Salz, Rich
In reply to this post by Nico Williams

> "have-atomics" must be known at compile time.
>
> "lock" should not be needed because we should always have atomics, even
> when we don't have true atomics: just use a global lock in a stub
> implementation of atomic_add() and such.  KISS.  Besides, this will add
> pressure to add true atomics wherever they are truly needed.

Agree.
_______________________________________________
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
Reply | Threaded
Open this post in threaded view
|

Re: [openssl-team] Discussion: design issue: async and -lpthread

Salz, Rich
In reply to this post by Dr Paul Dale

> The "have-atomics" is intended to test if the callback was installed by the
> user.

I want to move away from runtime callback installations.  It makes it too hard to know what the library is doing, if it is correct, and it complicates the code.  There is almost never any reason for the flexibility that OpenSSL "used to" have. :)
_______________________________________________
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
Reply | Threaded
Open this post in threaded view
|

Re: [openssl-team] Discussion: design issue: async and -lpthread

Blumenthal, Uri - 0553 - MITLL
In reply to this post by Nico Williams
+2 to Rich and Nico.

Sent from my BlackBerry 10 smartphone on the Verizon Wireless 4G LTE network.
  Original Message  
From: Salz, Rich
Sent: Wednesday, December 9, 2015 08:37
To: [hidden email]; [hidden email]; Nico Williams
Reply To: [hidden email]
Subject: Re: [openssl-dev] [openssl-team] Discussion: design issue: async and -lpthread


> The "have-atomics" is intended to test if the callback was installed by the
> user.

I want to move away from runtime callback installations. It makes it too hard to know what the library is doing, if it is correct, and it complicates the code. There is almost never any reason for the flexibility that OpenSSL "used to" have. :)
_______________________________________________
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev


_______________________________________________
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

smime.p7s (5K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: [openssl-team] Discussion: design issue: async and -lpthread

Dr Paul Dale
In reply to this post by Nico Williams
Nico,

Thanks for the clarification.  I was making an assumption that following the existing locking model, which did seem over complicated, was desirable.  Now that that is shot down, things can be much simpler.

It would make more sense to have a structure containing the reference counter and (optionally?) a lock to use for that counter.
With atomics, the lock isn't there or at least isn't used.  Without them, it is.  This is because, I somewhat suspect having a fall back global lock for all atomic operations would be worse than the current situation were at least a few different locks are used.

There is also the possibility of only using the per reference counter lock and not using atomic operations at all -- this would reduce the contention a lot and might not hurt performance much.  It would be easy to benchmark an uncontested lock/add/unlock versus atomic add on the target platforms to see the difference.


Thanks against for the insights,

Pauli
--
Oracle
Dr Paul Dale | Cryptographer | Network Security & Encryption
Phone +61 7 3031 7217
Oracle Australia
On Wed, 9 Dec 2015 03:27:51 AM Nico Williams wrote:

> On Wed, Dec 09, 2015 at 02:33:46AM -0600, Nico Williams wrote:
> > No more installing callbacks to get locking and atomics.
>
> I should explain why.
>
> First, lock callbacks are a serious detriment to usability.
>
> Second, they are an admission that OpenSSL is incomplete.
>
> Third, if we have lock callbacks to install, then we have the risk of
> racing (by multiple libraries using OpenSSL) to install them.  Unless
> there's a single function to install *all* such callbacks, then there's
> no way to install callbacks atomically.  But every once in a while we'll
> need to add an Nth callback, thus breaking the ABI or atomicity.
>
> So, no, no lock callbacks.  OpenSSL should work thread-safely out of the
> box like other libraries.  That means that the default configuration
> should be to use pthreads on *nix, for example.  We'll need an atomics
> library (e.g., OpenPA, or something new) with safe and sane -if not very
> performant- defaults that use global locks for platform/compiler
> combinations where there's no built-in atomics intrinsics or system
> library.  It should be possible to have a no-threads configuration where
> the locks and atomics are non-concurrent-safe implementations.
>
> Nico
> --



_______________________________________________
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
Reply | Threaded
Open this post in threaded view
|

Re: [openssl-team] Discussion: design issue: async and -lpthread

Nico Williams
On Thu, Dec 10, 2015 at 07:06:15AM +1000, Paul Dale wrote:
> Thanks for the clarification.  I was making an assumption that
> following the existing locking model, which did seem over complicated,
> was desirable.  Now that that is shot down, things can be much
> simpler.

Exactly :)

Sorry if I was a bit brusque.  Since inertia is strong, I figured I
needed to make a forceful argument.  However, it seems it was easy to
get consensus after all.

> It would make more sense to have a structure containing the reference
> counter and (optionally?) a lock to use for that counter.

It'd work but it'd be a complication, since now every integer to be used
with atomic increment/decrement, or CAS, or whatever, now needs to be a
struct type with the integer as one field and the rest as opaque.  It'd
be much nicer to be able to use ints normally, though I would agree that
having a special type for atomics has the benefit that it is
self-describing.

It's perfectly fine to have a worst-case atomics implementation that
uses a global lock.  Yes, that would be slow, but we need some incentive
to add true atomics for each platform, and making it slow is the exact
right incentive.

So even if for documentation and type-safety reasons we wanted to wrap
ints in structs for ints meant to be used with atomics, I'd still want
the worst-case atomics implementation to be slow.

> With atomics, the lock isn't there or at least isn't used.  Without
> them, it is.  This is because, I somewhat suspect having a fall back
> global lock for all atomic operations would be worse than the current
> situation were at least a few different locks are used.

That's a feature :)

Nico
--
_______________________________________________
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
Reply | Threaded
Open this post in threaded view
|

Re: [openssl-team] Discussion: design issue: async and -lpthread

Florian Weimer
In reply to this post by Nico Williams
* Nico Williams:

> On Tue, Dec 08, 2015 at 11:19:32AM +0100, Florian Weimer wrote:
>> > Maybe http://trac.mpich.org/projects/openpa/ would fit the bill?
>>
>> It seems to have trouble to keep up with new architectures.
>
> New architectures are not really a problem because between a) decent
> compilers with C11 and/or non-C11 atomic intrinsics,

Not on Windows.

> What's the alternative anyways?

Using C++11.
_______________________________________________
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
Reply | Threaded
Open this post in threaded view
|

Re: [openssl-team] Discussion: design issue: async and -lpthread

Kurt Roeckx
On Tue, Dec 15, 2015 at 01:24:12PM +0100, Florian Weimer wrote:

> * Nico Williams:
>
> > On Tue, Dec 08, 2015 at 11:19:32AM +0100, Florian Weimer wrote:
> >> > Maybe http://trac.mpich.org/projects/openpa/ would fit the bill?
> >>
> >> It seems to have trouble to keep up with new architectures.
> >
> > New architectures are not really a problem because between a) decent
> > compilers with C11 and/or non-C11 atomic intrinsics,
>
> Not on Windows.
>
> > What's the alternative anyways?
>
> Using C++11.

I think this is a relevant article:
http://herbsutter.com/2012/05/03/reader-qa-what-about-vc-and-c99/


Kurt

_______________________________________________
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
Reply | Threaded
Open this post in threaded view
|

Re: [openssl-team] Discussion: design issue: async and -lpthread

Benjamin Kaduk
On 12/15/2015 06:43 AM, Kurt Roeckx wrote:
> On Tue, Dec 15, 2015 at 01:24:12PM +0100, Florian Weimer wrote:
>> * Nico Williams:
>> Not on Windows.
>>
>>> What's the alternative anyways?
>> Using C++11.
> I think this is a relevant article:
> http://herbsutter.com/2012/05/03/reader-qa-what-about-vc-and-c99/
>

I think an article from 2012 is no longer current; something like
http://blogs.msdn.com/b/vcblog/archive/2015/06/19/c-11-14-17-features-in-vs-2015-rtm.aspx
might be a better source.

-Ben
_______________________________________________
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
Reply | Threaded
Open this post in threaded view
|

Re: [openssl-team] Discussion: design issue: async and -lpthread

Nico Williams
In reply to this post by Florian Weimer
On Tue, Dec 15, 2015 at 01:24:12PM +0100, Florian Weimer wrote:

> * Nico Williams:
>
> > On Tue, Dec 08, 2015 at 11:19:32AM +0100, Florian Weimer wrote:
> >> > Maybe http://trac.mpich.org/projects/openpa/ would fit the bill?
> >>
> >> It seems to have trouble to keep up with new architectures.
> >
> > New architectures are not really a problem because between a) decent
> > compilers with C11 and/or non-C11 atomic intrinsics,
>
> Not on Windows.

Windows has a family of functions for atomic addition, compare-and-swap,
etcetera:

https://msdn.microsoft.com/en-us/library/windows/desktop/ms686360%28v=vs.85%29.aspx#interlocked_functions

Solaris/Illumos has its own as well.

Linux has several atomics libraries.

And there are several open-source portable atomics libraries as well.

I.e., between compiler non-C11 atomic intrinsics, C11 intrinsics, OS
atomic function libraries, and portable open-source atomics libraries,
we can cover almost all the bases.

> > What's the alternative anyways?
>
> Using C++11.

Sure, but only for a C atomics library for the rest of OpenSSL.

So that makes five alternatives, plus the two stub implementations (one
with global locks, one with no locking/atomics).  Any platform not
covered will get one of the stub implementations and its users will have
to contribute a better implementation.

We have a surfeit of options, not a dearth of them.  I don't think lack
of atomics primitives is remotely a concern.  We should use atomic
primitives in OpenSSL.

Nico
--
_______________________________________________
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
Reply | Threaded
Open this post in threaded view
|

Re: [openssl-team] Discussion: design issue: async and -lpthread

Nico Williams
In reply to this post by Benjamin Kaduk
On Tue, Dec 15, 2015 at 09:57:32AM -0600, Benjamin Kaduk wrote:

> On 12/15/2015 06:43 AM, Kurt Roeckx wrote:
> > On Tue, Dec 15, 2015 at 01:24:12PM +0100, Florian Weimer wrote:
> >> Using C++11.
> > I think this is a relevant article:
> > http://herbsutter.com/2012/05/03/reader-qa-what-about-vc-and-c99/
> >
>
> I think an article from 2012 is no longer current; something like
> http://blogs.msdn.com/b/vcblog/archive/2015/06/19/c-11-14-17-features-in-vs-2015-rtm.aspx
> might be a better source.

Yes, but not everyone will have the latest and greatest compilers.
Still, the Windows interlocked function family is enough.  See my other
post just now.

Nico
--
_______________________________________________
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
Reply | Threaded
Open this post in threaded view
|

Re: [openssl-team] Discussion: design issue: async and -lpthread

Salz, Rich
In reply to this post by Nico Williams
> I.e., between compiler non-C11 atomic intrinsics, C11 intrinsics, OS atomic
> function libraries, and portable open-source atomics libraries, we can cover
> almost all the bases.

Agreed.

> We have a surfeit of options, not a dearth of them.  I don't think lack of
> atomics primitives is remotely a concern.  We should use atomic primitives in
> OpenSSL.

Yes.  It would be great if we could start collecting snippets so that it can easily be put into the next 1.1 alpha release.  Perhaps on the wiki:
        https://wiki.openssl.org/index.php/Examples
 
_______________________________________________
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
Reply | Threaded
Open this post in threaded view
|

Re: [openssl-team] Discussion: design issue: async and -lpthread

Kurt Roeckx
In reply to this post by Benjamin Kaduk
On Tue, Dec 15, 2015 at 09:57:32AM -0600, Benjamin Kaduk wrote:

> On 12/15/2015 06:43 AM, Kurt Roeckx wrote:
> > On Tue, Dec 15, 2015 at 01:24:12PM +0100, Florian Weimer wrote:
> >> * Nico Williams:
> >> Not on Windows.
> >>
> >>> What's the alternative anyways?
> >> Using C++11.
> > I think this is a relevant article:
> > http://herbsutter.com/2012/05/03/reader-qa-what-about-vc-and-c99/
> >
>
> I think an article from 2012 is no longer current; something like
> http://blogs.msdn.com/b/vcblog/archive/2015/06/19/c-11-14-17-features-in-vs-2015-rtm.aspx
> might be a better source.

That's all about C++, except for library functions from C99,
offsetof, __func__, and "long long" that are now also part of
C++.

The point of the article I pointed to is that they only do C90.
They have no intention of adding support for newer C standards
except for the cases where C++ just happens to be compatible.

And the C++11 and C11 atomics are compatible, they just have a
slightly different syntax, and as the URL I pointed to says are
implemented in it.

Also, if you want to use atomics we really want the C11 / C++11
memory model which prevents certain important optimazations.


Kurt

_______________________________________________
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
Reply | Threaded
Open this post in threaded view
|

Re: [openssl-team] Discussion: design issue: async and -lpthread

Nico Williams
In reply to this post by Salz, Rich
On Tue, Dec 15, 2015 at 06:15:32PM +0000, Salz, Rich wrote:
> > I.e., between compiler non-C11 atomic intrinsics, C11 intrinsics, OS atomic
> > function libraries, and portable open-source atomics libraries, we can cover
> > almost all the bases.
>
> Agreed.

Thanks.  This is helpful.  I now think we'll have an easy road to
consensus on how to get thread-safety in OpenSSL.  I will try to
contribute infrastructure, but there's a lot of heavy lifting to do that
will take a long time, and I can't do it all.

Nico
--
_______________________________________________
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
Reply | Threaded
Open this post in threaded view
|

Re: [openssl-team] Discussion: design issue: async and -lpthread

Nico Williams
In reply to this post by Kurt Roeckx
On Tue, Dec 15, 2015 at 07:54:35PM +0100, Kurt Roeckx wrote:
> Also, if you want to use atomics we really want the C11 / C++11
> memory model which prevents certain important optimazations.

Right, because compilers can reorder some operations.  But we've been
living with this pre-C11 for decades.  We can't yet require C11 (though
I'd sure like to).

BTW, there's this:

http://blogs.msdn.com/b/vcblog/archive/2015/05/01/bringing-clang-to-windows.aspx

which, IIUC means we can get C11 on Windows with MSVC with a clang
frontend.  So maybe Windows is a non-issue, and maybe C11 is a
non-issue.  Though I'm sure there's platforms where OpenSSL can't expect
C11 yet, so I suspect we're stuck with C90+ for a while.

Nico
--
_______________________________________________
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
Reply | Threaded
Open this post in threaded view
|

Re: [openssl-team] Discussion: design issue: async and -lpthread

Florian Weimer
In reply to this post by Kurt Roeckx
* Kurt Roeckx:

> On Tue, Dec 15, 2015 at 01:24:12PM +0100, Florian Weimer wrote:
>> * Nico Williams:
>>
>> > On Tue, Dec 08, 2015 at 11:19:32AM +0100, Florian Weimer wrote:
>> >> > Maybe http://trac.mpich.org/projects/openpa/ would fit the bill?
>> >>
>> >> It seems to have trouble to keep up with new architectures.
>> >
>> > New architectures are not really a problem because between a) decent
>> > compilers with C11 and/or non-C11 atomic intrinsics,
>>
>> Not on Windows.
>>
>> > What's the alternative anyways?
>>
>> Using C++11.
>
> I think this is a relevant article:
> http://herbsutter.com/2012/05/03/reader-qa-what-about-vc-and-c99/

Based on the MSDN documentation, this (e.g. C++11 features implemented
in the MSVC C compiler) has not actually happened:

  <https://msdn.microsoft.com/en-us/library/634ca0c2.aspx>
_______________________________________________
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
Reply | Threaded
Open this post in threaded view
|

Re: [openssl-team] Discussion: design issue: async and -lpthread

Kurt Roeckx
On Tue, Dec 15, 2015 at 10:35:58PM +0100, Florian Weimer wrote:

> * Kurt Roeckx:
>
> > On Tue, Dec 15, 2015 at 01:24:12PM +0100, Florian Weimer wrote:
> >> * Nico Williams:
> >>
> >> > On Tue, Dec 08, 2015 at 11:19:32AM +0100, Florian Weimer wrote:
> >> >> > Maybe http://trac.mpich.org/projects/openpa/ would fit the bill?
> >> >>
> >> >> It seems to have trouble to keep up with new architectures.
> >> >
> >> > New architectures are not really a problem because between a) decent
> >> > compilers with C11 and/or non-C11 atomic intrinsics,
> >>
> >> Not on Windows.
> >>
> >> > What's the alternative anyways?
> >>
> >> Using C++11.
> >
> > I think this is a relevant article:
> > http://herbsutter.com/2012/05/03/reader-qa-what-about-vc-and-c99/
>
> Based on the MSDN documentation, this (e.g. C++11 features implemented
> in the MSVC C compiler) has not actually happened:
>
>   <https://msdn.microsoft.com/en-us/library/634ca0c2.aspx>

And in
http://blogs.msdn.com/b/vcblog/archive/2014/06/11/c-11-14-feature-tables-for-visual-studio-14-ctp1.aspx
someone mentions that stdatomic.h doesn't exist in the 2014
version.  I can't find a reference that says anything in the 2015
changed related to that so I assume it didn't.

The only thing I could find is that the C library hasn't been
incorparated in the C++ standard.  But all the functions actually
exist in the C++ <atomic>.


Kurt

_______________________________________________
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
Reply | Threaded
Open this post in threaded view
|

Re: [openssl-team] Discussion: design issue: async and -lpthread

Nico Williams
In reply to this post by Florian Weimer
Another interesting portable atomics library is
https://github.com/mintomic/mintomic

FYI, I took a stab at a simple portable atomics that uses GCC/clang
__atomic, or __sync, or Win32 Interlocked*, or a single global lock, and
with a fallback to unsafe, non-atomic implementations for no-threads
configurations; adding C11 support will be trivial.  For just
incremdent/decrement and CAS this is really small, and I think that's
enough for OpenSSL for starters.
_______________________________________________
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
123