[openssl-users] Long startup time and poor entropy on Windows due to inefficient heap walking In RAND_poll()

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

[openssl-users] Long startup time and poor entropy on Windows due to inefficient heap walking In RAND_poll()

Noel Carboni

Hello, this is my first post to this list.

 

I'm a software developer.  The product line I'm developing indirectly uses OpenSSL through a 3rd party library.  Specifically, the library does a decryption of a license file right at the startup of our application. 

 

Trouble is, the OpenSSL startup is taking over 1 second on a modern computer running Windows 8.1 x64.

 

1 second doesn't sound like much, but that's time our user has to wait every time to see our user interface.  The entire rest of our startup logic literally takes a few tens of milliseconds.  It's making our application sluggish, and we need the license file verified before the application starts up.

 

We tracked the delay down to OpenSSL's entropy gathering heap walking logic in the RAND_poll() function in rand_win.c.  The Heap32First() and Heap32Next() Windows API calls are monstrously inefficient - each iteration taking a significant part of a second just to return a pointer to the first or next heap entry.  Since the logic attempts to walk through the entire heap, it doesn't take many calls at all to exceed the MAXDELAY time of 1000 milliseconds (one full second).

 

I have the impression, looking at the RAND_poll() code, that the MAXDELAY of 1000 milliseconds is a "watchdog" timeout meant to prevent a complete lockup under some unspecified condition, and it is not expected to be exercised in the normal case.   Researching a bit further, MAXDELAY appears to have been added to RAND_Poll() some 5+ years ago.

 

As a source of entropy, the number of bytes that COULD be "added" together by RAND_add() in the heap walking loop should be monstrous - millions or even billions.  But in practice that's just not happening, since it's taking so long to get the pointers.  It literally can just be a few thousands of bytes.

 

So...  Two things:

 

1.      Since Heap32First() and Heap32Next() move glacially, not very many heap blocks are traversed in 1000 milliseconds (possibly only one or a handful on an average computer).  This implies the heap walking loop is a VERY poor source of entropy given the amount of CPU time spent.  I'm no expert on entropy, but this appears to reduce the security of the library.

2.      The library in our case is being used in a client application where (unlike a server that starts OpenSSL up once and stays up) it's initialized every time the application is opened, and so the startup time is important.  Can the watchdog timeout for the heap walk loop be reduced to something more reasonable?  Say 100 milliseconds?

 

It's pretty clear that if a more efficient method is implemented to walk through heap blocks, a LOT more entropy can be gathered in a LOT less time.  Bear in mind also that computers are a LOT faster than they were even 5 years ago when the 1 second MAXDELAY timeout was first implemented.

 

If need be, I can likely put together source code to accomplish the above.

 

Thanks.

 

-Noel Carboni

ProDigital Software


_______________________________________________
openssl-users mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-users
Reply | Threaded
Open this post in threaded view
|

Re: [openssl-users] Long startup time and poor entropy on Windows due to inefficient heap walking In RAND_poll()

Jeffrey Walton-3
On Sun, Jan 18, 2015 at 10:17 PM, Noel Carboni
<[hidden email]> wrote:

> ...
> Trouble is, the OpenSSL startup is taking over 1 second on a modern computer
> running Windows 8.1 x64.
> ...
>
> We tracked the delay down to OpenSSL's entropy gathering heap walking logic
> in the RAND_poll() function in rand_win.c.  The Heap32First() and
> Heap32Next() Windows API calls are monstrously inefficient - each iteration
> taking a significant part of a second just to return a pointer to the first
> or next heap entry.  Since the logic attempts to walk through the entire
> heap, it doesn't take many calls at all to exceed the MAXDELAY time of 1000
> milliseconds (one full second).

You should seed OpenSSL's random number generator directly using
CryptGenRandom (and other entropy you might have).

Once the generator is seeded, it won't attempt to auto-seed itself
with the RAND_poll gear.

Also see http://wiki.openssl.org/index.php/Random_Numbers. It advises
that you seed the generator directly rather than depending on the
library doing it through RAND_poll.

Jeff
_______________________________________________
openssl-users mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-users
Reply | Threaded
Open this post in threaded view
|

Re: [openssl-users] Long startup time and poor entropy on Windows due to inefficient heap walking In RAND_poll()

Noel Carboni
> You should seed OpenSSL's random number generator directly using CryptGenRandom (and other entropy you might have).
> Once the generator is seeded, it won't attempt to auto-seed itself with the RAND_poll gear.

Thank you for that workaround advice, Jeff.  I've passed it on to the folks using the library calls.

The OpenSSL library developers might still want to consider alternate heap walk coding, if for no other reason than for those who
use the library without having seen the wiki page mentioned and knowing they should seed it themselves.  

The library designers may feel the 1 second used in RAND_poll() is time well spent adding up memory blocks, when mostly it's just
wasted.  We measured a 1000 to 1 ratio between the time spent in Heap32Next() retrieving memory block pointers and the time spent
accumulating entropy in RAND_add().

Thanks again.

-Noel

-----Original Message-----
From: openssl-users [mailto:[hidden email]] On Behalf Of Jeffrey Walton
Sent: Mon, January 19, 2015 12:02 AM
To: OpenSSL Users List
Subject: Re: [openssl-users] Long startup time and poor entropy on Windows due to inefficient heap walking In RAND_poll()

You should seed OpenSSL's random number generator directly using CryptGenRandom (and other entropy you might have).

Once the generator is seeded, it won't attempt to auto-seed itself with the RAND_poll gear.

Also see http://wiki.openssl.org/index.php/Random_Numbers. It advises that you seed the generator directly rather than depending on
the library doing it through RAND_poll.

Jeff

_______________________________________________
openssl-users mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-users
Reply | Threaded
Open this post in threaded view
|

Re: [openssl-users] Long startup time and poor entropy on Windows due to inefficient heap walking In RAND_poll()

Salz, Rich

> The OpenSSL library developers might still want to consider alternate heap
> walk coding

See http://rt.openssl.org/Ticket/Display.html?id=3594
_______________________________________________
openssl-users mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-users
Reply | Threaded
Open this post in threaded view
|

Re: [openssl-users] Long startup time and poor entropy on Windows due to inefficient heap walking In RAND_poll()

Noel Carboni
> See http://rt.openssl.org/Ticket/Display.html?id=3594

As a new mailing list user, I don't seem to have an account capable of viewing that ticket.  The login interface does not take my
mailing list credentials.  Can I assume it's a discussion of this very problem?  If so, that's good.

-Noel

_______________________________________________
openssl-users mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-users
Reply | Threaded
Open this post in threaded view
|

Re: [openssl-users] Long startup time and poor entropy on Windows due to inefficient heap walking In RAND_poll()

Steven Kneizys
It seems that user "guest" password "guest" allows one the ability to view it.

On Mon, Jan 19, 2015 at 11:53 AM, Noel Carboni <[hidden email]> wrote:
> See http://rt.openssl.org/Ticket/Display.html?id=3594

As a new mailing list user, I don't seem to have an account capable of viewing that ticket.  The login interface does not take my
mailing list credentials.  Can I assume it's a discussion of this very problem?  If so, that's good.

-Noel

_______________________________________________
openssl-users mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-users



--
Steve Kneizys
Senior Business Process Engineer @ Ferrilli
Voice: (610) 256-1396  [For Emergency Service (888)864-3282]
web: http://www.ferrilli.com/


_______________________________________________
openssl-users mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-users
Reply | Threaded
Open this post in threaded view
|

Re: [openssl-users] Long startup time and poor entropy on Windows due to inefficient heap walking In RAND_poll()

Noel Carboni

Perfect!

 

http://rt.openssl.org/Ticket/Display.html?id=3594&user=guest&pass=guest

 

Seems like the problem is well in hand.  Thanks for the conversation, guys!

 

-Noel


_______________________________________________
openssl-users mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-users