PRNG not available when multiple providers are configured?

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

PRNG not available when multiple providers are configured?

Thomas Dwyer III
I'm having trouble getting RAND_status() to return 1 when my openssl.cnf has both the default provider and the fips provider configured at the same time:

        openssl_conf = openssl_init

        [openssl_init]
        providers = provider_sect

        [provider_sect]
        default = default_sect
        fips = fips_sect

        [default_sect]
        activate = 1

        .include /conf/openssl/fips.cnf

If I remove either default or fips from [provider_sect] then RAND_status() returns 1. If I leave them both specified there, RAND_status() always returns 0. Is this the expected behavior or am I doing something wrong? I understand that I must specify properties when fetching algorithms in order to get deterministic behavior with multiple providers loaded. Is there an analogous API for the PRNG that I'm overlooking?

Interestingly, setting activate=0 for either provider is not sufficient to work around this issue.


Thanks,
Tom.III

Reply | Threaded
Open this post in threaded view
|

Re: PRNG not available when multiple providers are configured?

Matt Caswell-2


On 03/11/2020 00:55, Thomas Dwyer III wrote:

> I'm having trouble getting RAND_status() to return 1 when my openssl.cnf
> has both the default provider and the fips provider configured at the
> same time:
>
>         openssl_conf = openssl_init
>
>         [openssl_init]
>         providers = provider_sect
>
>         [provider_sect]
>         default = default_sect
>         fips = fips_sect
>
>         [default_sect]
>         activate = 1
>
>         .include /conf/openssl/fips.cnf
>
> If I remove either default or fips from [provider_sect] then
> RAND_status() returns 1. If I leave them both specified there,
> RAND_status() always returns 0. Is this the expected behavior or am I
> doing something wrong? I understand that I must specify properties when
> fetching algorithms in order to get deterministic behavior with multiple
> providers loaded. Is there an analogous API for the PRNG that I'm
> overlooking?
>
> Interestingly, setting activate=0 for either provider is not sufficient
> to work around this issue.

I tested this out and was able to replicate your behaviour.

The reasons are a little complicated (see below) but the TL;DR summary
is that there is an error in your config file. The ".include" line
should specify a config file relative to OPENSSLDIR (or
OPENSSL_CONF_INCLUDE if it is set). It cannot be an absolute path, and
hence fips.cnf is not being found.

I've seen this error a few times now so I'm thinking that we should
perhaps allow absolute paths. I'm not sure what the reason for
disallowing them was.

The reason it works if you comment out the "default" line is because
that means the only provider left is the FIPS one. But the config line
for that is faulty and therefore activating it fails. Ultimately we have
not succesfully activated any provider. When you call RAND_status() it
will attempt to fetch the RAND implementation and find that no providers
have been activated. In this case we fallback and automatically activate
the default provider. Hence you end up with RAND_status() still working.

If you comment out the "fips" line then it works because it doesn't
attempt to do anything with the fips provider, successfully activates
the default provider, and hence RAND_status() works as expected.

If you have both lines in the config file then it first successfully
activates the default provider. It next attempts to activate the fips
provider and fails. The way the config code works is that if any of the
configured providers fail to activate then it backs out and deactivates
all of them. At this point we're in a situation where a provider has
been successfully activated and then deactivated again. The fallback
activation of the default provider only kicks in if you've not attempted
to activate any providers by the time you first need one. Therefore the
default provider doesn't activate as a fallback either. Ultimately you
end up with no active providers and RAND_status() fails.

We really should have a way of getting more verbose output in the event
of config issues like this.

Matt
Reply | Threaded
Open this post in threaded view
|

Re: PRNG not available when multiple providers are configured?

Matt Caswell-2


On 03/11/2020 15:13, Matt Caswell wrote:
> I've seen this error a few times now so I'm thinking that we should
> perhaps allow absolute paths. I'm not sure what the reason for
> disallowing them was.

I raised this issue about this:

https://github.com/openssl/openssl/issues/13302

> We really should have a way of getting more verbose output in the event
> of config issues like this.

And for this one I've raised this:

https://github.com/openssl/openssl/issues/13303

Matt

Reply | Threaded
Open this post in threaded view
|

Re: PRNG not available when multiple providers are configured?

Tomas Mraz-2
In reply to this post by Matt Caswell-2
On Tue, 2020-11-03 at 15:13 +0000, Matt Caswell wrote:

>
> The reasons are a little complicated (see below) but the TL;DR
> summary
> is that there is an error in your config file. The ".include" line
> should specify a config file relative to OPENSSLDIR (or
> OPENSSL_CONF_INCLUDE if it is set). It cannot be an absolute path,
> and
> hence fips.cnf is not being found.
>
> I've seen this error a few times now so I'm thinking that we should
> perhaps allow absolute paths. I'm not sure what the reason for
> disallowing them was.

This is actually a regression. The absolute paths worked fine in 1.1.1
but it is also not clear to me why an absolute path would not work even
with the current master unless you set OPENSSL_CONF_INCLUDE. The
OPENSSL_CONF_INCLUDE is unconditionally prepended to the include path
so that is the reason why absolute paths do not work properly if you
set OPENSSL_CONF_INCLUDE.

--
Tomáš Mráz
No matter how far down the wrong road you've gone, turn back.
                                              Turkish proverb
[You'll know whether the road is wrong if you carefully listen to your
conscience.]


Reply | Threaded
Open this post in threaded view
|

Re: PRNG not available when multiple providers are configured?

Thomas Dwyer III
In reply to this post by Matt Caswell-2
On Tue, Nov 3, 2020 at 7:13 AM Matt Caswell <[hidden email]> wrote:


On 03/11/2020 00:55, Thomas Dwyer III wrote:
> I'm having trouble getting RAND_status() to return 1 when my openssl.cnf
> has both the default provider and the fips provider configured at the
> same time:
>
>         openssl_conf = openssl_init
>
>         [openssl_init]
>         providers = provider_sect
>
>         [provider_sect]
>         default = default_sect
>         fips = fips_sect
>
>         [default_sect]
>         activate = 1
>
>         .include /conf/openssl/fips.cnf
>
> If I remove either default or fips from [provider_sect] then
> RAND_status() returns 1. If I leave them both specified there,
> RAND_status() always returns 0. Is this the expected behavior or am I
> doing something wrong? I understand that I must specify properties when
> fetching algorithms in order to get deterministic behavior with multiple
> providers loaded. Is there an analogous API for the PRNG that I'm
> overlooking?
>
> Interestingly, setting activate=0 for either provider is not sufficient
> to work around this issue.

I tested this out and was able to replicate your behaviour.

The reasons are a little complicated (see below) but the TL;DR summary
is that there is an error in your config file. The ".include" line
should specify a config file relative to OPENSSLDIR (or
OPENSSL_CONF_INCLUDE if it is set). It cannot be an absolute path, and
hence fips.cnf is not being found.


This explanation does not match my observations. strace clearly shows my fips.cnf getting opened and read when my openssl.cnf has an absolute path. Likewise, strace shows stat64("fips.cnf", ...) failing (and thus no subsequent open() call) when I use a relative path. Furthermore, the documentation at https://www.openssl.org/docs/manmaster/man5/config.html says this should be an absolute path.

That said, see below..



I've seen this error a few times now so I'm thinking that we should
perhaps allow absolute paths. I'm not sure what the reason for
disallowing them was.

The reason it works if you comment out the "default" line is because
that means the only provider left is the FIPS one. But the config line
for that is faulty and therefore activating it fails. Ultimately we have
not succesfully activated any provider. When you call RAND_status() it
will attempt to fetch the RAND implementation and find that no providers
have been activated. In this case we fallback and automatically activate
the default provider. Hence you end up with RAND_status() still working.

If you comment out the "fips" line then it works because it doesn't
attempt to do anything with the fips provider, successfully activates
the default provider, and hence RAND_status() works as expected.

If you have both lines in the config file then it first successfully
activates the default provider. It next attempts to activate the fips
provider and fails. The way the config code works is that if any of the
configured providers fail to activate then it backs out and deactivates
all of them. At this point we're in a situation where a provider has
been successfully activated and then deactivated again. The fallback
activation of the default provider only kicks in if you've not attempted
to activate any providers by the time you first need one. Therefore the
default provider doesn't activate as a fallback either. Ultimately you
end up with no active providers and RAND_status() fails.

Ah ha! This explanation makes sense to me and indeed pointed me at the real problem. I had recompiled OpenSSL but I forgot to update the hmac in fips.cnf via fipsinstall. So yes, the fips provider was failing to activate because of that. As soon I fixed the hmac RAND_status() started working for me. So THANKS for that! :-)


Tom.III




We really should have a way of getting more verbose output in the event
of config issues like this.

Matt
Reply | Threaded
Open this post in threaded view
|

Re: PRNG not available when multiple providers are configured?

Dr Paul Dale
Adding:
    config_diagnostics = 1
At the same level as the openssl_conf line should produce more output.

Pauli
-- 
Dr Paul Dale | Distinguished Architect | Cryptographic Foundations 
Phone +61 7 3031 7217
Oracle Australia




On 4 Nov 2020, at 4:41 am, Thomas Dwyer III <[hidden email]> wrote:

On Tue, Nov 3, 2020 at 7:13 AM Matt Caswell <[hidden email]> wrote:


On 03/11/2020 00:55, Thomas Dwyer III wrote:

> I'm having trouble getting RAND_status() to return 1 when my openssl.cnf
> has both the default provider and the fips provider configured at the
> same time:
> 
>         openssl_conf = openssl_init
> 
>         [openssl_init]
>         providers = provider_sect
> 
>         [provider_sect]
>         default = default_sect
>         fips = fips_sect
> 
>         [default_sect]
>         activate = 1
> 
>         .include /conf/openssl/fips.cnf
> 
> If I remove either default or fips from [provider_sect] then
> RAND_status() returns 1. If I leave them both specified there,
> RAND_status() always returns 0. Is this the expected behavior or am I
> doing something wrong? I understand that I must specify properties when
> fetching algorithms in order to get deterministic behavior with multiple
> providers loaded. Is there an analogous API for the PRNG that I'm
> overlooking?
> 
> Interestingly, setting activate=0 for either provider is not sufficient
> to work around this issue.

I tested this out and was able to replicate your behaviour.

The reasons are a little complicated (see below) but the TL;DR summary
is that there is an error in your config file. The ".include" line
should specify a config file relative to OPENSSLDIR (or
OPENSSL_CONF_INCLUDE if it is set). It cannot be an absolute path, and
hence fips.cnf is not being found.


This explanation does not match my observations. strace clearly shows my fips.cnf getting opened and read when my openssl.cnf has an absolute path. Likewise, strace shows stat64("fips.cnf", ...) failing (and thus no subsequent open() call) when I use a relative path. Furthermore, the documentation at https://www.openssl.org/docs/manmaster/man5/config.html says this should be an absolute path.

That said, see below..



I've seen this error a few times now so I'm thinking that we should
perhaps allow absolute paths. I'm not sure what the reason for
disallowing them was.

The reason it works if you comment out the "default" line is because
that means the only provider left is the FIPS one. But the config line
for that is faulty and therefore activating it fails. Ultimately we have
not succesfully activated any provider. When you call RAND_status() it
will attempt to fetch the RAND implementation and find that no providers
have been activated. In this case we fallback and automatically activate
the default provider. Hence you end up with RAND_status() still working.

If you comment out the "fips" line then it works because it doesn't
attempt to do anything with the fips provider, successfully activates
the default provider, and hence RAND_status() works as expected.

If you have both lines in the config file then it first successfully
activates the default provider. It next attempts to activate the fips
provider and fails. The way the config code works is that if any of the
configured providers fail to activate then it backs out and deactivates
all of them. At this point we're in a situation where a provider has
been successfully activated and then deactivated again. The fallback
activation of the default provider only kicks in if you've not attempted
to activate any providers by the time you first need one. Therefore the
default provider doesn't activate as a fallback either. Ultimately you
end up with no active providers and RAND_status() fails.

Ah ha! This explanation makes sense to me and indeed pointed me at the real problem. I had recompiled OpenSSL but I forgot to update the hmac in fips.cnf via fipsinstall. So yes, the fips provider was failing to activate because of that. As soon I fixed the hmac RAND_status() started working for me. So THANKS for that! :-)


Tom.III




We really should have a way of getting more verbose output in the event
of config issues like this.

Matt

Reply | Threaded
Open this post in threaded view
|

Re: PRNG not available when multiple providers are configured?

Dr Paul Dale
In reply to this post by Thomas Dwyer III
Ah ha! This explanation makes sense to me and indeed pointed me at the real problem. I had recompiled OpenSSL but I forgot to update the hmac in fips.cnf via fipsinstall. So yes, the fips provider was failing to activate because of that. As soon I fixed the hmac RAND_status() started working for me. So THANKS for that! :-)

Not producing any diagnostic output for a failed checksum seems like a bug.


Pauli
-- 
Dr Paul Dale | Distinguished Architect | Cryptographic Foundations 
Phone +61 7 3031 7217
Oracle Australia


Reply | Threaded
Open this post in threaded view
|

Re: PRNG not available when multiple providers are configured?

Matt Caswell-2
In reply to this post by Tomas Mraz-2


On 03/11/2020 18:03, Tomas Mraz wrote:

> On Tue, 2020-11-03 at 15:13 +0000, Matt Caswell wrote:
>>
>> The reasons are a little complicated (see below) but the TL;DR
>> summary
>> is that there is an error in your config file. The ".include" line
>> should specify a config file relative to OPENSSLDIR (or
>> OPENSSL_CONF_INCLUDE if it is set). It cannot be an absolute path,
>> and
>> hence fips.cnf is not being found.
>>
>> I've seen this error a few times now so I'm thinking that we should
>> perhaps allow absolute paths. I'm not sure what the reason for
>> disallowing them was.
>
> This is actually a regression. The absolute paths worked fine in 1.1.1
> but it is also not clear to me why an absolute path would not work even
> with the current master unless you set OPENSSL_CONF_INCLUDE. The
> OPENSSL_CONF_INCLUDE is unconditionally prepended to the include path
> so that is the reason why absolute paths do not work properly if you
> set OPENSSL_CONF_INCLUDE.
>

This is indeed the case in my environment. I did have
OPENSSL_CONF_INCLUDE set - but I would expect an absolute path to
override it.

Matt
Reply | Threaded
Open this post in threaded view
|

Re: PRNG not available when multiple providers are configured?

Matt Caswell-2
In reply to this post by Dr Paul Dale
Ah! I had completely forgotten about this option.

Matt

On 03/11/2020 21:34, Dr Paul Dale wrote:

> Adding:
> |    config_diagnostics = 1|
> At the same level as the openssl_conf line should produce more output.
>
> Pauli
> -- 
> Dr Paul Dale | Distinguished Architect | Cryptographic Foundations 
> Phone +61 7 3031 7217
> Oracle Australia
>
>
>
>
>> On 4 Nov 2020, at 4:41 am, Thomas Dwyer III <[hidden email]
>> <mailto:[hidden email]>> wrote:
>>
>> On Tue, Nov 3, 2020 at 7:13 AM Matt Caswell <[hidden email]
>> <mailto:[hidden email]>> wrote:
>>
>>
>>
>>     On 03/11/2020 00:55, Thomas Dwyer III wrote:
>>     > I'm having trouble getting RAND_status() to return 1 when my
>>     openssl.cnf
>>     > has both the default provider and the fips provider configured
>>     at the
>>     > same time:
>>     > 
>>     >         openssl_conf = openssl_init
>>     > 
>>     >         [openssl_init]
>>     >         providers = provider_sect
>>     > 
>>     >         [provider_sect]
>>     >         default = default_sect
>>     >         fips = fips_sect
>>     > 
>>     >         [default_sect]
>>     >         activate = 1
>>     > 
>>     >         .include /conf/openssl/fips.cnf
>>     > 
>>     > If I remove either default or fips from [provider_sect] then
>>     > RAND_status() returns 1. If I leave them both specified there,
>>     > RAND_status() always returns 0. Is this the expected behavior or
>>     am I
>>     > doing something wrong? I understand that I must specify
>>     properties when
>>     > fetching algorithms in order to get deterministic behavior with
>>     multiple
>>     > providers loaded. Is there an analogous API for the PRNG that I'm
>>     > overlooking?
>>     > 
>>     > Interestingly, setting activate=0 for either provider is not
>>     sufficient
>>     > to work around this issue.
>>
>>     I tested this out and was able to replicate your behaviour.
>>
>>     The reasons are a little complicated (see below) but the TL;DR summary
>>     is that there is an error in your config file. The ".include" line
>>     should specify a config file relative to OPENSSLDIR (or
>>     OPENSSL_CONF_INCLUDE if it is set). It cannot be an absolute path, and
>>     hence fips.cnf is not being found.
>>
>>
>>
>> This explanation does not match my observations. strace clearly shows
>> my fips.cnf getting opened and read when my openssl.cnf has an
>> absolute path. Likewise, strace shows stat64("fips.cnf", ...) failing
>> (and thus no subsequent open() call) when I use a relative path.
>> Furthermore, the documentation
>> at https://www.openssl.org/docs/manmaster/man5/config.html
>> <https://urldefense.com/v3/__https://www.openssl.org/docs/manmaster/man5/config.html__;!!GqivPVa7Brio!INBlyBEyGaYipDhT7pzBHbqNWwT_X9g1JEID9D5HZmFB-cmNRMWGUEoRqaZMNvs$> says
>> this should be an absolute path.*
>> *
>>
>> That said, see below..
>>
>>
>>
>>     I've seen this error a few times now so I'm thinking that we should
>>     perhaps allow absolute paths. I'm not sure what the reason for
>>     disallowing them was.
>>
>>     The reason it works if you comment out the "default" line is because
>>     that means the only provider left is the FIPS one. But the config line
>>     for that is faulty and therefore activating it fails. Ultimately
>>     we have
>>     not succesfully activated any provider. When you call RAND_status() it
>>     will attempt to fetch the RAND implementation and find that no
>>     providers
>>     have been activated. In this case we fallback and automatically
>>     activate
>>     the default provider. Hence you end up with RAND_status() still
>>     working.
>>
>>     If you comment out the "fips" line then it works because it doesn't
>>     attempt to do anything with the fips provider, successfully activates
>>     the default provider, and hence RAND_status() works as expected.
>>
>>     If you have both lines in the config file then it first successfully
>>     activates the default provider. It next attempts to activate the fips
>>     provider and fails. The way the config code works is that if any
>>     of the
>>     configured providers fail to activate then it backs out and
>>     deactivates
>>     all of them. At this point we're in a situation where a provider has
>>     been successfully activated and then deactivated again. The fallback
>>     activation of the default provider only kicks in if you've not
>>     attempted
>>     to activate any providers by the time you first need one.
>>     Therefore the
>>     default provider doesn't activate as a fallback either. Ultimately you
>>     end up with no active providers and RAND_status() fails.
>>
>>
>> Ah ha! This explanation makes sense to me and indeed pointed me at the
>> real problem. I had recompiled OpenSSL but I forgot to update the hmac
>> in fips.cnf via fipsinstall. So yes, the fips provider was failing to
>> activate because of that. As soon I fixed the hmac RAND_status()
>> started working for me. So THANKS for that! :-)
>>
>>
>> Tom.III
>>
>>
>>
>>
>>     We really should have a way of getting more verbose output in the
>>     event
>>     of config issues like this.
>>
>>     Matt
>