What is the reason for error "SSL negotiation failed: error:04075070:rsa routines:RSA_sign:digest too big for rsa key"

classic Classic list List threaded Threaded
12 messages Options
Reply | Threaded
Open this post in threaded view
|

What is the reason for error "SSL negotiation failed: error:04075070:rsa routines:RSA_sign:digest too big for rsa key"

Nayna Jain

Hi all,

I have a server and client.

Both server and client authenticate each other. And so client also sends
the certificate.

Here client certificate has RSA 512 bits and md5. Server certificates has
RSA 1024 bits and md5.

Both server and client are using the API SSLv23_server_method() and
SSLv23_client_method() respectively

Opensssl version used is 1.0.1c.

So, when I initiate the connection from client, I get this error "SSL
negotiation failed: error:04075070:rsa routines:RSA_sign:digest too big for
rsa key" on client side.

Can someone please explain the reason for this error ?

If possible, also if someone can explain that how SSLv23 is able to
handshake with both TLSv1_2 related APIs as well as SSLv3 related APIs.

Thanks & Regards,
Nayna Jain

______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
User Support Mailing List                    [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: What is the reason for error "SSL negotiation failed: error:04075070:rsa routines:RSA_sign:digest too big for rsa key"

Viktor Dukhovni
On Tue, Feb 26, 2013 at 11:30:18PM +0530, Nayna Jain wrote:

> Both server and client authenticate each other. And so client also sends
> the certificate.
>
> Here client certificate has RSA 512 bits and md5. Server certificates has
> RSA 1024 bits and md5.

Bottom-line: DO NOT use 512-bit RSA moduli, they are trivially
factored on commodity hardware.

> Both server and client are using the API SSLv23_server_method() and
> SSLv23_client_method() respectively
>
> Opensssl version used is 1.0.1c.
>
> So, when I initiate the connection from client, I get this error "SSL
> negotiation failed: error:04075070:rsa routines:RSA_sign:digest too big for
> rsa key" on client side.
>
> Can someone please explain the reason for this error ?

        http://archives.neohapsis.com/archives/postfix/2013-02/0235.html

The negotiated TLSv1.2 digest produces output that is too wide to be
signed with an RSA 512-bit private key. The client key should be
at least 1024-bits, and in many cases stronger.

It is arguably the case that OpenSSL should not present a client
certificate that cannot sign using the agreed digest. One approach
is for the client to not offer ciphersuites that are too wide for
its private key.

This said, DO NOT use 512-bit RSA keys. I'm puzzled by their apparent
popularity, why does your client have such a key?

--
        Viktor.
______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
User Support Mailing List                    [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: What is the reason for error "SSL negotiation failed: error:04075070:rsa routines:RSA_sign:digest too big for rsa key"

Thulasi Goriparthi
Or use another hash type for signature which can produce not more than 53 bytes of hashed data. (i.e MD5, SHA1, SHA256, SHA384) while using 512-bit keys. OpenSSL by default uses SHA512 hash for signature. Change the code to use any other hash.

512 bit(64 byte) RSA key can only encrypt 53 bytes at max. 64 - 11 byte padding and SHA512 produces 64 bytes of hashed data.

Thanks,
Thulasi.

On Tue, Feb 26, 2013 at 11:42 PM, Viktor Dukhovni <[hidden email]> wrote:
On Tue, Feb 26, 2013 at 11:30:18PM +0530, Nayna Jain wrote:

> Both server and client authenticate each other. And so client also sends
> the certificate.
>
> Here client certificate has RSA 512 bits and md5. Server certificates has
> RSA 1024 bits and md5.

Bottom-line: DO NOT use 512-bit RSA moduli, they are trivially
factored on commodity hardware.

> Both server and client are using the API SSLv23_server_method() and
> SSLv23_client_method() respectively
>
> Opensssl version used is 1.0.1c.
>
> So, when I initiate the connection from client, I get this error "SSL
> negotiation failed: error:04075070:rsa routines:RSA_sign:digest too big for
> rsa key" on client side.
>
> Can someone please explain the reason for this error ?

        http://archives.neohapsis.com/archives/postfix/2013-02/0235.html

The negotiated TLSv1.2 digest produces output that is too wide to be
signed with an RSA 512-bit private key. The client key should be
at least 1024-bits, and in many cases stronger.

It is arguably the case that OpenSSL should not present a client
certificate that cannot sign using the agreed digest. One approach
is for the client to not offer ciphersuites that are too wide for
its private key.

This said, DO NOT use 512-bit RSA keys. I'm puzzled by their apparent
popularity, why does your client have such a key?

--
        Viktor.
______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
User Support Mailing List                    [hidden email]
Automated List Manager                           [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: What is the reason for error "SSL negotiation failed: error:04075070:rsa routines:RSA_sign:digest too big for rsa key"

Viktor Dukhovni
On Wed, Feb 27, 2013 at 12:49:55AM +0530, Thulasi Goriparthi wrote:

> Or use another hash type for signature which can produce not more than 53
> bytes of hashed data. (i.e MD5, SHA1, SHA256, SHA384) while using 512-bit
> keys. OpenSSL by default uses SHA512 hash for signature. Change the code to
> use any other hash.

Interestingly enough, it is in fact SHA384 that fails with RSA-512. The
client and server agree on:

        ECDHE-RSA-AES256-GCM-SHA384

> 512 bit(64 byte) RSA key can only encrypt 53 bytes at max. 64 - 11 byte
> padding and SHA512 produces 64 bytes of hashed data.

and the handshake fails when the client's key is RSA-512. Indeed
the shortest RSA key that seems to work is RSA-745, tests with
RSA-744 consistently fail. I don't know why the requisite key size
is substantially larger than the digest length + expected padding.

In any case, none of this should be exposed to the user. Ideally,
the client side should not offer ciphersuites it cannot use.
Perhaps the library does not generally know which if any client
key will be used until after the server's client certificate request.

The simplest answer is to avoid obsolete weak keys.

--
        Viktor.
______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
User Support Mailing List                    [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: What is the reason for error "SSL negotiation failed: error:04075070:rsa routines:RSA_sign:digest too big for rsa key"

Thulasi Goriparthi


On Wed, Feb 27, 2013 at 1:39 AM, Viktor Dukhovni <[hidden email]> wrote:
On Wed, Feb 27, 2013 at 12:49:55AM +0530, Thulasi Goriparthi wrote:

> Or use another hash type for signature which can produce not more than 53
> bytes of hashed data. (i.e MD5, SHA1, SHA256, SHA384) while using 512-bit
> keys. OpenSSL by default uses SHA512 hash for signature. Change the code to
> use any other hash.

Interestingly enough, it is in fact SHA384 that fails with RSA-512. The
client and server agree on:

        ECDHE-RSA-AES256-GCM-SHA384

Signature Hash type is not controlled by the CipherSuite and can be dynamically chosen by Signer. First two bytes of signature(prepended) will give us the information about the private key type and hash type that were used to do the signing. These additional two bytes will also be received along with signature for the verification.
 

> 512 bit(64 byte) RSA key can only encrypt 53 bytes at max. 64 - 11 byte
> padding and SHA512 produces 64 bytes of hashed data.

and the handshake fails when the client's key is RSA-512. Indeed
the shortest RSA key that seems to work is RSA-745, tests with
RSA-744 consistently fail. I don't know why the requisite key size
is substantially larger than the digest length + expected padding.

In any case, none of this should be exposed to the user. Ideally,
the client side should not offer ciphersuites it cannot use.
Perhaps the library does not generally know which if any client
key will be used until after the server's client certificate request.

The simplest answer is to avoid obsolete weak keys.

--
        Viktor.
______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
User Support Mailing List                    [hidden email]
Automated List Manager                           [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: What is the reason for error "SSL negotiation failed: error:04075070:rsa routines:RSA_sign:digest too big for rsa key"

Viktor Dukhovni
On Wed, Feb 27, 2013 at 01:55:24AM +0530, Thulasi Goriparthi wrote:

> > Interestingly enough, it is in fact SHA384 that fails with RSA-512. The
> > client and server agree on:
> >
> >         ECDHE-RSA-AES256-GCM-SHA384
> >
>
> Signature Hash type is not controlled by the CipherSuite and can be
> dynamically chosen by Signer. First two bytes of signature(prepended) will
> give us the information about the private key type and hash type that were
> used to do the signing. These additional two bytes will also be received
> along with signature for the verification.

Given:

    https://tools.ietf.org/html/rfc5246#section-7.4.8

        The hash and signature algorithms used in the signature MUST be
        one of those present in the supported_signature_algorithms field
        of the CertificateRequest message.  In addition, the hash and
        signature algorithms MUST be compatible with the key in the
        client's end-entity certificate.  RSA keys MAY be used with any
        permitted hash algorithm, subject to restrictions in the
        certificate, if any.

I took a look at the server response and saw that it offers:

        0601 SHA-512 + RSA
        0602 SHA-512 + DSA
        0603 SHA-512 + ECDSA

        0501 SHA-384 + RSA
        0502 SHA-384 + DSA
        0503 SHA-384 + ECDSA

        0401 SHA-256 + RSA
        0402 SHA-256 + DSA
        0403 SHA-256 + ECDSA

        0301 SHA-224 + RSA
        0302 SHA-224 + DSA
        0303 SHA-224 + ECDSA

        0201 SHA-1 + RSA
        0202 SHA-1 + DSA
        0203 SHA-1 + ECDSA

        0101 MD5 + RSA

So almost certainly the (OpenSSL) client chose SHA-512 + RSA, and
then fails, because while it supports the algorithm, it can't
actually do SHA-512 with the given key. This is I think a bug. The
client should choose the strongest digest compatible with its key.

And of course, more importantly, the client should not use weak keys.

--
        Viktor.
______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
User Support Mailing List                    [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: What is the reason for error "SSL negotiation failed: error:04075070:rsa routines:RSA_sign:digest too big for rsa key"

Dr. Stephen Henson
In reply to this post by Viktor Dukhovni
On Tue, Feb 26, 2013, Viktor Dukhovni wrote:

> On Wed, Feb 27, 2013 at 12:49:55AM +0530, Thulasi Goriparthi wrote:
>
> > Or use another hash type for signature which can produce not more than 53
> > bytes of hashed data. (i.e MD5, SHA1, SHA256, SHA384) while using 512-bit
> > keys. OpenSSL by default uses SHA512 hash for signature. Change the code to
> > use any other hash.
>
> Interestingly enough, it is in fact SHA384 that fails with RSA-512. The
> client and server agree on:
>
> ECDHE-RSA-AES256-GCM-SHA384
>
> > 512 bit(64 byte) RSA key can only encrypt 53 bytes at max. 64 - 11 byte
> > padding and SHA512 produces 64 bytes of hashed data.
>
> and the handshake fails when the client's key is RSA-512. Indeed
> the shortest RSA key that seems to work is RSA-745, tests with
> RSA-744 consistently fail. I don't know why the requisite key size
> is substantially larger than the digest length + expected padding.
>

The signature format for TLS 1.2 is the standard DigestInfo structures. For
SHA2 algorithms this adds an additional 19 bytes of structure before the
digest itself.

In (unreleased) OpenSSL 1.0.2 and later the signature algorithms used by
client authentication can be set by the application. For 1.0.1 it just includes
everything supported by the library and selects the first preference for the
key type specified. That means using OpenSSL 1.0.1 on both ends will use
SHA512 which is fine for everything but insecure key sizes with RSA.

It could be fixed to handle the insecure key sizes too but do we really want
people to use those?

Steve.
--
Dr Stephen N. Henson. OpenSSL project core developer.
Commercial tech support now available see: http://www.openssl.org
______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
User Support Mailing List                    [hidden email]
Automated List Manager                           [hidden email]
T J
Reply | Threaded
Open this post in threaded view
|

error:2D06B071:FIPS routines:FIPS_check_incore_fingerprint:fingerprint does not match segment aliasing .

T J
Hi All

I need some help with this one please...

What is segment aliasing and how can I fix this error?

In my makefile, I essentially have:

     export CROSS_COMPILE=$(CROSS_COMPILE) && \
     export SYSTEM=Linux && \
     export MACHINE=ppc && \
     export RELEASE=2.4 && \
     export FIPS_SIG=$(FIPS_SIG) && \
     export FIPSLD_CC=$(CROSS) && \
     export HOSTCC=$(HOSTCC) && \
     cd $(CURDIR)/openssl/openssl-fips-2.0 && \
     ./config no-asm && \
     $(MAKE) && \
     $(MAKE) install INSTALLTOP=$(SSLDIR)/fips
...
     export FIPS_SIG=$(FIPS_SIG) && \
     cd $(CURDIR)/openssl/openssl && \
     ./Configure fips no-asm no-hw shared
--install_prefix="$(SSLDIR)/base" --prefix="/usr"
--openssldir="/usr/extended/bin" --with-fipsdir="$(SSLDIR)/fips"
$(CROSS) && \
     $(MAKE) && \
     $(MAKE) install_sw

where:
FIPS_SIG = openssl-fips-2.0/util/incore
CROSS_COMPILE = ppc_8xx-
CROSS = $(CROSS_COMPILE)gcc

Where am I going wrong?? or how can I debug to find out more info...?


______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
User Support Mailing List                    [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: What is the reason for error "SSL negotiation failed: error:04075070:rsa routines:RSA_sign:digest too big for rsa key"

Nayna Jain
In reply to this post by Dr. Stephen Henson
Thanks Dr. Stephen and Victor for explanation

Some questions further in this :

   Is there an API to configure programmatically , what hash algorithm it
   can use.  My understanding is that final ciphers which are selected are
   for encryption and HMAC generation of application data packets. This
   might be required to maintain the compatibility issues.
   My understanding is that this error happens on client side, when it
   tries to sign the CertificateVerify message, where digest output comes
   too big for RSA to encrypt. Let me know if this is correct.

Thanks & Regards,
Nayna Jain




From: "Dr. Stephen Henson" <[hidden email]>
To: [hidden email]
Date: 02/27/2013 04:26 AM
Subject: Re: What is the reason for error "SSL negotiation failed:
            error:04075070:rsa routines:RSA_sign:digest too big for rsa
            key"
Sent by: [hidden email]



On Tue, Feb 26, 2013, Viktor Dukhovni wrote:

> On Wed, Feb 27, 2013 at 12:49:55AM +0530, Thulasi Goriparthi wrote:
>
> > Or use another hash type for signature which can produce not more than
53
> > bytes of hashed data. (i.e MD5, SHA1, SHA256, SHA384) while using
512-bit
> > keys. OpenSSL by default uses SHA512 hash for signature. Change the
code to

> > use any other hash.
>
> Interestingly enough, it is in fact SHA384 that fails with RSA-512. The
> client and server agree on:
>
> ECDHE-RSA-AES256-GCM-SHA384
>
> > 512 bit(64 byte) RSA key can only encrypt 53 bytes at max. 64 - 11 byte
> > padding and SHA512 produces 64 bytes of hashed data.
>
> and the handshake fails when the client's key is RSA-512. Indeed
> the shortest RSA key that seems to work is RSA-745, tests with
> RSA-744 consistently fail. I don't know why the requisite key size
> is substantially larger than the digest length + expected padding.
>

The signature format for TLS 1.2 is the standard DigestInfo structures. For
SHA2 algorithms this adds an additional 19 bytes of structure before the
digest itself.

In (unreleased) OpenSSL 1.0.2 and later the signature algorithms used by
client authentication can be set by the application. For 1.0.1 it just
includes
everything supported by the library and selects the first preference for
the
key type specified. That means using OpenSSL 1.0.1 on both ends will use
SHA512 which is fine for everything but insecure key sizes with RSA.

It could be fixed to handle the insecure key sizes too but do we really
want
people to use those?

Steve.
--
Dr Stephen N. Henson. OpenSSL project core developer.
Commercial tech support now available see: http://www.openssl.org
______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
User Support Mailing List                    [hidden email]
Automated List Manager                           [hidden email]



______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
User Support Mailing List                    [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: What is the reason for error "SSL negotiation failed: error:04075070:rsa routines:RSA_sign:digest too big for rsa key"

Dr. Stephen Henson
On Wed, Feb 27, 2013, Nayna Jain wrote:

> Thanks Dr. Stephen and Victor for explanation
>
> Some questions further in this :
>
>    Is there an API to configure programmatically , what hash algorithm it
>    can use.  My understanding is that final ciphers which are selected are
>    for encryption and HMAC generation of application data packets. This
>    might be required to maintain the compatibility issues.
>    My understanding is that this error happens on client side, when it
>    tries to sign the CertificateVerify message, where digest output comes
>    too big for RSA to encrypt. Let me know if this is correct.
>

Yes that's correct. If the RSA key size is too small the default SHA512
digest and the associated ASN1 and padding overhead exceeds the maximum for
the RSA algorithm. This is only a problem with insecure keys: most standards
now suggest 1024 bits is the bare minimum and 2048 bits recommended.

There is an API to modify the supported digest list but only in the unreleased
OpenSSL 1.0.2 and later. This can be accessed on the command line through the
-client_sigalgs command line switch, via the SSL_CONF API or through some ctrl
operations. There isn't anything in 1.0.1.

These are the digests used to sign handshake messages for TLS version 1.2.
The digests used for HMAC in application data records are determined by the
cipher suite.

Steve.
--
Dr Stephen N. Henson. OpenSSL project core developer.
Commercial tech support now available see: http://www.openssl.org
______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
User Support Mailing List                    [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: What is the reason for error "SSL negotiation failed: error:04075070:rsa routines:RSA_sign:digest too big for rsa key"

Jakob Bohm-7
On 2/27/2013 3:14 PM, Dr. Stephen Henson wrote:

> On Wed, Feb 27, 2013, Nayna Jain wrote:
>
>> Thanks Dr. Stephen and Victor for explanation
>>
>> Some questions further in this :
>>
>>     Is there an API to configure programmatically , what hash algorithm it
>>     can use.  My understanding is that final ciphers which are selected are
>>     for encryption and HMAC generation of application data packets. This
>>     might be required to maintain the compatibility issues.
>>     My understanding is that this error happens on client side, when it
>>     tries to sign the CertificateVerify message, where digest output comes
>>     too big for RSA to encrypt. Let me know if this is correct.
>>
>
> Yes that's correct. If the RSA key size is too small the default SHA512
> digest and the associated ASN1 and padding overhead exceeds the maximum for
> the RSA algorithm. This is only a problem with insecure keys: most standards
> now suggest 1024 bits is the bare minimum and 2048 bits recommended.
>
> There is an API to modify the supported digest list but only in the unreleased
> OpenSSL 1.0.2 and later. This can be accessed on the command line through the
> -client_sigalgs command line switch, via the SSL_CONF API or through some ctrl
> operations. There isn't anything in 1.0.1.
>
> These are the digests used to sign handshake messages for TLS version 1.2.
> The digests used for HMAC in application data records are determined by the
> cipher suite.

I still think this is a real bug in 1.0.1 (not the absence of an API,
but the failure to implement the hash algorithm selection correctly).

*Nothing in the following description requires any changes to the
OpenSSL API, just correct implementation of the protocol handling in
existing APIs*

To understand the problem more fully, imagine the following scenario:

1. Client user has configured 2 or more client certificates, one has
   an RSA-768 public key, the other an RSA-16384 key (throw in the
   possibility of DSA keys for extra credit).

2. Client connects to server without knowing in advance what
   certificates and algorithms the server will ask for (so client
   app code cannot reasonably limit its signing hash choices at
   this step, so no extra APIs are useful).

3. Server negotiates a connection and presents a list of acceptable
   client certificate issuers with an explicit (TLSv1.2) or implicit
   (older TLS versions)
   CertificateRequest.supported_signature_algorithms list.

4. Based on the usual logic, OpenSSL client code narrows down the
   list of available client certificates, mostly based on issuer
   and validity periods.

     In this stage code is missing to eliminate certificates whose
   public key is incompatible with ALL elements of
   CertificateRequest.supported_signature_algorithms.
    *This code is not minimally required, but really should be
   there to reduce user confusion*

   For example if the server list contains only SHA-512 + RSA and
   SHA-256 + ECDSA, then the users RSA-768 certificate goes away as it
   cannot be used with SHA-512+RSA (due to hash length), nor with
   SHA-256+ECDSA (due to not being a DSA certificate).

   But if the server offered the long list of choices in Viktor
   Dukhovni's post from yesterday at 22:21 UTC, then both certificates
   remain as possibilities.

5. Using the usual application specific mechanism, the user/app code
   chooses one of the remaining client certificates.

6. Now the OpenSSL client code has a chosen certificate with a given
   public key, a matching private key and a list of acceptable signing
   algorithms from CertificateRequest.supported_signature_algorithms.

    THE BUGGY BEHAVIOR is to choose an element of
   CertificateRequest.supported_signature_algorithms which is not
   compatible with the chosen certificate, and then error out.

    THE REQUIRED BEHAVIOR (as stated in RFC5246 sections 7.4.4, 7.4.6
   and 7.4.8) is for the OpenSSL client code to choose an element of
   CertificateRequest.supported_signature_algorithms which is
   compatible with the chosen client certificate.

    THE RECOMMENDED BEHAVIOR is to use the strongest such element,
   but without choosing something impossible.




Enjoy

Jakob
--
Jakob Bohm, CIO, Partner, WiseMo A/S.  http://www.wisemo.com
Transformervej 29, 2730 Herlev, Denmark.  Direct +45 31 13 16 10
This public discussion message is non-binding and may contain errors.
WiseMo - Remote Service Management for PCs, Phones and Embedded
______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
User Support Mailing List                    [hidden email]
Automated List Manager                           [hidden email]
T J
Reply | Threaded
Open this post in threaded view
|

incore

T J
In reply to this post by T J

Does anyone know what the incore script is supposed to do in some detail
or point me to some documentation about it? I don't know perl and am
struggling to get past a fingerprint mismatch error when attempting to
enter FIPS mode. In the previous version (0.9.8r + FIPSv1.2.3), where
incore was written as a shell script, we were able to change a value
called "INCORE_ADJUST" which solved the fingerprint mismatch error for
our platform. With the new version however, the script is in perl and
since I don't know what the INCORE_ADJUST actually did, or know how to
read/write perl, I need some help...please...
______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
User Support Mailing List                    [hidden email]
Automated List Manager                           [hidden email]