PKCS#7/CMS verify reports bad signature

classic Classic list List threaded Threaded
12 messages Options
Reply | Threaded
Open this post in threaded view
|

PKCS#7/CMS verify reports bad signature

Steffen
Hello,

I am struggling with using OpenSSL 1.1.1 to verify a PKCS #7/CMS structure. Verification succeeds when I use OpenSSL 1.0.2, but 1.1.0 and 1.1.1 fails with "bad signature". I initially had this problem when using the OpenSSL library but I see that the problem also applies to the OpenSSL CLI.

I am at loss and need some help with this issue. Please see the commands I used below. Thank you for any assistance you can provide!

Notes:
  • "-noverify" was used because the certificates expired.
  • Verification succeeds when specifying "-nosigs".
  • "openssl cms -verify [...]" behaves the same way.
  • Since the files I am working with (test.der and test-data.bin) are part of a private project, I am not ready to share these in public.
  • I do not know exactly how the message structure was created but I guess either with some OpenSSL 1.0.2, Java with or without BouncyCastle.
Commands used:

# Environment: macOS 10.14.3 / Homebrew
 
$ /usr/local/opt/openssl/bin/openssl version
OpenSSL 1.0.2r  26 Feb 2019
 
$ /usr/local/opt/openssl/bin/openssl smime -verify -inform der -in test.der -content test-data.bin -noverify
Verification successful
 
$ /usr/local/opt/openssl\@1.1/bin/openssl version
OpenSSL 1.1.1b  26 Feb 2019
 
$ /usr/local/opt/openssl\@1.1/bin/openssl smime -verify -inform der -in test.der -content test-data.bin -noverify
Verification failure
4563408320:error:04091068:rsa routines:int_rsa_verify:bad signature:crypto/rsa/rsa_sign.c:220:
4563408320:error:21071069:PKCS7 routines:PKCS7_signatureVerify:signature failure:crypto/pkcs7/pk7_doit.c:1037:
4563408320:error:21075069:PKCS7 routines:PKCS7_verify:signature failure:crypto/pkcs7/pk7_smime.c:353:
Reply | Threaded
Open this post in threaded view
|

Re: PKCS#7/CMS verify reports bad signature

Matt Caswell-2


On 01/04/2019 14:46, Steffen wrote:
> Hello,
>
> I am struggling with using OpenSSL 1.1.1 to verify a PKCS #7/CMS structure.
> Verification succeeds when I use OpenSSL 1.0.2, but 1.1.0 and 1.1.1 fails with
> "bad signature". I initially had this problem when using the OpenSSL library but
> I see that the problem also applies to the OpenSSL CLI.

Could be this (from CHANGES):

  *) Fixed a text canonicalisation bug in CMS

     Where a CMS detached signature is used with text content the text goes
     through a canonicalisation process first prior to signing or verifying a
     signature. This process strips trailing space at the end of lines, converts
     line terminators to CRLF and removes additional trailing line terminators
     at the end of a file. A bug in the canonicalisation process meant that
     some characters, such as form-feed, were incorrectly treated as whitespace
     and removed. This is contrary to the specification (RFC5485). This fix
     could mean that detached text data signed with an earlier version of
     OpenSSL 1.1.0 may fail to verify using the fixed version, or text data
     signed with a fixed OpenSSL may fail to verify with an earlier version of
     OpenSSL 1.1.0. A workaround is to only verify the canonicalised text data
     and use the "-binary" flag (for the "cms" command line application) or set
     the SMIME_BINARY/PKCS7_BINARY/CMS_BINARY flags (if using CMS_verify()).
     [Matt Caswell]

Matt

>
> I am at loss and need some help with this issue. Please see the commands I used
> below. Thank you for any assistance you can provide!
>
> Notes:
>
>   * "-noverify" was used because the certificates expired.
>   * Verification succeeds when specifying "-nosigs".
>   * "openssl cms -verify [...]" behaves the same way.
>   * Since the files I am working with (test.der and test-data.bin) are part of a
>     private project, I am not ready to share these in public.
>   * I do not know exactly how the message structure was created but I guess
>     either with some OpenSSL 1.0.2, Java with or without BouncyCastle.
>
> Commands used:
>
> # Environment: macOS 10.14.3 / Homebrew
>  
> $ /usr/local/opt/openssl/bin/openssl version
> OpenSSL 1.0.2r  26 Feb 2019
>  
> $ /usr/local/opt/openssl/bin/openssl smime -verify -inform der -in test.der
> -content test-data.bin -noverify
> Verification successful
>  
> $ /usr/local/opt/openssl\@1.1/bin/openssl version
> OpenSSL 1.1.1b  26 Feb 2019
>  
> $ /usr/local/opt/openssl\@1.1/bin/openssl smime -verify -inform der -in test.der
> -content test-data.bin -noverify
> Verification failure
> 4563408320:error:04091068:rsa routines:int_rsa_verify:bad
> signature:crypto/rsa/rsa_sign.c:220:
> 4563408320:error:21071069:PKCS7 routines:PKCS7_signatureVerify:signature
> failure:crypto/pkcs7/pk7_doit.c:1037:
> 4563408320:error:21075069:PKCS7 routines:PKCS7_verify:signature
> failure:crypto/pkcs7/pk7_smime.c:353:
Reply | Threaded
Open this post in threaded view
|

Re: PKCS#7/CMS verify reports bad signature

Steffen
Hello Matt,

Thank you for your reply!

I am not quite sure if I should do something more but specifying "-binary" alone does not seem to help:

# 1.0.2r
$ /usr/local/opt/openssl/bin/openssl cms -verify -inform der -in test.der -content test-data.bin -noverify -binary > /dev/null
Verification successful

# 1.1.1b
$ /usr/local/opt/openssl\@1.1/bin/openssl cms -verify -inform der -in test.der -content test-data.bin -noverify -binary > /dev/null
Verification failure
4465374656:error:04091068:rsa routines:int_rsa_verify:bad signature:crypto/rsa/rsa_sign.c:220:
4465374656:error:2E09809E:CMS routines:CMS_SignerInfo_verify:verification failure:crypto/cms/cms_sd.c:741:


Other ideas are much appreciated!

On Mon, Apr 1, 2019 at 3:58 PM Matt Caswell <[hidden email]> wrote:


On 01/04/2019 14:46, Steffen wrote:
> Hello,
>
> I am struggling with using OpenSSL 1.1.1 to verify a PKCS #7/CMS structure.
> Verification succeeds when I use OpenSSL 1.0.2, but 1.1.0 and 1.1.1 fails with
> "bad signature". I initially had this problem when using the OpenSSL library but
> I see that the problem also applies to the OpenSSL CLI.

Could be this (from CHANGES):

  *) Fixed a text canonicalisation bug in CMS

     Where a CMS detached signature is used with text content the text goes
     through a canonicalisation process first prior to signing or verifying a
     signature. This process strips trailing space at the end of lines, converts
     line terminators to CRLF and removes additional trailing line terminators
     at the end of a file. A bug in the canonicalisation process meant that
     some characters, such as form-feed, were incorrectly treated as whitespace
     and removed. This is contrary to the specification (RFC5485). This fix
     could mean that detached text data signed with an earlier version of
     OpenSSL 1.1.0 may fail to verify using the fixed version, or text data
     signed with a fixed OpenSSL may fail to verify with an earlier version of
     OpenSSL 1.1.0. A workaround is to only verify the canonicalised text data
     and use the "-binary" flag (for the "cms" command line application) or set
     the SMIME_BINARY/PKCS7_BINARY/CMS_BINARY flags (if using CMS_verify()).
     [Matt Caswell]

Matt

>
> I am at loss and need some help with this issue. Please see the commands I used
> below. Thank you for any assistance you can provide!
>
> Notes:
>
>   * "-noverify" was used because the certificates expired.
>   * Verification succeeds when specifying "-nosigs".
>   * "openssl cms -verify [...]" behaves the same way.
>   * Since the files I am working with (test.der and test-data.bin) are part of a
>     private project, I am not ready to share these in public.
>   * I do not know exactly how the message structure was created but I guess
>     either with some OpenSSL 1.0.2, Java with or without BouncyCastle.
>
> Commands used:
>
> # Environment: macOS 10.14.3 / Homebrew
>  
> $ /usr/local/opt/openssl/bin/openssl version
> OpenSSL 1.0.2r  26 Feb 2019
>  
> $ /usr/local/opt/openssl/bin/openssl smime -verify -inform der -in test.der
> -content test-data.bin -noverify
> Verification successful
>  
> $ /usr/local/opt/openssl\@1.1/bin/openssl version
> OpenSSL 1.1.1b  26 Feb 2019
>  
> $ /usr/local/opt/openssl\@1.1/bin/openssl smime -verify -inform der -in test.der
> -content test-data.bin -noverify
> Verification failure
> 4563408320:error:04091068:rsa routines:int_rsa_verify:bad
> signature:crypto/rsa/rsa_sign.c:220:
> 4563408320:error:21071069:PKCS7 routines:PKCS7_signatureVerify:signature
> failure:crypto/pkcs7/pk7_doit.c:1037:
> 4563408320:error:21075069:PKCS7 routines:PKCS7_verify:signature
> failure:crypto/pkcs7/pk7_smime.c:353:
Reply | Threaded
Open this post in threaded view
|

Re: PKCS#7/CMS verify reports bad signature

Steffen
Hello,

I believe that I have narrowed the problem down to one specific version of OpenSSL. Version 1.1.0b works as expected while OpenSSL 1.1.0c does not.

I have currently only verified this using PKCS7_verify and CMS_verify since I have no CLI at hand for these versions.

The changelog for 1.1.0c describes the following change for CMS:

  *) CMS Null dereference

     Applications parsing invalid CMS structures can crash with a NULL pointer
     dereference. This is caused by a bug in the handling of the ASN.1 CHOICE
     type in OpenSSL 1.1.0 which can result in a NULL value being passed to the
     structure callback if an attempt is made to free certain invalid encodings.
     Only CHOICE structures using a callback which do not handle NULL value are
     affected.

     This issue was reported to OpenSSL by Tyler Nighswander of ForAllSecure.
     (CVE-2016-7053)
     [Stephen Henson]
Reply | Threaded
Open this post in threaded view
|

Re: PKCS#7/CMS verify reports bad signature

Matt Caswell-2


On 01/04/2019 22:23, Steffen wrote:
> Hello,
>
> I believe that I have narrowed the problem down to one specific version of
> OpenSSL. Version 1.1.0b works as expected while OpenSSL 1.1.0c does not.

Using the cert/data files you provided me off-list (thanks), I was able to
confirm the above and narrow it down further to the following commit:

commit b71079a375116a8a52ed493afcd8f69cb08c195a
Author: David Benjamin <[hidden email]>
Date:   Sat Aug 20 13:35:17 2016 -0400

    Implement RSASSA-PKCS1-v1_5 as specified.

    RFC 3447, section 8.2.2, steps 3 and 4 states that verifiers must encode
    the DigestInfo struct and then compare the result against the public key
    operation result. This implies that one and only one encoding is legal.

    OpenSSL instead parses with crypto/asn1, then checks that the encoding
    round-trips, and allows some variations for the parameter. Sufficient
    laxness in this area can allow signature forgeries, as described in
    https://www.imperialviolet.org/2014/09/26/pkcs1.html

    Although there aren't known attacks against OpenSSL's current scheme,
    this change makes OpenSSL implement the algorithm as specified. This
    avoids the uncertainty and, more importantly, helps grow a healthy
    ecosystem. Laxness beyond the spec, particularly in implementations
    which enjoy wide use, risks harm to the ecosystem for all. A signature
    producer which only tests against OpenSSL may not notice bugs and
    accidentally become widely deployed. Thus implementations have a
    responsibility to honor the specification as tightly as is practical.

    In some cases, the damage is permanent and the spec deviation and
    security risk becomes a tax all implementors must forever pay, but not
    here. Both BoringSSL and Go successfully implemented and deployed
    RSASSA-PKCS1-v1_5 as specified since their respective beginnings, so
    this change should be compatible enough to pin down in future OpenSSL
    releases.

    See also https://tools.ietf.org/html/draft-thomson-postel-was-wrong-00

    As a bonus, by not having to deal with sign/verify differences, this
    version is also somewhat clearer. It also more consistently enforces
    digest lengths in the verify_recover codepath. The NID_md5_sha1 codepath
    wasn't quite doing this right.

    Reviewed-by: Kurt Roeckx <[hidden email]>
    Reviewed-by: Rich Salz <[hidden email]>

    GH: #1474
    (cherry picked from commit 608a026494c1e7a14f6d6cfcc5e4994fe2728836)

Implemented via this pull request:
https://github.com/openssl/openssl/pull/1474

So, based on the above description, it appears that older versions of OpenSSL
were unduly lenient in tolerating incorrectly formatted signatures. As a
security hardening measure that tolerance was removed. If you want to know more
then David Benjamin may be able to expand.

Matt

Reply | Threaded
Open this post in threaded view
|

Re: PKCS#7/CMS verify reports bad signature

Steffen
Hello Matt,

Thank you for looking into this!

So it seems like I have to figure out why the signatures are incorrectly formatted and then fix it at every source if possible, or convert the structures somehow if it can be done correctly. The only immediate solution I can see is to downgrade to OpenSSL 1.0.2r or 1.1.0b until I figure out what to do about it. These seem like pretty bad ideas right now but I am blocked otherwise.

I do not have any further questions because I guess this is a dead end if I cannot fix this correctly, but please feel free to share any thoughts. CC-ing David Benjamin as well.

Thank you!

Regards,
Steffen
Reply | Threaded
Open this post in threaded view
|

Re: PKCS#7/CMS verify reports bad signature

OpenSSL - User mailing list
In reply to this post by Matt Caswell-2
On 02/04/2019 10:44, Matt Caswell wrote:

>
> On 01/04/2019 22:23, Steffen wrote:
>> Hello,
>>
>> I believe that I have narrowed the problem down to one specific version of
>> OpenSSL. Version 1.1.0b works as expected while OpenSSL 1.1.0c does not.
> Using the cert/data files you provided me off-list (thanks), I was able to
> confirm the above and narrow it down further to the following commit:
>
> commit b71079a375116a8a52ed493afcd8f69cb08c195a
> Author: David Benjamin <[hidden email]>
> Date:   Sat Aug 20 13:35:17 2016 -0400
>
>      Implement RSASSA-PKCS1-v1_5 as specified.
>
>      RFC 3447, section 8.2.2, steps 3 and 4 states that verifiers must encode
>      the DigestInfo struct and then compare the result against the public key
>      operation result. This implies that one and only one encoding is legal.
>
>      OpenSSL instead parses with crypto/asn1, then checks that the encoding
>      round-trips, and allows some variations for the parameter. Sufficient
>      laxness in this area can allow signature forgeries, as described in
>      https://www.imperialviolet.org/2014/09/26/pkcs1.html
>
>      Although there aren't known attacks against OpenSSL's current scheme,
>      this change makes OpenSSL implement the algorithm as specified. This
>      avoids the uncertainty and, more importantly, helps grow a healthy
>      ecosystem. Laxness beyond the spec, particularly in implementations
>      which enjoy wide use, risks harm to the ecosystem for all. A signature
>      producer which only tests against OpenSSL may not notice bugs and
>      accidentally become widely deployed. Thus implementations have a
>      responsibility to honor the specification as tightly as is practical.
>
>      In some cases, the damage is permanent and the spec deviation and
>      security risk becomes a tax all implementors must forever pay, but not
>      here. Both BoringSSL and Go successfully implemented and deployed
>      RSASSA-PKCS1-v1_5 as specified since their respective beginnings, so
>      this change should be compatible enough to pin down in future OpenSSL
>      releases.
>
>      See also https://tools.ietf.org/html/draft-thomson-postel-was-wrong-00
>
>      As a bonus, by not having to deal with sign/verify differences, this
>      version is also somewhat clearer. It also more consistently enforces
>      digest lengths in the verify_recover codepath. The NID_md5_sha1 codepath
>      wasn't quite doing this right.
>
>      Reviewed-by: Kurt Roeckx <[hidden email]>
>      Reviewed-by: Rich Salz <[hidden email]>
>
>      GH: #1474
>      (cherry picked from commit 608a026494c1e7a14f6d6cfcc5e4994fe2728836)
>
> Implemented via this pull request:
> https://github.com/openssl/openssl/pull/1474
>
> So, based on the above description, it appears that older versions of OpenSSL
> were unduly lenient in tolerating incorrectly formatted signatures. As a
> security hardening measure that tolerance was removed. If you want to know more
> then David Benjamin may be able to expand.
Please note that CMS countersignatures made using a specific Symantec
server
(specifically to timestamp the original CMS signature with a long-valid
signed
timestamp) happened to use a different PKCS#1.x signature format until very
recently.  This signature had a different but similar input to the raw RSA
algorithm, and was only ever done with Sha1RSA.

According to my notes, this format was only ever used with sha1RSA and
md5RSA,
In this format, the DER encoding and hash OID is skipped, and instead
the raw
20 or 16 byte hash is placed where the DER encoded tuple should be.  The
surrounding padding was apparently the same is in PKCS#1.5 signatures,
as was
the OIDs identifying this algorithm.

I don't know if an older PKCS#1 document (before 1.5) actually specified
this
format, only that is was present in the wild.



Enjoy

Jakob
--
Jakob Bohm, CIO, Partner, WiseMo A/S.  https://www.wisemo.com
Transformervej 29, 2860 Søborg, Denmark.  Direct +45 31 13 16 10
This public discussion message is non-binding and may contain errors.
WiseMo - Remote Service Management for PCs, Phones and Embedded

Reply | Threaded
Open this post in threaded view
|

Re: PKCS#7/CMS verify reports bad signature

Michael Richardson
In reply to this post by Matt Caswell-2

Matt Caswell <[hidden email]> wrote:
    > Using the cert/data files you provided me off-list (thanks), I was able to
    > confirm the above and narrow it down further to the following commit:

What had produced the signatures?

    > In some cases, the damage is permanent and the spec deviation and
    > security risk becomes a tax all implementors must forever pay, but not
    > here. Both BoringSSL and Go successfully implemented and deployed
    > RSASSA-PKCS1-v1_5 as specified since their respective beginnings, so
    > this change should be compatible enough to pin down in future OpenSSL
    > releases.

    > So, based on the above description, it appears that older versions of OpenSSL
    > were unduly lenient in tolerating incorrectly formatted signatures. As a
    > security hardening measure that tolerance was removed. If you want to know more
    > then David Benjamin may be able to expand.

Did openssl ever produce these wrong signatures?




signature.asc (497 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: PKCS#7/CMS verify reports bad signature

Steffen
Hello,

> What had produced the signatures?

I received word from my end that the signatures may have been produced by OpenSSL 1.0.2 (no idea which letter release) in the Cygwin environment but I cannot confirm this.

Reply | Threaded
Open this post in threaded view
|

Re: PKCS#7/CMS verify reports bad signature

Matt Caswell-2


On 02/04/2019 17:34, Steffen wrote:
> Hello,
>
>> What had produced the signatures?
>
> I received word from my end that the signatures may have been produced by
> OpenSSL 1.0.2 (no idea which letter release) in the Cygwin environment but I
> cannot confirm this.
>

If that's the case, I'd really like to know what specific version and how the
signatures were generated (although it seems a little surprising if 1.0.2 is
creating these incorrect signatures that no-one else has encountered this, since
the commit in question went in over 2.5 years ago).

Matt
Reply | Threaded
Open this post in threaded view
|

Re: PKCS#7/CMS verify reports bad signature

Steffen
Hello,

I think the person I spoke with might have thought about another set of signatures for an in-house identity provider. If that is the case then those signatures were probably generated by OpenSSL 1.0.2 and are OK. I heard from another person today that the bad files were produced by the other primary identity provider we use, so we must support the existing format. Now I really do not see any other solution but to either downgrade or fork OpenSSL.

On Wed, Apr 3, 2019 at 9:59 AM Matt Caswell <[hidden email]> wrote:


On 02/04/2019 17:34, Steffen wrote:
> Hello,
>
>> What had produced the signatures?
>
> I received word from my end that the signatures may have been produced by
> OpenSSL 1.0.2 (no idea which letter release) in the Cygwin environment but I
> cannot confirm this.
>

If that's the case, I'd really like to know what specific version and how the
signatures were generated (although it seems a little surprising if 1.0.2 is
creating these incorrect signatures that no-one else has encountered this, since
the commit in question went in over 2.5 years ago).

Matt
Reply | Threaded
Open this post in threaded view
|

Re: PKCS#7/CMS verify reports bad signature

Steffen
Hello,

I have a small update in order to close this issue.

The identity provider that produced the invalid signatures have fixed their signatures so that we can verify them using the latest LTS version of OpenSSL. We use Bouncy Castle in some products and it does not catch the invalid signatures either, or at least not the way we use it.

Anyway, this is a satisfactory outcome.

Thank you for the help, everyone!

Regards,
Steffen