SMIME signed message verification

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

SMIME signed message verification

Harald Koch
Hello,

my task is to sign a message in C for SMIME exchange, which works as expected and openSSL is self-fulfilling with itself in successful verification (and unsuccessful in produced errors as expected). I've tested PKCS7 SMIME functions, as well as CMS ones, leading to the same result: the reference software endpoints (both written in Java; at least one uses BuncyCastle) are unable to verify the signature. See below the BASE64 blocks of a successful reference signature, and an unsuccessful openSSL variant of the same message, both signed with the same certificate and private key. The error message extracted from the Java implementations are:
- "Unable to verify content integrity: Missing data"
- "The system is unable to find out the sign algorithm of the inbound message"

I digged a bit deeper into the ASN1 data („cat signature.base64 | base64 -d | openssl asn1parse -inform DER" ), leading to my assumption that the algorithm provided for signature contained differs:
- openSSL indicates „rsaEncryption"
- Java indicates „sha512WithRSAEncryption"

Since message digest and signing timestamp are available in both signatures, this is my only point of difference. Is there something I can do in openSSL to change the message algorithm to be provided in a form which Java accepts?

Here the two base64 encoded signatures. Reference message (which works in all directions):

MIAGCSqGSIb3DQEHAqCAMIACAQExDzANBglghkgBZQMEAgMFADCABgkqhkiG9w0BBwEAAKCAMIID
ljCCAn6gAwIBAgICBF0wDQYJKoZIhvcNAQELBQAwgY0xEjAQBgNVBAMTCWRlZGljYXRlZDEXMBUG
A1UEChMOVElIIEl0YWxpYSBTUkwxDDAKBgNVBAsTA2VkaTEPMA0GA1UEBxMGUGFkb3ZhMQ4wDAYD
VQQIEwVJdGFseTELMAkGA1UEBhMCUEQxIjAgBgkqhkiG9w0BCQEWE2FzMkBkZWRpY2F0ZWQud29y
bGQwHhcNMTcxMTAyMTMyNzE3WhcNMjIxMTAxMTMyNzE3WjCBjTESMBAGA1UEAxMJZGVkaWNhdGVk
MRcwFQYDVQQKEw5USUggSXRhbGlhIFNSTDEMMAoGA1UECxMDZWRpMQ8wDQYDVQQHEwZQYWRvdmEx
DjAMBgNVBAgTBUl0YWx5MQswCQYDVQQGEwJQRDEiMCAGCSqGSIb3DQEJARYTYXMyQGRlZGljYXRl
ZC53b3JsZDCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBALZv/pezb4MuezuMVb40qyCD
z20xtPzWtjchXqeJnPo6b2VPHloSr2zGQ8mG9fsyfFj/WLz6d06ErB09Fr6M53Y7UjmMmik1BThL
VXFKEZThlWTtfTwlhinoIjbcZ+sr4BKsVPvUShIEE5p4E3xHEcbx/8cMovt5s1gWiIakiwB+I8XV
gg0+fr2IBxf9FvzNPyZZGN65sqdhNDHEyfkl65SFZoL8iDhmIgmeCB1ebW5syccFmAS984iBx0jk
RWGACZaWge5X0kP50wk1WnEaHcz/Uay+3pC/j4ZUZGVL2jw44fd1BUXelaj+XcqG341r6xN0fJlR
Xc0RhwmCcKINgFECAwEAATANBgkqhkiG9w0BAQsFAAOCAQEAJGJhUUcu/LTzRb+D7xCqwVi9h4EN
WTxv81uFBOtebUjdQkNSsaXD5eHOr//tvqgNti3uGVdIfJarGSbaM32RrTSzbQvo+6r9MUM+qkx5
o/KTB2GuG+yyKEYbgrRsTluxu0KcsX44t+HyZ2VJnt9u44E2e34So3ioC02B4CoTzOFtT+DsWznd
EWfB5r9GsY3IEfAOHtTgMFDPjb8pDfl/fTk6Pv3c+drSK/Os8qTa1gbe3GKOwIhXh01AsG2kaIss
g7tBXtHYlnZgBqoRuIJus+kmUkJ4s9VDr+HCy4sVg1Q/h0Y6voUdTkBECRWR5zK5eb6VHSZRD2i9
uJSoqQmjNQAAMYICgjCCAn4CAQEwgZQwgY0xEjAQBgNVBAMTCWRlZGljYXRlZDEXMBUGA1UEChMO
VElIIEl0YWxpYSBTUkwxDDAKBgNVBAsTA2VkaTEPMA0GA1UEBxMGUGFkb3ZhMQ4wDAYDVQQIEwVJ
dGFseTELMAkGA1UEBhMCUEQxIjAgBgkqhkiG9w0BCQEWE2FzMkBkZWRpY2F0ZWQud29ybGQCAgRd
MA0GCWCGSAFlAwQCAwUAoIG/MBgGCSqGSIb3DQEJAzELBgkqhkiG9w0BBwEwHAYJKoZIhvcNAQkF
MQ8XDTIwMDEyNDE5NDQzNlowNAYJKoZIhvcNAQkPMScwJTAKBggqhkiG9w0DBzAOBggqhkiG9w0D
AgICAIAwBwYFKw4DAgcwTwYJKoZIhvcNAQkEMUIEQEuNaTyufm0EkVoVX1YXteYJnQk+ViLhR9YL
lz5P8+s8PVPtY1bOazDhEC4nL2U4Cj99zlaq+FQrKRu58uOjZ9QwDQYJKoZIhvcNAQENBQAEggEA
RngLYRDUXYCYRb1Xlht8Ywl7MWawKvJqYMgH+YFJA2Wpsgi2soSrjA35TmvqOQtm1cJBJrqx8OU7
NdnE8y5rrJPNDp4QMnwzWAzWWyf6zZWGFQGGEMn729zAPjl170Ka+WqLjRbVIC9Yvq4fYktfZKpg
WosLs1zvLzJYAivWeFxmbuAmaUii/FkhFlZNEp/EM3FVbAG5GlqG/2vciE6uwcuW+Jk4/D+0deh5
N8ib3Dar9vBOlqfcqwUyvIY7RJVCrIa7xuwfJY/nrG/EK2p3uzHX542AKs0sBYk9Bm5DS0cXvONe
UiQl/4Cj3pOivvmGIdhy5o7N5pwCCJh6cZv14QAAAAAAAA==


openSSL’s version, which is not accepted by Java applications:
MIILuAYJKoZIhvcNAQcCoIILqTCCC6UCAQExDTALBglghkgBZQMEAgEwCwYJKoZI
hvcNAQcBoIII0zCCCM8wggS3oAMCAQICClLgJv3GpnHBcM0wDQYJKoZIhvcNAQEL
BQAwgYsxCzAJBgNVBAYTAkRFMRYwFAYDVQQHDA1Ib2x6Z2VybGluZ2VuMRUwEwYD
VQQKDAxjLXdvcmtzIEdtYkgxFTATBgNVBAsMDENlcnRpZmljYXRlczERMA8GA1UE
AwwIT0ZUUDIgQ0ExIzAhBgoJkiaJk/IsZAEZFhNvZnRwMmNhLmMtd29ya3MubmV0
MB4XDTE4MDQxMDEzMTQwMloXDTIwMDMxMDEzMTQwMlowcDELMAkGA1UEBhMCSVQx
DzANBgNVBAcMBlBhZG92YTEdMBsGA1UECgwUZEVESWNhdGVkIEl0YWxpYSBzcmwx
ETAPBgNVBAsMCEVESSBUZWFtMR4wHAYDVQQDDBVvZnRwMi5kZWRpY2F0ZWQud29y
bGQwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQDBHryKO7KqVZdIPrnB
ArhEjBqeNfBUFeM5n1lmSU+x5pYfAp5MYGMxPnFpH7Q4Vf+XK/6yTn3VI+hT/bcW
NiealiyRBjRQEX/was7bxDzY9y/Ms+tWGqhq0H6qN5q8gJ/15H6/B3jSV5h/gOAX
hdXUuh3NuC2aXubWVlF+IEWtFj9xTVf6V+6MHoEpIw+C9LCbVvTipasjxB8O8dFk
N3bvFhFfHUkXA2OlPxZZxBMFa26DARibS7YO5vjoCpofwBtsonxkUEps4vn4QFu3
0klEqRohhZnNzEdj9LqW6qdKPf3ZyVIgaOQa+WMaVDve2yZ9mh/T9lypM6NEBUDV
WT11AgMBAAGjggJNMIICSTAJBgNVHRMEAjAAMBEGCWCGSAGG+EIBAQQEAwIGwDAO
BgNVHQ8BAf8EBAMCBaAwHQYDVR0lBBYwFAYIKwYBBQUHAwEGCCsGAQUFBwMCMDAG
CWCGSAGG+EIBDQQjFiFPRlRQMiBDZXJ0aWZpY2F0ZSBvZiBjLXdvcmtzIEdtYkgw
UQYDVR0RBEowSIYWb2Z0cDovL08wMTc3WDBZTTAwMDAwMYIVb2Z0cDIuZGVkaWNh
dGVkLndvcmxkgRdzdXBwb3J0QGRlZGljYXRlZC53b3JsZDAdBgNVHQ4EFgQUtcZm
yFMAlxg78z9qTJsX7grfEV4wHwYDVR0jBBgwFoAU2/OITatwz/A2aA5O8GYErWWT
oNwwUAYIKwYBBQUHAQEERDBCMEAGCCsGAQUFBzAChjRodHRwOi8vb2Z0cDJjYS5j
LXdvcmtzLm5ldC9wa2kvcHViL2NhY2VydC9jYWNlcnQuY2VyMB4GA1UdEgQXMBWC
E29mdHAyY2EuYy13b3Jrcy5uZXQwPwYJYIZIAYb4QgEEBDIWMGh0dHA6Ly9vZnRw
MmNhLmMtd29ya3MubmV0L3BraS9wdWIvY3JsL2NhY3JsLmNybDA/BglghkgBhvhC
AQMEMhYwaHR0cDovL29mdHAyY2EuYy13b3Jrcy5uZXQvcGtpL3B1Yi9jcmwvY2Fj
cmwuY3JsMEEGA1UdHwQ6MDgwNqA0oDKGMGh0dHA6Ly9vZnRwMmNhLmMtd29ya3Mu
bmV0L3BraS9wdWIvY3JsL2NhY3JsLmNybDANBgkqhkiG9w0BAQsFAAOCBAEAE3KU
a89sGS4Cz7EFqmC2cjawfeaKM0ryihuLqaiMlm1dhuqiUaBTDDXZyoK/Z908uHGA
YQGkHsyDL50Rn7FOemf4vgkw+FJ5Jz8AVUNQ0Ai8lFNOirvbXbOjI9BjIIdWrfu+
cTrqw746ZUl+gsfeLtuneWhJLbAbvOJxQIczzRkHW/VkBSzRzgbLDoaQFEKNJyfg
Gj7uZRDZBFjaVMuObyNQgDLyv167+zjwRXsjzGGk2q830nM2qxH3jsWXjBGr5h0H
mtrZu4/x+ek7wXqr+i2azE9iN7cV55ni5tbcSXmQOT3bHd+8gQ+lG6ZB5a4u94pS
LU4LsbNFjprQmMQJ+HvgjiWLS3cAXo6TA6Kc/ojrmRVinkPzD7fEXAg/MxROAnRI
VGCYiXipMPputi9h1FdOnIakwOHxgW7mAVFh1kdhsJ4Sjv42sbbUwgPkkxww9dG0
Uk78dCALUfZCSjTFVYXpOh0OxO8SgY9CCHVPjcPQV0e7k/N3Je3erTCNw20pLXtF
wINtpxp0SdCRl3NsYG+6YJ1WTK4TF4DOw9Hi8rKjnU6Fz/aYqVPD2I4IfIXter5B
f26+2utLMr0VG+QV9cXkoRaYEgXZOiuEY+AmHn+tSL9/ICcRG1ya/4xRNI5USP5K
mzeh5AZXIFN4R6Q8cctGn4Bl1LZqgPoIja029N8PEiqefzSFv/fcf6GMkl9GKQRy
WaPCLsjRYMHUsn5IjYnb2FS34LtrVIzHAZ/W+kiG5gX/hz13B1yFMnFj4p65XFWN
Bfp8UKuPgUshFMKA8CaOTDmHLsIzZDC3NMbcVJAxhFWGz8FhXoKh47x+VzR+E5oL
cxnV1z3P1pTGyRn1ro+1sBqmiwPZsuyA80z3W1sfUs1XPMSeB1rDUsX0EFivGKnT
++s9nykHA/vysY1kerh5/UJ7O/72uGFCj6BB1DsXniexC1KW5Z41HmlvIlwvDA6g
DopNII6Kej2kDw2VWVoCLYNFo7xV/qoRcTonS5bvGdYC/Id4PZ6XA89WT5PyvXyK
Ptl1pOLd0GNLreyGGeCMhEYM6vQAXGB6yC/ugR2bhUc+MSUbkey4WXFDrXiPHqGo
QWIbgiRf51Y5+fHEoUFGNWGcxAg7UMUZv7OctGEH2tx4A+7SC2ZGO6rnuGUwTe0z
ack4+ypFSih6O59Y8wgeesbTCAFFZsSyzH/aggPTuydz8ARiZgN2td33iSDDw3Ec
5mhpBJf08l/H4abGKV1t3VNdQEkOFSn+N7/ksTsSXmcC24ONBDN+Ra0MwdCidO7B
/yc5jz75mwHf5AKd34ToO2RivgQxCHoCnTEBUE/kt4+8cNtv0crLTS04C97ByAod
29b75rnqhnmcO6kZlzGCAqswggKnAgEBMIGaMIGLMQswCQYDVQQGEwJERTEWMBQG
A1UEBwwNSG9semdlcmxpbmdlbjEVMBMGA1UECgwMYy13b3JrcyBHbWJIMRUwEwYD
VQQLDAxDZXJ0aWZpY2F0ZXMxETAPBgNVBAMMCE9GVFAyIENBMSMwIQYKCZImiZPy
LGQBGRYTb2Z0cDJjYS5jLXdvcmtzLm5ldAIKUuAm/camccFwzTALBglghkgBZQME
AgGggeQwGAYJKoZIhvcNAQkDMQsGCSqGSIb3DQEHATAcBgkqhkiG9w0BCQUxDxcN
MjAwMTI0MTkzODUzWjAvBgkqhkiG9w0BCQQxIgQgQBCQAoZud7bmLBC3cTB621kM
lRfJ4pkbKN52sbAELHoweQYJKoZIhvcNAQkPMWwwajALBglghkgBZQMEASowCwYJ
YIZIAWUDBAEWMAsGCWCGSAFlAwQBAjAKBggqhkiG9w0DBzAOBggqhkiG9w0DAgIC
AIAwDQYIKoZIhvcNAwICAUAwBwYFKw4DAgcwDQYIKoZIhvcNAwICASgwDQYJKoZI
hvcNAQEBBQAEggEAsz2Lq/bBiX20RCodqm1k+DmNRMsNNvb2maZnL8//lcJkz73t
A38rICeH53CYDXjk/5SD6T7ogfMgxBu7tdNpFaWXMgXALBrmYiE6X0vk9Mvc6bvW
4Hrqc8FY0WLgUOExMS9LJ4zqbCNxGZGJmhPCkuFGAK0cd+ycijSNCY3WbwSmNTIP
Qvhhh5b+y252efMDDWcxl831JWmDBBTjsNAzk+mISDTm+asAJQjtUNbZMWIWgoQ/
rs7BhD+UKLbzlS7OyCh2yJAT472CXlJmA6Wv2ULhDCTqAC21ZXhs8JNIKNrLWoqO
1NGDfyQiKEk+91070jO28hXTvwol2wDyvvH4dg==

Any input is welcome.

Regards,
Harald
Reply | Threaded
Open this post in threaded view
|

Re: SMIME signed message verification

Michael Richardson

Harald Koch <[hidden email]> wrote:
    > my task is to sign a message in C for SMIME exchange, which works as
    > expected and openSSL is self-fulfilling with itself in successful
    > verification (and unsuccessful in produced errors as expected). I've
    > tested PKCS7 SMIME functions, as well as CMS ones, leading to the same
    > result: the reference software endpoints (both written in Java; at
    > least one uses BuncyCastle) are unable to verify the signature. See
    > below the BASE64 blocks of a successful reference signature, and an
    > unsuccessful openSSL variant of the same message, both signed with the
    > same certificate and private key. The error message extracted from the
    > Java implementations are:

I have exchanged CMS signed artifacts with Java implementations.
I have CC'ed the author of the Java code to understand if they use
BouncyCastle or are using an OpenSSL wrapper in Java code.

    > - "Unable to verify content integrity: Missing data"
    > - "The system is unable to find out the sign algorithm of the inbound message"

    > I digged a bit deeper into the ASN1 data („cat signature.base64 | base64 -d | openssl asn1parse -inform DER" ), leading to my assumption that the algorithm provided for signature contained differs:
    > - openSSL indicates „rsaEncryption"
    > - Java indicates „sha512WithRSAEncryption"

The first error you got seems inconsistent with this problem.
Is is possible that one of you are sending CMS structures with out-of-band content?

--
]               Never tell me the odds!                 | ipv6 mesh networks [
]   Michael Richardson, Sandelman Software Works        |    IoT architect   [
]     [hidden email]  http://www.sandelman.ca/        |   ruby on rails    [


signature.asc (497 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: SMIME signed message verification

Harald Koch
Dear Michael,

> Am 29.10.2020 um 14:12 schrieb Michael Richardson <[hidden email]>
>> - "Unable to verify content integrity: Missing data"
>> - "The system is unable to find out the sign algorithm of the inbound message"
>
>> I digged a bit deeper into the ASN1 data („cat signature.base64 | base64 -d | openssl asn1parse -inform DER" ), leading to my assumption that the algorithm provided for signature contained differs:
>> - openSSL indicates „rsaEncryption"
>> - Java indicates „sha512WithRSAEncryption"
>
> The first error you got seems inconsistent with this problem.
> Is is possible that one of you are sending CMS structures with out-of-band content?
Yes, the signed message is contained in a HTTP(S) multipart request with more payload and header information, sure. The only different part is the signed content, all other content has been manually checked, they are exactly the same. May it be possible that the CMS data which openSSL generates is much bigger due to unneeded certificate information, which makes the Java process stumble over the input?

Regards,
Harald
Reply | Threaded
Open this post in threaded view
|

Re: SMIME signed message verification

Michael Richardson

Harald Koch <[hidden email]> wrote:
    >> Am 29.10.2020 um 14:12 schrieb Michael Richardson <[hidden email]>
    >>> - "Unable to verify content integrity: Missing data"
    >>> - "The system is unable to find out the sign algorithm of the inbound message"
    >>
    >>> I digged a bit deeper into the ASN1 data („cat signature.base64 | base64 -d | openssl asn1parse -inform DER" ), leading to my assumption that the algorithm provided for signature contained differs:
    >>> - openSSL indicates „rsaEncryption"
    >>> - Java indicates „sha512WithRSAEncryption"
    >>
    >> The first error you got seems inconsistent with this problem.
    >> Is is possible that one of you are sending CMS structures with
    >> out-of-band content?

    > Yes, the signed message is contained in a HTTP(S) multipart request
    > with more payload and header information, sure. The only different part
    > is the signed content, all other content has been manually checked,
    > they are exactly the same. May it be possible that the CMS data which
    > openSSL generates is much bigger due to unneeded certificate
    > information, which makes the Java process stumble over the input?

so, do have detached content then?

And MIME and HTTP is involved?  My bet is that you have CRLF/LF issues, which
you might not see unless you look at the raw packets --- after the TLS is
removed, which is a hassle, but there is a way in openssl to get that data
put somewhere, but I can't recall what it is.

--
]               Never tell me the odds!                 | ipv6 mesh networks [
]   Michael Richardson, Sandelman Software Works        |    IoT architect   [
]     [hidden email]  http://www.sandelman.ca/        |   ruby on rails    [



signature.asc (497 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: SMIME signed message verification

Harald Koch
Dear Michael,

> Am 29.10.2020 um 22:55 schrieb Michael Richardson <[hidden email]>:
>
>> Yes, the signed message is contained in a HTTP(S) multipart request
>> with more payload and header information, sure. The only different part
>> is the signed content, all other content has been manually checked,
>> they are exactly the same. May it be possible that the CMS data which
>> openSSL generates is much bigger due to unneeded certificate
>> information, which makes the Java process stumble over the input?
> so, do have detached content then?
Yes.

> And MIME and HTTP is involved?  My bet is that you have CRLF/LF issues, which
> you might not see unless you look at the raw packets --- after the TLS is
> removed, which is a hassle, but there is a way in openssl to get that data
> put somewhere, but I can't recall what it is.

The CRLF issue known to me, and the used „vi“ editor is showing them good in blue color with „^M“ at every line ending, so there is no error here (since I use the flag PKCS7_CRLFEOL / CMS_CRLFEOL for correct encoding of line endings for HTTP(S) transmission).
I’ve obtained the raw message via netcat and examined it in detail, the only difference seems to be the signed content. One big difference in the contained ASN1 data is that openSSL includes much more information from the used certificate, i.e. the X509v3 extensions, which Java doesn’t do. Perhaps it’s a matter of what certificate information is included in the signed data?