PKCS#7 issues

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

PKCS#7 issues

Thomas J. Hruska
I just had the weirdest day ever.  At first I thought PayPal's EWP
system decided to flake out.  From staring at the problem all day, and
talking to PayPal support for a couple hours, and then having the
problem fix itself (without either PayPal or myself doing anything!)
makes it appear to me that PKCS#7 generation may have a timing bug
somewhere that causes verification problems when PayPal goes to either
verify the embedded public cert. or decrypt the data block itself.

It didn't matter what version of OpenSSL I tried on my end (see below),
all versions failed until about 4p.m. when every encrypted and signed
data block started working just fine.  However, I still have several of
the encrypted and signed data blocks that were NOT working and I tried
those again and they still fail.  I even tried using brand new certs.
It is definitely some sort of problem with OpenSSL and PKCS#7 - all
versions.

Windows versions of OpenSSL (default builds) that I tested with that
failed and then suddenly started working again around 4 p.m. today:
0.9.6m
0.9.7d
0.9.7e
0.9.7f
0.9.7g
0.9.8

All of those versions are working properly (for the moment), but that
doesn't mean they won't break again.  Also note that the data being
encrypted was identical throughout the entire process.  Also, I'm pretty
certain that the source code to interface with OpenSSL can't be at fault
because each and every function used is tested for returning error values.

Suggestions?  I could try hammering PayPal's service with an automated
script that generates an encrypted and signed block and then submits it
until I get one that is deemed bogus, but I don't think they would
appreciate that.


--
Thomas Hruska
Shining Light Productions

Home of BMP2AVI, Nuclear Vision, ProtoNova, and Win32 OpenSSL.
http://www.slproweb.com/

Ask me about discounts on any Shining Light Productions product!
______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
User Support Mailing List                    [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: PKCS#7 issues

Dr. Stephen Henson
On Wed, Jul 20, 2005, Thomas J. Hruska wrote:

>
> Suggestions?  I could try hammering PayPal's service with an automated
> script that generates an encrypted and signed block and then submits it
> until I get one that is deemed bogus, but I don't think they would
> appreciate that.
>

Well an indication of the error codes would help. Something just "not
working" doesn't help much...

Steve.
--
Dr Stephen N. Henson. Email, S/MIME and PGP keys: see homepage
OpenSSL project core developer and freelance consultant.
Funding needed! Details on homepage.
Homepage: http://www.drh-consultancy.demon.co.uk
______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
User Support Mailing List                    [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: PKCS#7 issues

Thomas J. Hruska
Dr. Stephen Henson wrote:

> On Wed, Jul 20, 2005, Thomas J. Hruska wrote:
>
>
>>Suggestions?  I could try hammering PayPal's service with an automated
>>script that generates an encrypted and signed block and then submits it
>>until I get one that is deemed bogus, but I don't think they would
>>appreciate that.
>>
>
>
> Well an indication of the error codes would help. Something just "not
> working" doesn't help much...
>
> Steve.
> --
> Dr Stephen N. Henson. Email, S/MIME and PGP keys: see homepage
> OpenSSL project core developer and freelance consultant.
> Funding needed! Details on homepage.
> Homepage: http://www.drh-consultancy.demon.co.uk

I don't have access to PayPal's internal systems and neither did the
people I spoke with in Technical Support.  Apparently only the
developers are allowed to know what error messages are generated - and
the developers don't get out much.  The encrypted and signed data block
on my end are generated perfectly every time without any errors from any
function (each function is checked for error conditions).

The only error condition given is error 5302.  This is an internal
PayPal error code and the only known "fix" that Tech. Support has is to
remove all whitespace from the PKCS#7 data block.  That didn't fix my
problem, which only served to confuse the support person.  It seemed
like a hack'ish sort of fix to me anyway.

Also, it can't be cookie related as Google searches might indicate
because I can still feed it the "bad" data block that I have saved
locally and get the error and then feed it a "good" data block and have
it work fine.  I also seriously doubt conventional memory has been
exhausted as other Google results might say.

I'm willing to send the data blocks I've got and the relevant section of
source code that generated them.  The source code is fairly complex, but
might shed some light on something that should or should not be done.


--
Thomas Hruska
Shining Light Productions

Home of BMP2AVI, Nuclear Vision, ProtoNova, and Win32 OpenSSL.
http://www.slproweb.com/

Ask me about discounts on any Shining Light Productions product!
______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
User Support Mailing List                    [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: PKCS#7 issues

Dr. Stephen Henson
On Wed, Jul 20, 2005, Thomas J. Hruska wrote:

> Dr. Stephen Henson wrote:
> >On Wed, Jul 20, 2005, Thomas J. Hruska wrote:
> >
> >
> >Well an indication of the error codes would help. Something just "not
> >working" doesn't help much...
> >
>
> I don't have access to PayPal's internal systems and neither did the
> people I spoke with in Technical Support.  Apparently only the
> developers are allowed to know what error messages are generated - and
> the developers don't get out much.  The encrypted and signed data block
> on my end are generated perfectly every time without any errors from any
> function (each function is checked for error conditions).
>
> The only error condition given is error 5302.  This is an internal
> PayPal error code and the only known "fix" that Tech. Support has is to
> remove all whitespace from the PKCS#7 data block.  That didn't fix my
> problem, which only served to confuse the support person.  It seemed
> like a hack'ish sort of fix to me anyway.
>

IIRC Paypal use OpenSSL though I don't know any more details about how they
are using it. If there's a signed data structure witing an enveloped data one
then whitespace isn't likely to be a problem.

It could be something related to the signing time. When you sign data it
includes a signing time. Its possible they are checking that against the
time or receipt and if there is a large skew they will reject it.

Steve.
--
Dr Stephen N. Henson. Email, S/MIME and PGP keys: see homepage
OpenSSL project core developer and freelance consultant.
Funding needed! Details on homepage.
Homepage: http://www.drh-consultancy.demon.co.uk
______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
User Support Mailing List                    [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: PKCS#7 issues

Thomas J. Hruska
Dr. Stephen Henson wrote:

> On Wed, Jul 20, 2005, Thomas J. Hruska wrote:
>
>
>>Dr. Stephen Henson wrote:
>>
>>>On Wed, Jul 20, 2005, Thomas J. Hruska wrote:
>>>
>>>
>>>Well an indication of the error codes would help. Something just "not
>>>working" doesn't help much...
>>>
>>
>>I don't have access to PayPal's internal systems and neither did the
>>people I spoke with in Technical Support.  Apparently only the
>>developers are allowed to know what error messages are generated - and
>>the developers don't get out much.  The encrypted and signed data block
>>on my end are generated perfectly every time without any errors from any
>>function (each function is checked for error conditions).
>>
>>The only error condition given is error 5302.  This is an internal
>>PayPal error code and the only known "fix" that Tech. Support has is to
>>remove all whitespace from the PKCS#7 data block.  That didn't fix my
>>problem, which only served to confuse the support person.  It seemed
>>like a hack'ish sort of fix to me anyway.
>>
>
>
> IIRC Paypal use OpenSSL though I don't know any more details about how they
> are using it. If there's a signed data structure witing an enveloped data one
> then whitespace isn't likely to be a problem.
>
> It could be something related to the signing time. When you sign data it
> includes a signing time. Its possible they are checking that against the
> time or receipt and if there is a large skew they will reject it.

As I said, their solution to the "problem" seemed hack'ish.  As to the
signing time, I just tried rolling my system clock back to the timeframe
where I was having problems and it still works fine.  Again, I seriously
doubted this would affect anything because PayPal EWP data is generated
one time on the client and used from that point on.  It would make
sense, in that case, to raise an error condition if the data generated
had a timestamp that pre-dated either certificate used to encrypt it.
Also, because of the way they handle it, it would make sense that all
enveloped data that I generate would not work for several hours, but
then would suddenly start working.  However, anything I generate now
works, anything I generated then _still_ doesn't work and will probably
never work.

BTW, I might have forgotten to mention, but I have signed data from 3
weeks ago that worked fine every time I tested those today during the
time that the "bad" data didn't work.  Those same bits of signed data
from 3 weeks ago still work fine, the "bad" data from earlier today
still doesn't work, and the data I'm generating now works just fine.

PayPal actually uses OpenSSL pretty much everywhere.  Only in the past
couple years have they actually give their users a reason to use it
themselves.  EWP (Encrypted Website Payments) is one thing.  The SOAP
interface to their web services APIs is another.


--
Thomas Hruska
Shining Light Productions

Home of BMP2AVI, Nuclear Vision, ProtoNova, and Win32 OpenSSL.
http://www.slproweb.com/

Ask me about discounts on any Shining Light Productions product!
______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
User Support Mailing List                    [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: PKCS#7 issues

Dr. Stephen Henson
On Wed, Jul 20, 2005, Thomas J. Hruska wrote:

>
> As I said, their solution to the "problem" seemed hack'ish.  As to the
> signing time, I just tried rolling my system clock back to the timeframe
> where I was having problems and it still works fine.  Again, I seriously
> doubted this would affect anything because PayPal EWP data is generated
> one time on the client and used from that point on.  It would make
> sense, in that case, to raise an error condition if the data generated
> had a timestamp that pre-dated either certificate used to encrypt it.
> Also, because of the way they handle it, it would make sense that all
> enveloped data that I generate would not work for several hours, but
> then would suddenly start working.  However, anything I generate now
> works, anything I generated then _still_ doesn't work and will probably
> never work.
>

Hmmm... in the past there have been a few base64 bugs in OpenSSL, though there
are no known cases in the current versions.

Most of the bugs only occur when data is a certain size and the line splits
occur at certain points. If that is the cause then adding or removing a few
redundant characters would fix it.

Steve.
--
Dr Stephen N. Henson. Email, S/MIME and PGP keys: see homepage
OpenSSL project core developer and freelance consultant.
Funding needed! Details on homepage.
Homepage: http://www.drh-consultancy.demon.co.uk
______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
User Support Mailing List                    [hidden email]
Automated List Manager                           [hidden email]