[openssl.org #2789] SHA-256 bug - Big endian arch

classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

[openssl.org #2789] SHA-256 bug - Big endian arch

Rich Salz via RT
 
Hi all,
 
I think that there is a bug in SHA-256 implementation.
 
I have tried to compile least Openssl of 0.9.8 branch  - 0.9.8u
but make test failed on SHA-256 test.
 
../util/shlib_wrap.sh ./shatest
test 1 ok
test 2 ok
test 3 ok
../util/shlib_wrap.sh ./sha1test
test 1 ok
test 2 ok
test 3 ok
../util/shlib_wrap.sh ./sha256t
Testing SHA-256
TEST 1 of 3 failed.
make[1]: *** [test_sha] Error 1
 
 
I also notice that previous version of OpenSSL also calculate wrong SHA-256 hash of data.
Compared to little endian machines where the hash is correct.
 
Platform is Renesas SH3E in BIG-endian mode
( used compiler sh3eb-linux-gcc)
 
 
OpenSSL self-test report:

OpenSSL version:  0.9.8u
Last change:      Fix MMA (Bleichenbacher's attack on PKCS #1 v1.5 RSA pa...
Options:          --prefix=/usr enable-zlib-dynamic -DB_ENDIAN no-camellia no-capieng no-cms no-gmp no-hw no-jpake no-krb5 no-mdc2 no-montasm no-rc5 no-rfc3779
no-seed no-shared
OS (uname):       Linux 10.1.5.102 2.6.15-29 #2 Tue Jan 10 15:23:09 CET 2012 sh3eb unknown
OS (config):      sh3eb-whatever-linux2
Target (default): linux-generic32
Target:           linux-generic32
Compiler:         Configured with: ../configure --prefix=/usr --mandir=/usr/man --target=sh3eb-linux --host=sh3eb-linux --build=i686-pc-linux-gnu --enable-langu
ages=c,c++,java --disable-checking --with-x=no --enable-threads=posix --enable-long-long
Thread model: posix
gcc version 3.0.4
 
 
Thanks for any advance.
 
Pavel Stanek

______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
Development Mailing List                       [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

RE: [openssl.org #2789] SHA-256 bug - Big endian arch

Green, Paul
Pavel Stanek wrote:
 
> I think that there is a bug in [OpenSSL 0.9.8 ] SHA-256 implementation [ when on a big-endian x86 ]
> Platform is Renesas SH3E in BIG-endian mode ( used compiler sh3eb-linux-gcc)
>
> OpenSSL version:  0.9.8u
> Options:          -DB_ENDIAN
> Target:           linux-generic32
> Compiler:         Configured with: --build=i686-pc-linux-gnu

[ above information starkly edited by me to emphasize big-endian on x86]

OpenSSL is just one of many open-source packages whose authors have assumed that all IA32 implementations are little endian. I discovered the same problem several years ago when we ported OpenSSL to the Stratus VOS operating system, which is also big-endian on IA32. This issue is so pervasive that when a numeric package fails on my system, I immediately start looking for C macros of the form "__i386".

The good news is that the OpenSSL team has been kind enough to accept patches from me that correct this situation, and I think you will find that later versions of OpenSSL will work just fine. I have had no trouble with the current versions of OpenSSL (i.e., 1.0.0 or 1.0.1).

See the patches that I submitted under RT #1130 to correct this issue, and which were applied by Richard Levitte on June 23, 2005 or so (the RT comment doesn't say to which branch it was applied, however).

PG
:��I"Ϯ��r�m���� (���Z+�7�zZ)���1���x ��h���W^��^��%����&jם.+-1�ځ��j:+v�������h�
Reply | Threaded
Open this post in threaded view
|

Re: [openssl.org #2789] SHA-256 bug - Big endian arch

Andy Polyakov-2
Paul,

> OpenSSL is just one of many open-source packages whose authors have
> assumed that all IA32 implementations are little endian.

No-no-no. IA is reference to processor architecture and all IA32
implementations *are* little-endian. You emulate big-endian execution
environment on little-endian processor, but it doesn't make processor
big-endian. This is unlike several families of CPUs that actually can
operate in different endiannesses. Some can switch under software
control, e.g. PowerPC, some are soldered to specific endianness, e.g.
MIPS, when endianness is determined when you place processor on PCB and
arrange specific pin to be connected to ground or not (or whatever
equivalent). I'd guess SH is of latter kind, i.e. you can build either
endian system with it. Then there also are processors that can access
data [and data only] in specific byte order, e.g. Itanium, ARMv7, but
it's different story.

And do you mean that, unlike "authors of open-source packages",
developers of commercial software are not prone to make such
assumptions? :-) :-) :-)

> I discovered
> the same problem several years ago when we ported OpenSSL to the
> Stratus VOS operating system, which is also big-endian on IA32.

What is it you're trying to assert exactly? There that is bug in plain C
SHA256 that has everything to do with endianness? But your own
experience contradicts it. Indeed, if there was bug, you would suffer
from it too. VOS-specific changes simply switch off inline assembler and
assumptions of little endianness, but don't change a line of
computationally significant code. Basically it's very same source after
pre-processing that is fed to your and originator's compiler.

> This
> issue is so pervasive that when a numeric package fails on my system,
> I immediately start looking for C macros of the form "__i386".

I believe that we've touched suitability of __i386 on your platform. I'd
argue that it shouldn't be even defined on your platform. You could have
avoided all the problems (at least with badly authored open-source
packages ;-) altogether by pre-defining say __i386be instead...

______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
Development Mailing List                       [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: [openssl.org #2789] SHA-256 bug - Big endian arch

Rich Salz via RT
In reply to this post by Rich Salz via RT
> I think that there is a bug in SHA-256 implementation.
>  
> I have tried to compile least Openssl of 0.9.8 branch  - 0.9.8u
> but make test failed on SHA-256 test.
>  
> ../util/shlib_wrap.sh ./shatest
> test 1 ok
> test 2 ok
> test 3 ok
> ../util/shlib_wrap.sh ./sha1test
> test 1 ok
> test 2 ok
> test 3 ok
> ../util/shlib_wrap.sh ./sha256t
> Testing SHA-256
> TEST 1 of 3 failed.
> make[1]: *** [test_sha] Error 1
>  
>  
> I also notice that previous version of OpenSSL also calculate wrong SHA-256 hash of data.
> Compared to little endian machines where the hash is correct.

What exactly do you mean by that? Code is maintained and was observed to
work correctly on either endian system. There are two possibilities:
system is not believed endianness or compiler is buggy. If first was
case, then even SHA1 test would fail too, so the failure sounds more
like compiler bug. Try to drop optimization level: edit Makefile in text
editor and replace -O3 with -O0, remove crypto/sha/sha256.o and 'make'.
This, dropping optimization level, is suggested in README.

> Platform is Renesas SH3E in BIG-endian mode
> ( used compiler sh3eb-linux-gcc)


______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
Development Mailing List                       [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

RE: [openssl.org #2789] SHA-256 bug - Big endian arch

Green, Paul
In reply to this post by Andy Polyakov-2
Andy,

Your points are entirely valid (as usual!). By using the term IA32, I was trying to avoid discussing any specific product. You are correct that our implementation has no choice but to live within the confines of the existing instruction set. Our big-endian environment is entirely a product of the software.

For what it is worth, I have yet to encounter a commercial product that has reported a problem with endianness. It is certainly open to possibility, as you note. In my experience porting open-source packages, the trouble comes when developers attempt to extract the last bit of performance by dropping into assembly language. Perhaps commercial products aren't that ambitious, or perhaps I just haven't dealt with enough of them.

I apologize for jumping to conclusions. It all sounded so familiar. I pretty much assumed the ancient version he was using was subjected to this issue. I should have dug deeper.

I will take up the idea of __i386be with our compiler folks. I have not run across that macro before.

For the record, Stratus supports many VOS/OpenVOS customers who use OpenSSL (and many other open-source packages) in their daily processing. OpenSSL works very well for them, and on their behalf (and mine), we're all grateful for the work that you and the other developers perform to maintain and enhance it.

PG

-----Original Message-----
From: [hidden email] [mailto:[hidden email]] On Behalf Of Andy Polyakov
Sent: Friday, April 13, 2012 5:32 AM
To: [hidden email]
Subject: Re: [openssl.org #2789] SHA-256 bug - Big endian arch

Paul,

> OpenSSL is just one of many open-source packages whose authors have
> assumed that all IA32 implementations are little endian.

No-no-no. IA is reference to processor architecture and all IA32 implementations *are* little-endian. You emulate big-endian execution environment on little-endian processor, but it doesn't make processor big-endian. This is unlike several families of CPUs that actually can operate in different endiannesses. Some can switch under software control, e.g. PowerPC, some are soldered to specific endianness, e.g.
MIPS, when endianness is determined when you place processor on PCB and arrange specific pin to be connected to ground or not (or whatever equivalent). I'd guess SH is of latter kind, i.e. you can build either endian system with it. Then there also are processors that can access data [and data only] in specific byte order, e.g. Itanium, ARMv7, but it's different story.

And do you mean that, unlike "authors of open-source packages", developers of commercial software are not prone to make such assumptions? :-) :-) :-)

> I discovered
> the same problem several years ago when we ported OpenSSL to the
> Stratus VOS operating system, which is also big-endian on IA32.

What is it you're trying to assert exactly? There that is bug in plain C
SHA256 that has everything to do with endianness? But your own experience contradicts it. Indeed, if there was bug, you would suffer from it too. VOS-specific changes simply switch off inline assembler and assumptions of little endianness, but don't change a line of computationally significant code. Basically it's very same source after pre-processing that is fed to your and originator's compiler.

> This
> issue is so pervasive that when a numeric package fails on my system,
> I immediately start looking for C macros of the form "__i386".

I believe that we've touched suitability of __i386 on your platform. I'd argue that it shouldn't be even defined on your platform. You could have avoided all the problems (at least with badly authored open-source packages ;-) altogether by pre-defining say __i386be instead...

______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
Development Mailing List                       [hidden email]
Automated List Manager                           [hidden email]
:��I"Ϯ��r�m���� (���Z+�7�zZ)���1���x ��h���W^��^��%����&jם.+-1�ځ��j:+v�������h�
Reply | Threaded
Open this post in threaded view
|

Re: [openssl.org #2789] SHA-256 bug - Big endian arch

Andy Polyakov-2
> I will take up the idea of __i386be with our compiler folks. I have
> not run across that macro before.

I didn't mean that you'd do anything about it. Omitting __i386 in favor
of replacement pre-defined macro was appropriate in the very beginning,
but now it's kind of too late, it's just reality to deal with. I mean
when I wrote "you *could*" I was referring to *missed* opportunity.
______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
Development Mailing List                       [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: [openssl.org #2789] SHA-256 bug - Big endian arch

Rich Salz via RT
In reply to this post by Rich Salz via RT

HI,
 
We have been using older version of Openssl (0.9.8c) and this one has same problem.
 
Used compiler is not perfect and often has prolem with O3 so i already tried Os (doesn't work) and O2 - thats work ok. So I suppose that optimalisations are not the issue.
 
I tested both SMALL FOOTPRINT and normal code and both has same problem.
 
I also have own SHA256 implementation ( implemented as defined in standard ) from another project. So when I replace the core of SHA256 computation
(sha256 hash update ) with my implementation it works ok.
Then all tests (make tests) passed without errors. So it's not exactly endian problem, but SHA256 implementation on SH architecture.
 
Our implementation is using rotation as defined in standart but Openssl uses opposite.
SH3 architecture in Big endian is not so common so there could be some unexpected bahaviour in compiler.
 
What we can say is that problem is not in padding but only in hash update.
 
SH3E - this is renesas SuperH architrecture with DSP also so there could be some instructions to modify only part of word etc.
I found suspected code in OpenSSL - union of long and unsigned char - to detect endiannes.
This could be one of problem, but when I tried to force edian into code it still does not work.
 
If I will have more time i'll try to check where could be problem (but on SMALL footprint code).
 
Best regards
Pavel
 
______________________________________________________________

> Od: "Andy Polyakov via RT"
> Komu:
> Datum: 13.04.2012 11:38
> P?edm?t: Re: [openssl.org #2789] SHA-256 bug - Big endian arch
>
> I think that there is a bug in SHA-256 implementation.
>  
> I have tried to compile least Openssl of 0.9.8 branch  - 0.9.8u
> but make test failed on SHA-256 test.
>  
> ../util/shlib_wrap.sh ./shatest
> test 1 ok
> test 2 ok
> test 3 ok
> ../util/shlib_wrap.sh ./sha1test
> test 1 ok
> test 2 ok
> test 3 ok
> ../util/shlib_wrap.sh ./sha256t
> Testing SHA-256
> TEST 1 of 3 failed.
> make[1]: *** [test_sha] Error 1
>  
>  
> I also notice that previous version of OpenSSL also calculate wrong SHA-256 hash of data.
> Compared to little endian machines where the hash is correct.

What exactly do you mean by that? Code is maintained and was observed to
work correctly on either endian system. There are two possibilities:
system is not believed endianness or compiler is buggy. If first was
case, then even SHA1 test would fail too, so the failure sounds more
like compiler bug. Try to drop optimization level: edit Makefile in text
editor and replace -O3 with -O0, remove crypto/sha/sha256.o and 'make'.
This, dropping optimization level, is suggested in README.

> Platform is Renesas SH3E in BIG-endian mode
> ( used compiler sh3eb-linux-gcc)


______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
Development Mailing List                       [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: [openssl.org #2789] SHA-256 bug - Big endian arch

Rich Salz via RT
> We have been using older version of Openssl (0.9.8c) and this one has
> same problem.

Even though 0.9.8c is more sensitive to endianness (sha256_block
function accepts additional 'host' flag, which caller can get wrong), it
should have worked as long as -DB_ENDIAN is passed. Later versions work
even without [LB]_ENDIAN. But in either case endianness was always taken
into account and code was actually tested on both big- and little-endian
platforms.

> Used compiler is not perfect and often has prolem with O3 so i
> already tried Os (doesn't work) and O2 - thats work ok. So I suppose
> that optimalisations are not the issue.

You don't know that till you pass -O0. Please do test.

> I tested both SMALL FOOTPRINT and normal code and both has same
> problem.
>
> I also have own SHA256 implementation ( implemented as defined in
> standard ) from another project. So when I replace the core of SHA256
> computation (sha256 hash update ) with my implementation it works ok.
>  Then all tests (make tests) passed without errors. So it's not
> exactly endian problem, but SHA256 implementation on SH architecture.

Does it mean that we agree that it looks more like compiler problem than
problem with OpenSSL source code?

> Our implementation is using rotation as defined in standart but
> Openssl uses opposite.

I find it hard to believe that rotation direction is responsible.

> SH3 architecture in Big endian is not so
> common so there could be some unexpected bahaviour in compiler.

SH processors are capable of operating in either endianness. So that
formally you can't say "SH *architecture* is big-endian", you can only
say "this particular SH-based system is big-endian." As far as I
understand endianness is determined by logical level fed to specific CPU
pin at power-up.


______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
Development Mailing List                       [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: [openssl.org #2789] SHA-256 bug - Big endian arch

Rich Salz via RT

HI,
 
You are right, the optimalisations are the issue.
When I compile it with O0 then it's OK.
 
I also found that problem occurs in macros Siga0,1 Maj, ...
Wth O0 there are correct valus but with O2 there are incorrect values.
 
Compiler must have some problem there.
( I also test to modify variables declaration to volatile and that helps too. )
 
So definetly we could close this issue with result "broken compiler".
( which really makes me unhappy)
 
Many thanks for your time.
 
Best regars
Pavel
 
 
______________________________________________________________
> Od: "Andy Polyakov via RT"
> Komu:
> Datum: 14.04.2012 10:41
> P?edm?t: Re: [openssl.org #2789] SHA-256 bug - Big endian arch
>
> We have been using older version of Openssl (0.9.8c) and this one has
> same problem.

Even though 0.9.8c is more sensitive to endianness (sha256_block
function accepts additional 'host' flag, which caller can get wrong), it
should have worked as long as -DB_ENDIAN is passed. Later versions work
even without [LB]_ENDIAN. But in either case endianness was always taken
into account and code was actually tested on both big- and little-endian
platforms.

> Used compiler is not perfect and often has prolem with O3 so i
> already tried Os (doesn't work) and O2 - thats work ok. So I suppose
> that optimalisations are not the issue.

You don't know that till you pass -O0. Please do test.

> I tested both SMALL FOOTPRINT and normal code and both has same
> problem.
>
> I also have own SHA256 implementation ( implemented as defined in
> standard ) from another project. So when I replace the core of SHA256
> computation (sha256 hash update ) with my implementation it works ok.
>  Then all tests (make tests) passed without errors. So it's not
> exactly endian problem, but SHA256 implementation on SH architecture.

Does it mean that we agree that it looks more like compiler problem than
problem with OpenSSL source code?

> Our implementation is using rotation as defined in standart but
> Openssl uses opposite.

I find it hard to believe that rotation direction is responsible.

> SH3 architecture in Big endian is not so
> common so there could be some unexpected bahaviour in compiler.

SH processors are capable of operating in either endianness. So that
formally you can't say "SH *architecture* is big-endian", you can only
say "this particular SH-based system is big-endian." As far as I
understand endianness is determined by logical level fed to specific CPU
pin at power-up.


______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
Development Mailing List                       [hidden email]
Automated List Manager                           [hidden email]