[openssl.org #3171] integer undefined behaviors

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

[openssl.org #3171] integer undefined behaviors

Rich Salz via RT
I built openssl-SNAP-20131112 on an x86-64 Linux machine using Clang's
undefined behavior sanitizer, ran "make check", and got the problems
below.  The shifts by 32 are potentially serious whereas the signed left
shift errors are thought to be benign at the moment.

John Regehr


c_enc.c:114:2: runtime error: shift exponent 32 is too large for 32-bit
type 'unsigned int'
c_enc.c:115:2: runtime error: shift exponent 32 is too large for 32-bit
type 'unsigned int'
c_enc.c:116:2: runtime error: shift exponent 32 is too large for 32-bit
type 'unsigned int'
c_enc.c:117:2: runtime error: shift exponent 32 is too large for 32-bit
type 'unsigned int'
c_enc.c:71:2: runtime error: shift exponent 32 is too large for 32-bit
type 'unsigned int'
c_enc.c:72:2: runtime error: shift exponent 32 is too large for 32-bit
type 'unsigned int'
c_enc.c:73:2: runtime error: shift exponent 32 is too large for 32-bit
type 'unsigned int'
c_enc.c:74:2: runtime error: shift exponent 32 is too large for 32-bit
type 'unsigned int'
c_enc.c:75:2: runtime error: shift exponent 32 is too large for 32-bit
type 'unsigned int'
c_enc.c:76:2: runtime error: shift exponent 32 is too large for 32-bit
type 'unsigned int'
c_enc.c:77:2: runtime error: shift exponent 32 is too large for 32-bit
type 'unsigned int'
c_enc.c:78:2: runtime error: shift exponent 32 is too large for 32-bit
type 'unsigned int'
c_enc.c:79:2: runtime error: shift exponent 32 is too large for 32-bit
type 'unsigned int'
c_enc.c:80:2: runtime error: shift exponent 32 is too large for 32-bit
type 'unsigned int'
c_enc.c:81:2: runtime error: shift exponent 32 is too large for 32-bit
type 'unsigned int'
c_enc.c:82:2: runtime error: shift exponent 32 is too large for 32-bit
type 'unsigned int'
c_enc.c:85:6: runtime error: shift exponent 32 is too large for 32-bit
type 'unsigned int'
c_enc.c:86:6: runtime error: shift exponent 32 is too large for 32-bit
type 'unsigned int'
c_enc.c:87:6: runtime error: shift exponent 32 is too large for 32-bit
type 'unsigned int'
c_enc.c:88:6: runtime error: shift exponent 32 is too large for 32-bit
type 'unsigned int'

a_int.c:397:4: runtime error: left shift of 63112885863764107 by 8
places cannot be represented in type 'long'
a_int.c:397:4: runtime error: left shift of 66133636318339381 by 8
places cannot be represented in type 'long'
a_int.c:397:4: runtime error: left shift of 70657489905646480 by 8
places cannot be represented in type 'long'
gost89.c:123:48: runtime error: left shift of 173 by 24 places cannot be
represented in type 'int'
gost89.c:143:42: runtime error: left shift of 130 by 24 places cannot be
represented in type 'int'
gost89.c:144:42: runtime error: left shift of 192 by 24 places cannot be
represented in type 'int'
gost89.c:176:42: runtime error: left shift of 150 by 24 places cannot be
represented in type 'int'
gost89.c:177:42: runtime error: left shift of 196 by 24 places cannot be
represented in type 'int'
gost89.c:278:48: runtime error: left shift of 139 by 24 places cannot be
represented in type 'int'
gost89.c:327:58: runtime error: left shift of 246 by 24 places cannot be
represented in type 'int'
gost89.c:326:58: runtime error: left shift of 227 by 24 places cannot be
represented in type 'int'
gost_crypt.c:244:49: runtime error: left shift of 203 by 24 places
cannot be represented in type 'int'
gost_crypt.c:250:49: runtime error: left shift of 166 by 24 places
cannot be represented in type 'int'
obj_dat.c:143:15: runtime error: left shift of 2 by 30 places cannot be
represented in type 'int'

______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
Development Mailing List                       [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: [openssl.org #3171] integer undefined behaviors

Yuriy Kaminskiy
John Regehr via RT wrote:

> I built openssl-SNAP-20131112 on an x86-64 Linux machine using Clang's
> undefined behavior sanitizer, ran "make check", and got the problems
> below.  The shifts by 32 are potentially serious whereas the signed left
> shift errors are thought to be benign at the moment.
>
> John Regehr
>
>
> c_enc.c:114:2: runtime error: shift exponent 32 is too large for 32-bit
> type 'unsigned int'

crypto/cast/cast_lcl.h

#define ROTL(a,n)     ((((a)<<(n))&0xffffffffL)|((a)>>(32-(n))))
...
#define E_CAST(n,key,L,R,OP1,OP2,OP3) \
...
        t=ROTL(t,(key[n*2+1])); \

If key[] is zero, ROTL become ((t<<0))|(t>>32)). Fortunately, it result will be
exactly same no matter if implementation `% 32` shift exponent (like x86{,_64}),
or not. (Besides, gcc replaces two shifts with single "rotll" instruction).

[Note: key[n*2+1] can be only between 0 and 31, see CAST_set_key]

On one hand, it is still "undefined behavior", and there should be added check
for key[] == 0. On other, it will only add slow down code without any positive
effect in practice :-| (gcc [at least, up to 4.6.*] fails to recognize that this
check won't change result and can be omitted; hmm... maybe, someone should fill
feature enhancement request at gcc bugtracker?).

> c_enc.c:115:2: runtime error: shift exponent 32 is too large for 32-bit
> type 'unsigned int'
> c_enc.c:116:2: runtime error: shift exponent 32 is too large for 32-bit
> type 'unsigned int'
> c_enc.c:117:2: runtime error: shift exponent 32 is too large for 32-bit
> type 'unsigned int'
> c_enc.c:71:2: runtime error: shift exponent 32 is too large for 32-bit
> type 'unsigned int'
> c_enc.c:72:2: runtime error: shift exponent 32 is too large for 32-bit
> type 'unsigned int'
> c_enc.c:73:2: runtime error: shift exponent 32 is too large for 32-bit
> type 'unsigned int'
> c_enc.c:74:2: runtime error: shift exponent 32 is too large for 32-bit
> type 'unsigned int'
> c_enc.c:75:2: runtime error: shift exponent 32 is too large for 32-bit
> type 'unsigned int'
> c_enc.c:76:2: runtime error: shift exponent 32 is too large for 32-bit
> type 'unsigned int'
> c_enc.c:77:2: runtime error: shift exponent 32 is too large for 32-bit
> type 'unsigned int'
> c_enc.c:78:2: runtime error: shift exponent 32 is too large for 32-bit
> type 'unsigned int'
> c_enc.c:79:2: runtime error: shift exponent 32 is too large for 32-bit
> type 'unsigned int'
> c_enc.c:80:2: runtime error: shift exponent 32 is too large for 32-bit
> type 'unsigned int'
> c_enc.c:81:2: runtime error: shift exponent 32 is too large for 32-bit
> type 'unsigned int'
> c_enc.c:82:2: runtime error: shift exponent 32 is too large for 32-bit
> type 'unsigned int'
> c_enc.c:85:6: runtime error: shift exponent 32 is too large for 32-bit
> type 'unsigned int'
> c_enc.c:86:6: runtime error: shift exponent 32 is too large for 32-bit
> type 'unsigned int'
> c_enc.c:87:6: runtime error: shift exponent 32 is too large for 32-bit
> type 'unsigned int'
> c_enc.c:88:6: runtime error: shift exponent 32 is too large for 32-bit
> type 'unsigned int'
>
> a_int.c:397:4: runtime error: left shift of 63112885863764107 by 8
> places cannot be represented in type 'long'
> a_int.c:397:4: runtime error: left shift of 66133636318339381 by 8
> places cannot be represented in type 'long'
> a_int.c:397:4: runtime error: left shift of 70657489905646480 by 8
> places cannot be represented in type 'long'
> gost89.c:123:48: runtime error: left shift of 173 by 24 places cannot be
> represented in type 'int'
> gost89.c:143:42: runtime error: left shift of 130 by 24 places cannot be
> represented in type 'int'
> gost89.c:144:42: runtime error: left shift of 192 by 24 places cannot be
> represented in type 'int'
> gost89.c:176:42: runtime error: left shift of 150 by 24 places cannot be
> represented in type 'int'
> gost89.c:177:42: runtime error: left shift of 196 by 24 places cannot be
> represented in type 'int'
> gost89.c:278:48: runtime error: left shift of 139 by 24 places cannot be
> represented in type 'int'
> gost89.c:327:58: runtime error: left shift of 246 by 24 places cannot be
> represented in type 'int'
> gost89.c:326:58: runtime error: left shift of 227 by 24 places cannot be
> represented in type 'int'
> gost_crypt.c:244:49: runtime error: left shift of 203 by 24 places
> cannot be represented in type 'int'
> gost_crypt.c:250:49: runtime error: left shift of 166 by 24 places
> cannot be represented in type 'int'
> obj_dat.c:143:15: runtime error: left shift of 2 by 30 places cannot be
> represented in type 'int'
______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
Development Mailing List                       [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: [openssl.org #3171] integer undefined behaviors

David Jacobson-3
In reply to this post by Rich Salz via RT
On 11/13/13 2:51 AM, John Regehr via RT wrote:

> I built openssl-SNAP-20131112 on an x86-64 Linux machine using Clang's
> undefined behavior sanitizer, ran "make check", and got the problems
> below.  The shifts by 32 are potentially serious whereas the signed left
> shift errors are thought to be benign at the moment.
>
> John Regehr
>
>
> c_enc.c:114:2: runtime error: shift exponent 32 is too large for 32-bit
> type 'unsigned int'
> c_enc.c:115:2: runtime error: shift exponent 32 is too large for 32-bit
> type 'unsigned int'
> c_enc.c:116:2: runtime error: shift exponent 32 is too large for 32-bit
> type 'unsigned int'
> c_enc.c:117:2: runtime error: shift exponent 32 is too large for 32-bit
> type 'unsigned int'
> c_enc.c:71:2: runtime error: shift exponent 32 is too large for 32-bit
> type 'unsigned int'
> c_enc.c:72:2: runtime error: shift exponent 32 is too large for 32-bit
> type 'unsigned int'
> c_enc.c:73:2: runtime error: shift exponent 32 is too large for 32-bit
> type 'unsigned int'
> c_enc.c:74:2: runtime error: shift exponent 32 is too large for 32-bit
> type 'unsigned int'
> c_enc.c:75:2: runtime error: shift exponent 32 is too large for 32-bit
> type 'unsigned int'
> c_enc.c:76:2: runtime error: shift exponent 32 is too large for 32-bit
> type 'unsigned int'
> c_enc.c:77:2: runtime error: shift exponent 32 is too large for 32-bit
> type 'unsigned int'
> c_enc.c:78:2: runtime error: shift exponent 32 is too large for 32-bit
> type 'unsigned int'
> c_enc.c:79:2: runtime error: shift exponent 32 is too large for 32-bit
> type 'unsigned int'
> c_enc.c:80:2: runtime error: shift exponent 32 is too large for 32-bit
> type 'unsigned int'
> c_enc.c:81:2: runtime error: shift exponent 32 is too large for 32-bit
> type 'unsigned int'
> c_enc.c:82:2: runtime error: shift exponent 32 is too large for 32-bit
> type 'unsigned int'
> c_enc.c:85:6: runtime error: shift exponent 32 is too large for 32-bit
> type 'unsigned int'
> c_enc.c:86:6: runtime error: shift exponent 32 is too large for 32-bit
> type 'unsigned int'
> c_enc.c:87:6: runtime error: shift exponent 32 is too large for 32-bit
> type 'unsigned int'
> c_enc.c:88:6: runtime error: shift exponent 32 is too large for 32-bit
> type 'unsigned int'
>
> a_int.c:397:4: runtime error: left shift of 63112885863764107 by 8
> places cannot be represented in type 'long'
> a_int.c:397:4: runtime error: left shift of 66133636318339381 by 8
> places cannot be represented in type 'long'
> a_int.c:397:4: runtime error: left shift of 70657489905646480 by 8
> places cannot be represented in type 'long'
> gost89.c:123:48: runtime error: left shift of 173 by 24 places cannot be
> represented in type 'int'
> gost89.c:143:42: runtime error: left shift of 130 by 24 places cannot be
> represented in type 'int'
> gost89.c:144:42: runtime error: left shift of 192 by 24 places cannot be
> represented in type 'int'
> gost89.c:176:42: runtime error: left shift of 150 by 24 places cannot be
> represented in type 'int'
> gost89.c:177:42: runtime error: left shift of 196 by 24 places cannot be
> represented in type 'int'
> gost89.c:278:48: runtime error: left shift of 139 by 24 places cannot be
> represented in type 'int'
> gost89.c:327:58: runtime error: left shift of 246 by 24 places cannot be
> represented in type 'int'
> gost89.c:326:58: runtime error: left shift of 227 by 24 places cannot be
> represented in type 'int'
> gost_crypt.c:244:49: runtime error: left shift of 203 by 24 places
> cannot be represented in type 'int'
> gost_crypt.c:250:49: runtime error: left shift of 166 by 24 places
> cannot be represented in type 'int'
> obj_dat.c:143:15: runtime error: left shift of 2 by 30 places cannot be
> represented in type 'int'
>
> ______________________________________________________________________
> OpenSSL Project                                 http://www.openssl.org
> Development Mailing List                       [hidden email]
> Automated List Manager                           [hidden email]
>
I know that this is for x86-64, but on SPARC shifting a 32 bit word by
32 bits does nothing.  It appears bits for the shift amount the hardware
only looks at the lower 5 bits.

     --David Jacobson
______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
Development Mailing List                       [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

RE: [openssl.org #3171] integer undefined behaviors

J. J. Farrell-2
In reply to this post by Rich Salz via RT
> From: John Regehr via RT [mailto:[hidden email]]
> Sent: Wednesday, November 13, 2013 10:52 AM
>
> I built openssl-SNAP-20131112 on an x86-64 Linux machine using Clang's
> undefined behavior sanitizer, ran "make check", and got the problems
> below.  The shifts by 32 are potentially serious whereas the signed
> left shift errors are thought to be benign at the moment.
>
> c_enc.c:114:2: runtime error: shift exponent 32 is too large for
> 32-bit type 'unsigned int'
> ...
> a_int.c:397:4: runtime error: left shift of 63112885863764107 by 8
> places cannot be represented in type 'long'
> ...

It's not safe to say that the signed shift errors are benign. C's behaviour is undefined in all these cases, the compiler could do anything (including doing what the coder hoped for with no other side effects, but that's one of an infinite number of possibilities).

Regards,
                   jjf
______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
Development Mailing List                       [hidden email]
Automated List Manager                           [hidden email]