[openssl.org #1090] [BUG] Segfault in dgst signing with rsa private key

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

[openssl.org #1090] [BUG] Segfault in dgst signing with rsa private key

Rich Salz via RT

Did you check that you actually get the right library?  Do it like this:

LD_LIBRARY_PATH=. ldd ./apps/openssl

If you don't get the library paths you expected, you need to use
LD_PRELOAD.

[[hidden email] - Wed Jun  1 17:20:13 2005]:

> Hello!
>
> I use 0.9.8-stable-SNAP-20050601 snapshot. Problem occurs on digest
> signing.
>
> make report:
> ========
> OpenSSL self-test report:
>
> OpenSSL version:  0.9.8-beta4-dev
> Last change:      Correct naming of the 'chil' and '4758cca' ENGINEs.
>    Thi...
> Options:          -g enable-shared enable-zlib no-gmp no-krb5 no-mdc2
>    no-rc5 no-zlib-dynamic
> OS (uname):       Linux manul 2.4.26-1-386 #1 Tue Aug 24 13:31:19 JST
>    2004 i686 GNU/Linux
> OS (config):      i686-whatever-linux2
> Target (default): linux-elf
> Target:           linux-elf
> Compiler:         Configured with: ../src/configure -v
>    --enable-languages=c,c++,java,f77,pascal,objc,ada,treelang
>    --prefix=/usr --mandir=/usr/share/man --infodir=/usr/share/info
>    --with-gxx-include-dir=/usr/include/c++/3.3 --enable-shared
>    --enable-__cxa_atexit --with-system-zlib --enable-nls
>    --without-included-gettext --enable-clocale=gnu --enable-debug
>    --enable-java-gc=boehm --enable-java-awt=xlib --enable-objc-gc
>    i486-linux
> Thread model: posix
> gcc version 3.3.5 (Debian 1:3.3.5-12)
>
> Test skipped.
> =======
> Native tests are passed. So I do:
>
> LD_LIBRARY_PATH=. ./apps/openssl req -newkey rsa:512 -nodes -batch
>    -keyout keyrsa.pem -out reqrsa.pem -config apps/openssl.cnf
> LD_LIBRARY_PATH=. ./apps/openssl x509 -req -set_serial 1 -signkey
>    keyrsa.pem -in reqrsa.pem -out certrsa.pem
> LD_LIBRARY_PATH=. ./apps/openssl dgst -sha1 -sign keyrsa.pem -out
>    dsignrsa.bin CHANGES
>
> It causes a segfault with backtrace
> ======
> #0  0x400bdca8 in BN_BLINDING_set_thread_id (b=0x803, n=1076728596)
>      at bn_blind.c:267
> 267             b->thread_id = n;
> (gdb) bt
> #0  0x400bdca8 in BN_BLINDING_set_thread_id (b=0x803, n=1076728596)
>      at bn_blind.c:267
> #1  0x400d7f85 in RSA_setup_blinding (rsa=0x80b4068, in_ctx=0x80b3e18)
>      at rsa_lib.c:405
> #2  0x400d6fc9 in rsa_get_blinding (rsa=0x80b4068, r=0xbffff198,
>      local=0xbffff19c, ctx=0x80b3e18) at rsa_eay.c:251
> #3  0x400d5e3e in RSA_eay_private_encrypt (flen=35,
>      from=0x80b3c30 "0!0\t\006\005+\016\003\002\032\005",
>      to=0x80b1a10 "S. Engelschall]\n\n  *) Fix the various library and
>    apps files to free up pkeys obtained from\n     X509_PUBKEY_get()
>    et al. Also allow x509.c to handle netscape extensions.\n
>    [Steve Henson]\n\n  *) Fix"...,
>      rsa=0x80b4068, padding=1) at rsa_eay.c:361
> #4  0x400d7c50 in RSA_private_encrypt (flen=2051,
>      from=0x803 <Address 0x803 out of bounds>,
>      to=0x803 <Address 0x803 out of bounds>, rsa=0x402d9314,
>    padding=2051)
>      at rsa_lib.c:288
> #5  0x400d83f7 in RSA_sign (type=64, m=0x1 <Address 0x1 out of
>    bounds>,
>      m_len=64,
>      sigret=0x80b1a10 "S. Engelschall]\n\n  *) Fix the various library
>    and apps files to free up pkeys obtained from\n
>    X509_PUBKEY_get() et al. Also allow x509.c to handle netscape
>    extensions.\n     [Steve Henson]\n\n  *) Fix"...,
>      siglen=0xbffff330, rsa=0x23) at rsa_sign.c:132
> #6  0x400fb7e1 in EVP_SignFinal (ctx=0x80b1a10,
>      sigret=0x803 <Address 0x803 out of bounds>, siglen=0xbffff330,
>      pkey=0xbffff2c0) at p_sign.c:111
> #7  0x0805c722 in do_fp (out=0x80b3be8,
>      buf=0x80b1a10 "S. Engelschall]\n\n  *) Fix the various library
>    and apps files to free up pkeys obtained from\n
>    X509_PUBKEY_get() et al. Also allow x509.c to handle netscape
>    extensions.\n     [Steve Henson]\n\n  *) Fix"...,
>      bp=0xbffff330, sep=0, binout=1, key=0x80b3de8, sigin=0x0,
>    siglen=2051,
>      title=0x80920cb "", file=0x803 <Address 0x803 out of bounds>) at
>    dgst.c:453
> #8  0x0805bea5 in dgst_main (argc=0, argv=0xbffff9d0) at dgst.c:385
> #9  0x08055d3a in do_cmd (prog=0x80b1290, argc=7, argv=0xbffff9b8)
>      at openssl.c:382
> #10 0x08055b6e in main (Argc=7, Argv=0xbffff9b8) at openssl.c:301
> ======
>
> When I specify -rand, segfault doesn't occur
>
> LD_LIBRARY_PATH=. ./apps/openssl dgst -sha1 -sign keyrsa.pem -out
>    dsignrsa.bin -rand /dev/random CHANGES
>
> finish successfully.
>
> The same problem is on smime.
>
> I do:
> ======
> LD_LIBRARY_PATH=. ./apps/openssl smime -encrypt -binary -in CHANGES
>    -aes256 -out encryptionrsa.pem -outform pem certrsa.pem
> LD_LIBRARY_PATH=. ./apps/openssl smime -decrypt -binary -in
>    encryptionrsa.pem -recip certrsa.pem -inkey keyrsa.pem -out
>    smime_decrrsa.dump -inform pem
> =====
>
> Segfault occurs on decrypt with the same backtrace:
> =========
> #0  0x400bdca8 in BN_BLINDING_set_thread_id (b=0xb51, n=1076728596)
>      at bn_blind.c:267
> 267             b->thread_id = n;
> (gdb) bt
> #0  0x400bdca8 in BN_BLINDING_set_thread_id (b=0xb51, n=1076728596)
>      at bn_blind.c:267
> #1  0x400d7f85 in RSA_setup_blinding (rsa=0x80b1e48, in_ctx=0x80b4050)
>      at rsa_lib.c:405
> #2  0x400d6fc9 in rsa_get_blinding (rsa=0x80b1e48, r=0xbfffe198,
>      local=0xbfffe19c, ctx=0x80b4050) at rsa_eay.c:251
> #3  0x400d649f in RSA_eay_private_decrypt (flen=64,
>      from=0xb51 <Address 0xb51 out of bounds>,
>      to=0x80b4000
>    "MobvxUlZUTDmtnqei5qEsbdjUzWrlWk/yhAu1MpYYjtAOmUh/
0OwN+ske\nKGegsfJuRc1C1alZTc1",

>    rsa=0x80b1e48, padding=1) at rsa_eay.c:482
> #4  0x400d7c90 in RSA_private_decrypt (flen=2897,
>      from=0xb51 <Address 0xb51 out of bounds>,
>      to=0xb51 <Address 0xb51 out of bounds>, rsa=0x402d9314,
>    padding=2897)
>      at rsa_lib.c:294
> #5  0x400fc61f in EVP_PKEY_decrypt (key=0xb51 <Address 0xb51 out of
>    bounds>,
>      ek=0xb51 <Address 0xb51 out of bounds>, ekl=2897, priv=0xb51) at
>    p_dec.c:83
> #6  0x4013edf9 in PKCS7_dataDecode (p7=0x80b1bf8, pkey=0x80b1bd8,
>    in_bio=0x0,
>      pcert=0x80b1eb8) at pk7_doit.c:442
> #7  0x40141126 in PKCS7_decrypt (p7=0x80b1bf8, pkey=0x80b1bd8,
>    cert=0x0,
>      data=0x80b1d98, flags=128) at pk7_smime.c:450
> #8  0x08089995 in smime_main (argc=13, argv=0xbffff968) at smime.c:687
> #9  0x08055d3a in do_cmd (prog=0x80b1290, argc=13, argv=0xbffff968)
>      at openssl.c:382
> #10 0x08055b6e in main (Argc=13, Argv=0xbffff968) at openssl.c:301
> =========
>
> The problem doesn't exist on 20050523 snapshot.
>
> Thank you.
>
> PS. We have a extra test suite testing openssl executable.
> Unfortunately, it's GNU make specific. Are you interested in it?


--
Richard Levitte
[hidden email]
______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
Development Mailing List                       [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: [openssl.org #1090] [BUG] Segfault in dgst signing with rsa private key

Dmitry Belyavsky
Hello!

On Mon, 6 Jun 2005, Richard Levitte via RT wrote:

>
> Did you check that you actually get the right library?  Do it like this:
>
> LD_LIBRARY_PATH=. ldd ./apps/openssl

beldmit@manul$ LD_LIBRARY_PATH=. ldd ./apps/openssl
         libssl.so.0.9.8 => ./libssl.so.0.9.8 (0x40018000)
         libcrypto.so.0.9.8 => ./libcrypto.so.0.9.8 (0x40056000)
         libdl.so.2 => /lib/libdl.so.2 (0x40194000)
         libz.so.1 => /usr/lib/libz.so.1 (0x40197000)
         libc.so.6 => /lib/libc.so.6 (0x401a9000)
         /lib/ld-linux.so.2 => /lib/ld-linux.so.2 (0x40000000)

I've tried it again with 20050606 snapshot.

>> make report:
>> ========
OpenSSL self-test report:

OpenSSL version:  0.9.8-beta4-dev
Last change:      Correct naming of the 'chil' and '4758cca' ENGINEs. Thi...
Options:          -g enable-shared enable-zlib no-gmp no-krb5 no-mdc2 no-rc5 no-zlib-dynamic
OS (uname):       Linux manul 2.4.26-1-386 #1 Tue Aug 24 13:31:19 JST 2004 i686 GNU/Linux
OS (config):      i686-whatever-linux2
Target (default): linux-elf
Target:           linux-elf
Compiler:         Configured with: ../src/configure -v --enable-languages=c,c++,java,f77,pascal,objc,ada,treelang --prefix=/usr --mandir=/usr/share/man --infodir=/usr/share/info --with-gxx-include-dir=/usr/include/c++/3.3 --enable-shared --enable-__cxa_atexit --with-system-zlib --enable-nls --without-included-gettext --enable-clocale=gnu --enable-debug --enable-java-gc=boehm --enable-java-awt=xlib --enable-objc-gc i486-linux
Thread model: posix
gcc version 3.3.5 (Debian 1:3.3.5-12)

Test skipped.
============

>> LD_LIBRARY_PATH=. ./apps/openssl req -newkey rsa:512 -nodes -batch
>>    -keyout keyrsa.pem -out reqrsa.pem -config apps/openssl.cnf
>> LD_LIBRARY_PATH=. ./apps/openssl x509 -req -set_serial 1 -signkey
>>    keyrsa.pem -in reqrsa.pem -out certrsa.pem
>> LD_LIBRARY_PATH=. ./apps/openssl dgst -sha1 -sign keyrsa.pem -out
>>    dsignrsa.bin CHANGES
>>
>> It causes a segfault with backtrace

This sequence of commands doesn't cause a segfault now. It fails with
correct error message.

>> The same problem is on smime.
>>
>> I do:
>> ======
>> LD_LIBRARY_PATH=. ./apps/openssl smime -encrypt -binary -in CHANGES
>>    -aes256 -out encryptionrsa.pem -outform pem certrsa.pem
>> LD_LIBRARY_PATH=. ./apps/openssl smime -decrypt -binary -in
>>    encryptionrsa.pem -recip certrsa.pem -inkey keyrsa.pem -out
>>    smime_decrrsa.dump -inform pem
>> =====
>>
>> Segfault occurs on decrypt with the same backtrace:

Segfault doesn't occurs here too. But it doesn't decrypt without
specified -rand key. It seems to be a bug.

>> PS. We have a extra test suite testing openssl executable.
>> Unfortunately, it's GNU make specific. Are you interested in it?

Are you interested in this test suite?

Thank you.

--
SY, Dmitry Belyavsky (ICQ UIN 11116575)

______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
Development Mailing List                       [hidden email]
Automated List Manager                           [hidden email]