openssl-0.9.8-beta3, Sun Forte compiler, 64-bit Solaris 8, core dump and RNG error.

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

openssl-0.9.8-beta3, Sun Forte compiler, 64-bit Solaris 8, core dump and RNG error.

Jostein Tveit
Hi, OpenSSL developers.

I try to compile openssl-0.9.8-beta3 on Solaris 8, using 64-bit
Sun Forte compiler (cc: Sun C 5.7 2005/01/07)

When doing 'make test' I get a core dump. But after applying a
simple patch, I am unable to track down the bug any further.

The Solaris PRNG patch 112438 (as mentioned in the FAQ) is
installed.


$ ./Configure solaris64-sparcv9-cc shared
$ make
$ make test

[...]

-----
Country Name (2 letter code) [AU]:AU
Organization Name (eg, company) []:Dodgy Brothers
Common Name (eg, YOUR name) []:Dodgy CA

convert the certificate request into a self signed certificate
using 'x509'
unable to load 'random state'
This means that the random number generator has not been seeded
with much random data.
Consider setting the RANDFILE environment variable to point at a
file that
'random' data can be kept in (the file will be overwritten).
Signature ok
subject=/C=AU/O=Dodgy Brothers/CN=Dodgy CA
Getting Private key
Segmentation Fault - core dumped
error using 'x509' to self sign a certificate request
make[1]: *** [test_ss] Error 1
make[1]: Leaving directory
`/opt/home/jtv/apps/openssl-0.9.8-beta3/test'
make: *** [tests] Error 2



core dump stack trace:

program terminated by signal SEGV (no mapping at the fault
address)
0xffffffff7f0af3cc: BN_BLINDING_set_thread_id+0x0004:   stx
%o1, [%o0 + 32]
(dbx) where
=>[1] BN_BLINDING_set_thread_id(0x0, 0x5b0c, 0x1001dcca0, 0x0, 0xffffffff7f2881d8, 0x30), at 0xffffffff7f0af3cc
  [2] RSA_setup_blinding(0x0, 0x1001eb000, 0x1001eb000, 0x1b87c4,0x0, 0x1001dcc10), at 0xffffffff7f0cfc44
  [3] RSA_eay_private_encrypt(0x0, 0x1001aeaf0, 0x1001eb000,0x1001dcd00, 0x1, 0x1001e91a0), at 0xffffffff7f0cd6b8
  [4] RSA_sign(0x1001aeaf0, 0xffffffff7fffdf70, 0x23,0x1001ea470, 0xffffffff7fffe0ac, 0x1001dcd00), at0xffffffff7f0d007c
  [5] EVP_SignFinal(0xffffffff7fffe088, 0x1001ea470,0xffffffff7fffe0ac, 0x100190860, 0xffffffff7fffdf70, 0x18fe9c),at 0xffffffff7f0f8414
  [6] ASN1_item_sign(0x1001ea470, 0x100194620, 0x100194540,0x1001dbbf0, 0x40, 0x100190860), at 0xffffffff7f1049e4
  [7] X509_sign(0x1001ae480, 0x100190860, 0xffffffff7f2a7f50,0x100187e68, 0x15895c, 0x3c00), at 0xffffffff7f12f8ac
  [8] sign(0x1001ae480, 0x100190860, 0x1e, 0x0,0xffffffff7f2a7f50, 0x5c00), at 0x100031dc0
  [9] x509_main(0x2bc8, 0xffffffff7ffff19d, 0x0, 0x100169c68,0x0, 0x1001800e0), at 0x1000310dc
  [10] do_cmd(0x10017fe10, 0x10, 0xffffffff7fffed70,0xffffffff7ffff128, 0x158ed4, 0xffffffff7fffed70), at0x100010dcc
  [11] main(0x10017fe10, 0xffffffff7fffed68, 0x100176a68,0x100176a90, 0xffffffff7e7bf5f0, 0x1001d3450), at 0x100010b18



Then I applied the following patch to prevent the core dump:

diff -ur openssl-0.9.8-beta3-orig/crypto/rsa/rsa_lib.c openssl-0.9.8-beta3-work/crypto/rsa/rsa_lib.c
--- openssl-0.9.8-beta3-orig/crypto/rsa/rsa_lib.c 2005-05-11 03:45:33.000000000 +0000
+++ openssl-0.9.8-beta3-work/crypto/rsa/rsa_lib.c 2005-06-01 12:31:33.490002000 +0000
@@ -402,6 +402,11 @@

        ret = BN_BLINDING_create_param(NULL, e, rsa->n, ctx,
                        rsa->meth->bn_mod_exp,
        rsa->_method_mod_n);
+       if (!ret)
+               {
+               RSAerr(RSA_F_RSA_SETUP_BLINDING, ERR_R_MALLOC_FAILURE);
+               goto err;
+               }
        BN_BLINDING_set_thread_id(ret, CRYPTO_thread_id());
 err:
        BN_CTX_end(ctx);


And then compiled again:
$ ./Configure solaris64-sparcv9-cc shared
$ make
$ make test

[...]

convert the certificate request into a self signed certificate
using 'x509'
unable to load 'random state'
This means that the random number generator has not been seeded
with much random data.
Consider setting the RANDFILE environment variable to point at a
file that
'random' data can be kept in (the file will be overwritten).
Signature ok
subject=/C=AU/O=Dodgy Brothers/CN=Dodgy CA
Getting Private key
29274:error:24064064:random number
generator:SSLEAY_RAND_BYTES:PRNG not seeded:md_rand.c:503:You
need to read the OpenSSL FAQ,
http://www.openssl.org/support/faq.html
29274:error:04088041:rsa routines:RSA_setup_blinding:malloc
failure:rsa_lib.c:407:
29274:error:04066044:rsa
routines:RSA_EAY_PRIVATE_ENCRYPT:internal error:rsa_eay.c:364:
29274:error:0D0C3006:asn1 encoding routines:ASN1_item_sign:EVP
lib:a_sign.c:276:
error using 'x509' to self sign a certificate request
make[1]: *** [test_ss] Error 1
make[1]: Leaving directory
`/opt/home/jtv/apps/openssl-0.9.8-beta3-work/test'
make: *** [tests] Error 2


It seemes to be some kind of problem with RNG seeding.
I tried to set RANDFILE, but that didn't help.

Is it possible to turn on some RNG debugging?

Regards,
--
Jostein Tveit <[hidden email]>
______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
Development Mailing List                       [hidden email]
Automated List Manager                           [hidden email]