OpenSSL 0.9.8a Crashes Apache--and a Fix

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

OpenSSL 0.9.8a Crashes Apache--and a Fix

Aaron Greenspan
Hello,

I'm pretty sure I've found a bug in something, though I'm not positive.
So, I'll just relate my experience troubleshooting over the past few
days, and hope someone can investigate further.

I'm working on a Cobalt RaQ 550 server, which runs a version of Red Hat
Linux. (I've removed the hostname from the command output below for
privacy.)

[root apache]# uname -a  
Linux 2.4.19C13_V #1 Fri Feb 20 01:55:03 PST 2004 i686 unknown

It comes with an old version of Apache and PHP, so I worked out a
process to upgrade them both without breaking the Cobalt interface. So
far, so good. The problem arose when trying to compile Apache 1.3.34
with OpenSSL 0.9.8a as a DSO via mod_ssl. I could get Apache to run on
its own without a problem, but as soon as I enabled the following line
in /etc/httpd/conf/httpd.conf...

#LoadModule ssl_module modules/libssl.so

...the Apache httpd process would yield a segmentation fault, even
before it could start the web server, or even finish loading other DSOs.
I re-compiled Apache, mod_ssl, other module packages and OpenSSL
countless times to try to figure out what was going wrong. (Neither the
Apache documentation nor the mod_ssl documentation made it clear that by
compiling mod_ssl as a DSO, the final "libssl.so" file would be placed
in the source tree for Apache, instead of its proper location in a /lib
directory somewhere. This took me a very long while to figure out, since
other files called libssl.so were already on the system. It was news to
me that you had to copy it over manually, but I finally solved that
problem.)

My configuration command for mod_ssl 2.8.25 was as follows:

./configure --with-apache=/usr/local/apache_1.3.34
--with-ssl=/usr/local/openssl-0.9.8a --prefix=/usr/local/apache
--enable-shared=ssl

My configuration command for OpenSSL 0.9.8a was:

./config zlib shared no-threads --prefix=/usr/local
--openssldir=/usr/local/openssl

My configuration command for Apache, which I wanted to match Cobalt's,
was incredibly long and necessitated an entire file (which in turn
necessitated writing a shell script to debug it and automate the whole
installation process), so I'm going to leave it out since it's pretty
much irrelevant. Just know that other modules were not the problem, even
though there were many.

Meanwhile, the OpenSSL 0.9.8a configuration program would automatically
detect my system type as "linux-elf," and the entire "make" process ran
fine--until "make test." Then, the testing routine would fail with the
following:

[root openssl-0.9.8a]# make test
<SNIP>
test BN_add
test BN_sub
test BN_lshift1
test BN_lshift (fixed)
test BN_lshift
test BN_rshift1
test BN_rshift
test BN_sqr
make[1]: *** [test_bn] Error 139
make[1]: Leaving directory `/usr/local/openssl-0.9.8a/test'
make: *** [tests] Error 2
[root openssl-0.9.8a]#

Running "make install" created the expected files and put them in the
right place, but compiling and linking Apache's httpd against them
caused the aforementioned segmentation fault. Using gdb against the core
dump yielded this output:

[root /tmp]# gdb /usr/sbin/httpd core
GNU gdb 5.0rh-5 Red Hat Linux 7.1
Copyright 2001 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain
conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "i386-redhat-linux"...
Core was generated by `/usr/sbin/httpd'.
Program terminated with signal 11, Segmentation fault.
<SNIP>
Reading symbols from /lib/libcrypt.so.1...done.
Loaded symbols for /lib/libcrypt.so.1
<SNIP>
Reading symbols from /etc/httpd/modules/libssl.so...done.
Loaded symbols for /etc/httpd/modules/libssl.so
Reading symbols from /usr/local/lib/libssl.so.0.9.8...done.
Loaded symbols for /usr/local/lib/libssl.so.0.9.8
Reading symbols from /usr/local/lib/libcrypto.so.0.9.8...done.
Loaded symbols for /usr/local/lib/libcrypto.so.0.9.8
Reading symbols from /usr/lib/libz.so.1...done.
Loaded symbols for /usr/lib/libz.so.1
<SNIP>
Reading symbols from
/usr/local/zend/lib/Optimizer-2.5.10/php-4.4.x/ZendOptimizer.so...done.
Loaded symbols for
/usr/local/zend/lib/Optimizer-2.5.10/php-4.4.x/ZendOptimizer.so
#0  0x40298d51 in bn_mul_add_words () from /usr/local/lib/libcrypto.so.0.9.8
(gdb) bt
#0  0x40298d51 in bn_mul_add_words () from /usr/local/lib/libcrypto.so.0.9.8
#1  0x00000008 in __strtol_internal (nptr=Cannot access memory at
address 0xd8ddd08f
) at eval.c:35
Cannot access memory at address 0xd8ddd087
(gdb) quit

The culprit looked like the bn_mul_add_words() function. After repeating
the building process probably a dozen or more times, I finally tried
everything using exactly the same commands (and changing version numbers
where appropriate) with OpenSSL 0.9.7i. It worked like a charm--no
failed tests, no errors, and no segmentation faults. I noticed only one
key difference: the 0.9.7i configuration program automatically detected
the system as "linux-pentium," instead of "linux-elf." I tried
configuring and compiling 0.9.8a with "linux-pentium" as the system
type, but I was given an error implying that no such system type existed
for that configuration program.

Eventually, I forced the system type as "linux-generic32" with 0.9.8a
and everything finally worked, thanks to the following command:

/Configure linux-generic32 zlib shared no-threads --prefix=/usr/local
--openssldir=/usr/local/openssl

However, the small difference in terminology between "generic32" and
"pentium" and "elf" was enough to take up twenty hours of my time. Is
there a page somewhere that I missed blatantly outlining this issue? (Is
this the "compatibility issue" alluded to on the OpenSSL home page?)
Though I'm not sure what the technical difference is between the system
types, all of which sound generic to me, I think someone should probably
look into it if no one has already.

Hope this helps someone...

Aaron

Aaron Greenspan
President & CEO
Think Computer Corporation

http://www.thinkcomputer.com
______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
User Support Mailing List                    [hidden email]
Automated List Manager                           [hidden email]