When building OpenSSH 4.1p1 against OpenSSL 0.9.7g on Itanium (Linux) the
OpenSSH 'make tests' regression tests fail wrt the RC4 cipher.
At first glance, this appears to be an OpenSSL issue: The tests are successful
when OpenSSH is built against 0.9.7e, but later versions fail. Specifically,
the following versions have been tried: 0.9.7f, 0.9.7g, and 0.9.8-beta4. Both
OpenSSH 4.1p1 and OpenSSH 3.9po1 have been tried.
However, the OpenSSL test suite reveals no problem. The ciphertext output from
the tests is identical between 0.9.7e and 0.9.7g. It should also be noted
that the issue with OpenSSH seems to only occur after the authentication stage
when the daemon rekeys the session.
If OpenSSL is built with the 'no_asm' flag, the problem goes away. Alternatively,
if RC4_CHAR is set and SZ in crypto/rc4/asm/rc4-ia64.S is changed from 4 to 1,
the problem also goes away.
This has also been filed as bug #1055 with the OpenSSH folks.
Re: [openssl.org #1114] Bug: RC4 on IA64 and OpenSSH
On Tue Jun 14 13:12:00 2005, Iain Morgan via RT wrote:
> If OpenSSL is built with the 'no_asm' flag, the problem goes away. Alternatively,
> if RC4_CHAR is set and SZ in crypto/rc4/asm/rc4-ia64.S is changed from 4 to 1,
> the problem also goes away.
Oops. These workarounds don't actually work. I had modified try-ciphers.sh in the
OpenSSH regression suite in order to bypass any testing of the RC4 cipher so
that the rest of the regression tests could be completed. Unfortunately, I forgot
to clean up after myself after that.
Repeating those tests, with a clean distribution, shows that the workarounds don't