Problems porting Openssl 1.1.1d to zos.

classic Classic list List threaded Threaded
14 messages Options
Reply | Threaded
Open this post in threaded view
|

Problems porting Openssl 1.1.1d to zos.

OpenSSL - User mailing list
Is there anyone on this group with experience with ebcdic platforms,
specifically zOS?  I have built 1.1.1d on zOS and connections to my
server work for firefox 60 but not newer versions.  I don't know exactly
where the cut off is or what they changed but current versions get an
HMAC error.  I strongly suspect that it is keying the hmac with some
combination of inputs that include an improperly translated text string,
but I don't know for sure.  Its quite hard to track down when you don't
have a debugger.

The error message:

> An error occurred during a connection to cafe.na.tibco.com:1802. SSL
> received a record with an incorrect Message Authentication Code. Error
> code: SSL_ERROR_BAD_MAC_READ

If anyone can suggest an approach to figuring this out I'd be grateful.


Wendell Nichols

Reply | Threaded
Open this post in threaded view
|

Re: Problems porting Openssl 1.1.1d to zos.

Matthias St. Pierre

On 11.11.19 16:42, Wendell Nichols via openssl-users wrote:

> Is there anyone on this group with experience with ebcdic platforms, specifically zOS?  I have built 1.1.1d on zOS and connections to my server work for firefox 60 but not newer versions.  I don't know exactly where the cut off is or what they changed but current versions get an HMAC error.  I strongly suspect that it is keying the hmac with some combination of inputs that include an improperly translated text string, but I don't know for sure.  Its quite hard to track down when you don't have a debugger.
>
> The error message:
>
>> An error occurred during a connection to cafe.na.tibco.com:1802. SSL received a record with an incorrect Message Authentication Code. Error code: SSL_ERROR_BAD_MAC_READ
>
> If anyone can suggest an approach to figuring this out I'd be grateful.
>
>
> Wendell Nichols
>

Incidentally, I just merged a pull request that fixed a misspelled EBCDIC string to master and 1.1.1.

https://github.com/openssl/openssl/pull/10396#issuecomment-552506972

But I have no idea whether it is related to your problem. Nevertheless, you might want to retry with the current tip of the OpenSSL_1_1_1-stable branch.


Regards,
Matthias

Reply | Threaded
Open this post in threaded view
|

Re: Problems porting Openssl 1.1.1d to zos.

Matthias St. Pierre
Please see also GitHub issue #4154, in particular

https://github.com/openssl/openssl/issues/4154#issuecomment-552838141


Reply | Threaded
Open this post in threaded view
|

Re: Problems porting Openssl 1.1.1d to zos.

Patrick Steuer-2
In reply to this post by OpenSSL - User mailing list
 > An error occurred during a connection to cafe.na.tibco.com:1802. SSL
 > received a record with an incorrect Message Authentication Code. Error
 > code: SSL_ERROR_BAD_MAC_READ

In case this error occurs with a chacha-poly cipher suite,
the following PR probably has a fix:
https://github.com/openssl/openssl/pull/10417

Patrick
Reply | Threaded
Open this post in threaded view
|

Re: Problems porting Openssl 1.1.1d to zos.

K Lengauer
In reply to this post by OpenSSL - User mailing list
Dear all,

I stumbled across this mails when looking for information regarding OpenSSL
on zOS. Currently, I am working on getting OpenSSL 1.1.1c running on zOS. So
far I created my own config "target" inside 10-main.conf based on the old
configuration that was used pre OpenSSL 1.1.0.

Still, I was not able to get that far yet and was wondering how you
proceeded regarding this issue? Apart from some github issues and the likes
regarding EBCDIC patches I was unable to find much advice on how to
configure the build and how to tackle common issues. One config target for
zOS OS390 was created a while back but then removed again from OpenSSL
1.1.1. With my config target I made it to the compile step but still face
build issues. Reaching a complete build like Wendell Nichols would already
by a great step.

Are any of you aware of zOS specific OpenSSL development branches that I
have yet to find? Or is this mainly done in private repositories/branches
and not offered for PRs to the OpenSSL repository?

Any advice is greatly appreciated.



--
Sent from: http://openssl.6102.n7.nabble.com/OpenSSL-User-f3.html
Reply | Threaded
Open this post in threaded view
|

Re: Problems porting Openssl 1.1.1d to zos.

Michael Mueller
We recently abandoned our effort to port 1.1.1d to zos. Attempting to use GSK now. Lack of a zos dev community is a hurdle.

M

On Mon, Mar 2, 2020, 6:04 AM K Lengauer <[hidden email]> wrote:
Dear all,

I stumbled across this mails when looking for information regarding OpenSSL
on zOS. Currently, I am working on getting OpenSSL 1.1.1c running on zOS. So
far I created my own config "target" inside 10-main.conf based on the old
configuration that was used pre OpenSSL 1.1.0.

Still, I was not able to get that far yet and was wondering how you
proceeded regarding this issue? Apart from some github issues and the likes
regarding EBCDIC patches I was unable to find much advice on how to
configure the build and how to tackle common issues. One config target for
zOS OS390 was created a while back but then removed again from OpenSSL
1.1.1. With my config target I made it to the compile step but still face
build issues. Reaching a complete build like Wendell Nichols would already
by a great step.

Are any of you aware of zOS specific OpenSSL development branches that I
have yet to find? Or is this mainly done in private repositories/branches
and not offered for PRs to the OpenSSL repository?

Any advice is greatly appreciated.



--
Sent from: http://openssl.6102.n7.nabble.com/OpenSSL-User-f3.html
Reply | Threaded
Open this post in threaded view
|

Re: Problems porting Openssl 1.1.1d to zos.

Patrick Steuer-2
In reply to this post by K Lengauer
> I stumbled across this mails when looking for information regarding OpenSSL
> on zOS. Currently, I am working on getting OpenSSL 1.1.1c running on zOS. So
> far I created my own config "target" inside 10-main.conf based on the old
> configuration that was used pre OpenSSL 1.1.0.

> Still, I was not able to get that far yet and was wondering how you
> proceeded regarding this issue? Apart from some github issues and the likes
> regarding EBCDIC patches I was unable to find much advice on how to
> configure the build and how to tackle common issues. One config target for
> zOS OS390 was created a while back but then removed again from OpenSSL
> 1.1.1. With my config target I made it to the compile step but still face
> build issues. Reaching a complete build like Wendell Nichols would already
> by a great step.

Regarding z/OS, the build environment is usually the hard part:
We build from EBCDIC sources (gunzip the tar file, use pax to extract
from tar) using xplink linkage, 64 bit and a perl version that works
with EBCDIC.

Relevant parts from the config files that we are using:

10-main.conf
--------
      "OS390-Unix" => {
          inherit_from     => [ "BASE_unix", asm("zos_asm") ],
          cc               => "./tools/c99.sh",
          cflags           => "-O -Wc,dll,XPLINK,exportall,hgpr,lp64
-Wa,'GOFF,SYSPARM(USE_XPLINK)' -qlongname -qlanglvl=extc99 -DB_ENDIAN
-DCHARSET_EBCDIC -DNO_SYS_PARAM_H -D_ALL_SOURCE -D_OPEN_THREADS=2
-D_POSIX_SOURCE  -D_OPEN_MSGQ_EXT",
          module_ldflags   => "-Wl,XPLINK,LP64",
          shared_ldflags   => "-Wl,dll,XPLINK,LP64",
          bn_ops           => "THIRTY_TWO_BIT RC4_CHAR",
          thread_scheme    => "(unknown)",
          perlasm_scheme   => "zOS64",
      },
--------

00-base-templates.conf
--------
# Either comment out all the asm below, or use a no-asm build.
     zos_asm => {
         template        => 1,
         cpuid_asm_src   => "s390xcap.c s390xcpuid.s",
         bn_asm_src      => "bn_asm.c", # s390x-gf2m.s",
         ec_asm_src      => "ecp_s390x_nistp.c",
         aes_asm_src     => "aes-s390x.s aes-ctr.fake aes-xts.fake",
         sha1_asm_src    => "sha1-s390x.s sha256-s390x.s sha512-s390x.s",
         rc4_asm_src     => "rc4-s390x.s",
         modes_asm_src   => "ghash-s390x.s",
#       chacha_asm_src  => "chacha-s390x.s",
#       poly1305_asm_src => "poly1305-s390x.s",
         keccak1600_asm_src      => "keccak1600-s390x.s",
--------

tools/c99.sh
--------
#!/bin/sh -k
#
# Re-order arguments so that -L comes first
#
opts=""
lopts=""

for arg in $* ; do
   case $arg in
     -L*) lopts="$lopts $arg" ;;
     *) opts="$opts $arg" ;;
   esac
done

c99 -Wl,dll $lopts $opts
--------


> Are any of you aware of zOS specific OpenSSL development branches that I
> have yet to find? Or is this mainly done in private repositories/branches
> and not offered for PRs to the OpenSSL repository?

We are working on porting OpenSSL for z/OS internally right now and will
share the work in progress (via a PR to the OpenSSL repo) as soon as it
makes sense, probably in the near future.

Best,
Patrick
Reply | Threaded
Open this post in threaded view
|

Re: Problems porting Openssl 1.1.1d to zos.

Michael Mueller
This is a very helpful post. Thank you.

We lacked Perl and had no clear path to getting it. Can't say this deficiency caused our project to miss generous targets, but it certainly contributed.


On Wed, Mar 4, 2020, 6:07 AM Patrick Steuer <[hidden email]> wrote:
> I stumbled across this mails when looking for information regarding OpenSSL
> on zOS. Currently, I am working on getting OpenSSL 1.1.1c running on zOS. So
> far I created my own config "target" inside 10-main.conf based on the old
> configuration that was used pre OpenSSL 1.1.0.

> Still, I was not able to get that far yet and was wondering how you
> proceeded regarding this issue? Apart from some github issues and the likes
> regarding EBCDIC patches I was unable to find much advice on how to
> configure the build and how to tackle common issues. One config target for
> zOS OS390 was created a while back but then removed again from OpenSSL
> 1.1.1. With my config target I made it to the compile step but still face
> build issues. Reaching a complete build like Wendell Nichols would already
> by a great step.

Regarding z/OS, the build environment is usually the hard part:
We build from EBCDIC sources (gunzip the tar file, use pax to extract
from tar) using xplink linkage, 64 bit and a perl version that works
with EBCDIC.

Relevant parts from the config files that we are using:

10-main.conf
--------
      "OS390-Unix" => {
          inherit_from     => [ "BASE_unix", asm("zos_asm") ],
          cc               => "./tools/c99.sh",
          cflags           => "-O -Wc,dll,XPLINK,exportall,hgpr,lp64
-Wa,'GOFF,SYSPARM(USE_XPLINK)' -qlongname -qlanglvl=extc99 -DB_ENDIAN
-DCHARSET_EBCDIC -DNO_SYS_PARAM_H -D_ALL_SOURCE -D_OPEN_THREADS=2
-D_POSIX_SOURCE  -D_OPEN_MSGQ_EXT",
          module_ldflags   => "-Wl,XPLINK,LP64",
          shared_ldflags   => "-Wl,dll,XPLINK,LP64",
          bn_ops           => "THIRTY_TWO_BIT RC4_CHAR",
          thread_scheme    => "(unknown)",
          perlasm_scheme   => "zOS64",
      },
--------

00-base-templates.conf
--------
# Either comment out all the asm below, or use a no-asm build.
     zos_asm => {
         template        => 1,
         cpuid_asm_src   => "s390xcap.c s390xcpuid.s",
         bn_asm_src      => "bn_asm.c", # s390x-gf2m.s",
         ec_asm_src      => "ecp_s390x_nistp.c",
         aes_asm_src     => "aes-s390x.s aes-ctr.fake aes-xts.fake",
         sha1_asm_src    => "sha1-s390x.s sha256-s390x.s sha512-s390x.s",
         rc4_asm_src     => "rc4-s390x.s",
         modes_asm_src   => "ghash-s390x.s",
#       chacha_asm_src  => "chacha-s390x.s",
#       poly1305_asm_src => "poly1305-s390x.s",
         keccak1600_asm_src      => "keccak1600-s390x.s",
--------

tools/c99.sh
--------
#!/bin/sh -k
#
# Re-order arguments so that -L comes first
#
opts=""
lopts=""

for arg in $* ; do
   case $arg in
     -L*) lopts="$lopts $arg" ;;
     *) opts="$opts $arg" ;;
   esac
done

c99 -Wl,dll $lopts $opts
--------


> Are any of you aware of zOS specific OpenSSL development branches that I
> have yet to find? Or is this mainly done in private repositories/branches
> and not offered for PRs to the OpenSSL repository?

We are working on porting OpenSSL for z/OS internally right now and will
share the work in progress (via a PR to the OpenSSL repo) as soon as it
makes sense, probably in the near future.

Best,
Patrick
Reply | Threaded
Open this post in threaded view
|

Re: Problems porting Openssl 1.1.1d to zos.

K Lengauer
In reply to this post by Patrick Steuer-2
Thank you very much Patrick Steuer.

This certainly helps! I am now also in the progress of building OpenSSL and
come across missing "cflags" and the likes so with your config I can
hopefully save some time as well as verify what I already use.
I will also share my config in the near future once I am getting closer to a
successful build. Most of the flags I added in my "OS390-Unix" target are
the same or similar to yours. Again, thank you.

@Michael Mueller: The zOS system Perl was also below the minimum required
Perl version but our system administrator pointed me to using
rocketsoftware's Perl 5.24 which he already added to the zOS machine I am
using some time ago. I hear mixed opinions regarding rocketsoftware but so
far I was able to work with it. Just make sure to adjust your LIBPATH,
PERL5LIB and so on to make that Perl work.
Just sharing in case you did not yet try that. If you get stuck configuring
the new Perl I could share my settings.



--
Sent from: http://openssl.6102.n7.nabble.com/OpenSSL-User-f3.html
Reply | Threaded
Open this post in threaded view
|

Re: Problems porting Openssl 1.1.1d to zos.

Patrick Steuer-2
Regarding perl, this is the version which works for us :

 > perl -v

This is perl 5, version 24, subversion 0 (v5.24.0) built for os390

Copyright 1987-2016, Larry Wall
MVS (OS390) port by Mortice Kern Systems, 1997-1999

Perl may be copied only under the terms of either the Artistic License
or the
GNU General Public License, which may be found in the Perl 5 source kit.

Complete documentation for Perl, including FAQ lists, should be found on
this system using "man perl" or "perldoc perl".  If you have access to the
Internet, point your browser at http://www.perl.org/, the Perl Home Page.
Reply | Threaded
Open this post in threaded view
|

Re: Problems porting Openssl 1.1.1d to zos.

OpenSSL - User mailing list
In reply to this post by K Lengauer
Perhaps someone should writeup and submit a "NOTES.zos" file to add?


Reply | Threaded
Open this post in threaded view
|

Re: Problems porting Openssl 1.1.1d to zos.

Patrick Steuer-2
On 3/4/20 5:31 PM, Salz, Rich via openssl-users wrote:
> Perhaps someone should writeup and submit a "NOTES.zos" file to add?

I could put the contents of my previous mail in a NOTES.zos file,
if that would be considered helpful, knowing it works for us
at the moment and might not to the trick for other build
environments.

Thinking of a more comprehensive guide, id prefer to invest my
time in helping the ongoing porting effort instead.
Reply | Threaded
Open this post in threaded view
|

Re: Problems porting Openssl 1.1.1d to zos.

K Lengauer
Dear Patrick and co

I am currently stuck during my build with the following error:

IKJ56228I DATA SET CEE.SCEEBND2 NOT IN CATALOG OR CATALOG CAN NOT BE
ACCESSED    
FSUM3052 The data definition name SYSLIB cannot be resolved. The data set
was not found. Ensure that data set name CEE.SCEEBND2 is specified
correctly.

It seems there is some binder error. Did you face this or a similar error as
well?
Sadly the zOS data sets are still a bit confusing to me and I am still going
through the IBM documentation to get a better understanding.
I was told by a colleague to try adding some "INCLUDE"s to include various
CEE.SCEELIB(*) and setting SYSLIB="//CEE.SCEEBND2" but that did not work so
far.
How did you configure this with respect to OpenSSL? Or is this something
specific to our system?






--
Sent from: http://openssl.6102.n7.nabble.com/OpenSSL-User-f3.html
Reply | Threaded
Open this post in threaded view
|

Re: Problems porting Openssl 1.1.1d to zos.

Michael Wojcik
It's been years since I compiled C programs on z, but CEE.SCEEBND2 is the C library dataset (for AMODE 64), so it looks like the message is simply saying that it's not cataloged. That would appear to be a site-specific installation issue with the C compiler.

See for example this topic from the IBM docs:

https://www.ibm.com/support/knowledgecenter/SSLTBW_2.1.0/com.ibm.zos.v2r1.ceeam00/cllexx.htm

in which the JCL includes a straightforward DD card for CEE.SCEEBND2. It appears IBM expects that in a normal installation of the C compiler, that dataset will be in the catalog.

That said, the specific location of the SCEEBND2 dataset appears to be a convention. The text of that topic goes on to say: "The Language Environment resident routines are contained in the SCEEBND2 library; the data set name could be CEE.SCEEBND2. If you are unsure where SCEEBND2 has been installed at your location, contact your system administrator."

So, I suggest contacting your system administrator and asking where SCEEBND2 is. Or you could try searching the catalog for it using the appropriate ISPF or TSO commands.

From: openssl-users <[hidden email]> on behalf of K Lengauer <[hidden email]>
Sent: Monday, March 9, 2020 10:29
To: [hidden email] <[hidden email]>
Subject: Re: Problems porting Openssl 1.1.1d to zos.
 
Dear Patrick and co

I am currently stuck during my build with the following error:

IKJ56228I DATA SET CEE.SCEEBND2 NOT IN CATALOG OR CATALOG CAN NOT BE
ACCESSED   
FSUM3052 The data definition name SYSLIB cannot be resolved. The data set
was not found. Ensure that data set name CEE.SCEEBND2 is specified
correctly.

It seems there is some binder error. Did you face this or a similar error as
well?
Sadly the zOS data sets are still a bit confusing to me and I am still going
through the IBM documentation to get a better understanding.
I was told by a colleague to try adding some "INCLUDE"s to include various
CEE.SCEELIB(*) and setting SYSLIB="//CEE.SCEEBND2" but that did not work so
far.
How did you configure this with respect to OpenSSL? Or is this something
specific to our system?






--
Sent from: http://openssl.6102.n7.nabble.com/OpenSSL-User-f3.html