Re: [openssl.org #2449] [BUG] openssl 1.0.0d warnings during build and ACCVIO on OpenVMS

classic Classic list List threaded Threaded
54 messages Options
123
Reply | Threaded
Open this post in threaded view
|

Re: [openssl.org #2449] [BUG] openssl 1.0.0d warnings during build and ACCVIO on OpenVMS

Steven M. Schweda-2
From: "Arpadffy Zoltan via RT" <[hidden email]>

> test BN_mont
> Montgomery multiplication test failed!

   This seems to be caused by an apparently prevalent belief that the
way to do integer operations on a pointer is to type-cast that pointer
to size_t.  Sadly, on VMS, size_t is 32 bits (unsigned int), regardless
of the pointer size, so that, with 64-bit pointers, code like, for
example, the following (from "crypto/bn/bn_mont.c") fails:

[...]
        BN_ULONG *ap,*np,*rp,n0,v,*nrp;
[...]
        nrp=(BN_ULONG *)(((size_t)rp&~m1)|((size_t)ap&m1));
[...]

My ignorance of the algorithms involved is great, but I'd guess that
"crypto/bn/bn_nist.c" suffers from a lot of this stuff, too.  A global
search for "(size_t)" finds many suspicious applications of "(size_t)"
to pointers (and a bunch of loose variables declared as "size_t" which
might be better as something else, too).

   What seems (to me) to be needed in these cases is some macro or
typedef which is an integer whose size is reliably the same as that of a
pointer, which size_t is not.  For testing, I've inserted this into
"crypto/bn/bn.h":

#if defined(OPENSSL_SYS_VMS)
# if __INITIAL_POINTER_SIZE == 64
#  define PTR_SIZE_INT long long
# else /* __INITIAL_POINTER_SIZE == 64 */
#  define PTR_SIZE_INT int
# endif /* __INITIAL_POINTER_SIZE == 64 [else] */
#else /* defined(OPENSSL_SYS_VMS) */
# define PTR_SIZE_INT size_t
#endif /* defined(OPENSSL_SYS_VMS) [else] */

and then substituted "PTR_SIZE_INT" for "size_t", where (I thought)
appropriate, in "crypto/bn/bn_mont.c" and "crypto/bn/bn_nist.c".  I
suspect that more code in more places could use changes like these, but
these changes, along with a bunch of other, more VMS-specific builder
and code changes, seem to clear the compiler complaints and test
failures.  (Is "size_t" really a good choice everywhere else?)  I didn't
look beyond "crypto/bn", but if this kind of thing is done elsewhere,
too, then it would probably make more sense to get this thing
established at a higher level than "crypto/bn/bn.h".

   Speaking of other VMS-specific builder and code changes, why is this
product being built with the highly restrictive C compiler option
/STANDARD=ANSI89 instead of the default, /STANDARD=RELAXED?  The
VMS-specific gmtime()-replacement code (in "crypto/o_time.c") depends on
32-bit pointers.  (It also appears to be sub-optimal.  For example, when
is "logvalue" calculated?  Always.  When is it used?  Sometimes.)  The
gmtime() in the C RTL (since VMS V7.0) seems to work with pointers of
either size, but it's not visible with /STANDARD=ANSI89 (which defines
_ANSI_C_SOURCE).   Rather than try to fix the old VMS-specific
gmtime()-replacement code, I just started using the existing
gmtime()-using code, and saying:
      USER_CCFLAGS == "/STANDARD=RELAXED"
before running the builder.  I didn't see any problems from that.  (And
it would probably allow tossing all that junk which is currently needed
to suppress compiler complaints like %CC-I-DOLLARID (and others?).)

   I don't know where or how anyone would like to add the
pointer-sized-integer macro/typedef/whatever, so I didn't try to find
and fix every place where it ought to be used.  If someone could decide
how to do that, then I could revise, collect, and submit my suggested
changes.  (Most of which, if experience is any guide, would probably
then vanish without a trace, but that doesn't seem to have stopped me
yet.)

------------------------------------------------------------------------

   Steven M. Schweda               sms@antinode-info
   382 South Warwick Street        (+1) 651-699-9818
   Saint Paul  MN  55105-2547
______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
Development Mailing List                       [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

RE: [openssl.org #2449] [BUG] openssl 1.0.0d warnings during build and ACCVIO on OpenVMS

Green, Paul
Steven M. Schweda wrote:

> What seems (to me) to be needed in these cases is some macro or
> typedef which is an integer whose size is reliably the same as
> that of a pointer, which size_t is not.  

Hi Steve, Please take a look at your copy of stdint.h.  See if you have
a definition for the "intptr_t" and "uintptr_t" types.

The POSIX standard(*) says "The following type designates a signed
integer type with the property that any valid pointer to void can be
converted to this type, then converted back to a pointer to void, and
the result will compare equal to the original pointer: intptr_t."  Ditto
for uintptr_t, except that it is unsigned.

The standard also says that intptr_t and uintptr_t are required on
XSI-conformant systems; otherwise, they are optional. So you might have
to define _XSI_SOURCE to make their declarations visible.

It seems to me that this data type is just what you (and OpenSSL) are
looking for.  Hope this info helps.

(*) The POSIX-2008 standard is online at
http://pubs.opengroup.org/onlinepubs/9699919799/toc.htm.  You must
pre-register to view it, but the registration step carries no charge and
is simple to perform.

Thanks
PG
--
Paul Green, Senior Technical Consultant, Stratus Technologies.

______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
Development Mailing List                       [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: [openssl.org #2449] [BUG] openssl 1.0.0d warnings during build and ACCVIO on OpenVMS

Steven M. Schweda-2
In reply to this post by Steven M. Schweda-2
From: "Green, Paul" <[hidden email]>

> > What seems (to me) to be needed in these cases is some macro or
> > typedef which is an integer whose size is reliably the same as
> > that of a pointer, which size_t is not. =20
>
> Hi Steve, Please take a look at your copy of stdint.h.  See if you have
> a definition for the "intptr_t" and "uintptr_t" types.

   Around here, they're in <inttypes.h>.  (We don't have to show you any
stinking <stdint.h>!)

> The POSIX standard(*) says "The following type designates a signed
> integer type with the property that any valid pointer to void can be
> converted to this type, then converted back to a pointer to void, and
> the result will compare equal to the original pointer: intptr_t."  Ditto
> for uintptr_t, except that it is unsigned.
>
> The standard also says that intptr_t and uintptr_t are required on
> XSI-conformant systems; otherwise, they are optional. So you might have
> to define _XSI_SOURCE to make their declarations visible.
>
> It seems to me that this data type is just what you (and OpenSSL) are
> looking for.  Hope this info helps.

   You might think so, but, in its boundless wisdom, DEC/Compaq/HP has
implemented these things in an unfortunate way:

/*
**  Declare [un]signed integral types large enough to hold any pointer.
*/
#if !defined(__VAX)
   typedef int64_t intptr_t;
   typedef uint64_t uintptr_t;
#else
   typedef int32_t intptr_t;
   typedef uint32_t uintptr_t;
#endif

So, on a non-VAX (64-bit-capable) system, these types are _always_ 64
bits, irregardful of the actual pointer size.  This leads to annoying
complaints like %CC-I-INTCONSTTRUNC (and friends) when trying to get
from one of these types back to a 32-bit pointer.  I sent a complaint
about this to HP back in 2009, but the responding Indian didn't see it
as a problem, so I gave up.  (And even if they did fix it now, many old
systems would be stuck with the problem, so it's mostly hopeless.)

   Because of these sorry facts, I see no easy way to avoid adding and
using an OpenSSL-defined type, although on most systems, "intptr_t"
would seem to be an ideal value to use for the thing.

> (*) The POSIX-2008 standard is online at
> http://pubs.opengroup.org/onlinepubs/9699919799/toc.htm.  You must
> pre-register to view it, but the registration step carries no charge and
> is simple to perform.

   Thanks for the link.  Perhaps I'll forward that to HP when I get
sufficiently bored.


   On a different point, a closer look at the 64-bit-pointer test
results shows a problem on Alpha (but not on IA64) somewhere in the "CMS
=> PKCS#7 compatibility test" sequence.  Perhaps some file I/O thing?
The perl script doesn't seem to handle an explosion here very well.  A
search for "fail" in the test results transcript won't detect the
absence of an "ALL TESTS SUCCESSFUL" message, making it easy for a
casual inspection to miss this kind of failure.

------------------------------------------------------------------------

   Steven M. Schweda               sms@antinode-info
   382 South Warwick Street        (+1) 651-699-9818
   Saint Paul  MN  55105-2547
______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
Development Mailing List                       [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: [openssl.org #2449] [BUG] openssl 1.0.0d warnings during build and ACCVIO on OpenVMS

Steven M. Schweda-2
In reply to this post by Steven M. Schweda-2
From: "Steven M. Schweda" <sms@antinode-info>

>    On a different point, a closer look at the 64-bit-pointer test
> results shows a problem on Alpha (but not on IA64) somewhere in the "CMS
> => PKCS#7 compatibility test" sequence.  Perhaps some file I/O thing?
> The perl script doesn't seem to handle an explosion here very well.  A
> search for "fail" in the test results transcript won't detect the
> absence of an "ALL TESTS SUCCESSFUL" message, making it easy for a
> casual inspection to miss this kind of failure.

   The cause of this test failure seems to be yet another unexpected
quirk -- not really file I/O.  So far as I can see, all the popular
"app/*.c" main programs (cms.c, genpkey.c, nseq.c, ocsp.c, pkcs12.c,
pkcs8.c, pkey.c, pkeyparam.c, and smime.c) scan their argument list
using this scheme:

[...]
int MAIN(int argc, char **argv)
[...]
        char **args;
[...]
        args = argv + 1;
[...]
                while (*args)   [or similar]
[...]
                        args++;
[...]

That is, they look for a NULL terminator at (past) the end of "argv".

   My apparent problem is that on VMS V8.3 Alpha (at least), with 64-bit
pointers, the "argv" array is not reliably NULL-terminated, and when
it's not, these programs ("cms" in the case at hand) will grab at least
one noise pointer after the end of the actual argument list.  For
example, some diagnostic output looked like this:

 cms.  argc = 10.
[Here, it's looking at the non-option (file-name) arguments...]
 cms(0).  argc_x = 3, *args: >smime-certs/smrsa1.pem<.
 cms(1).  argc_x = 3, *args: >smime-certs/smrsa1.pem<.
 file_ctrl().  ptr: %x00000000004e077a >smime-certs/smrsa1.pem<.
 cms(1).  argc_x = 2, *args: >smime-certs/smrsa2.pem<.
 file_ctrl().  ptr: %x00000000004e0791 >smime-certs/smrsa2.pem<.
 cms(1).  argc_x = 1, *args: >smime-certs/smrsa3.pem<.
 file_ctrl().  ptr: %x00000000004e07a8 >smime-certs/smrsa3.pem<.
 cms(1).  argc_x = 0, *args: >ÀØÜz<.
 file_ctrl().  ptr: %x000000007adcd870 >ÀØÜz<.
Error opening recipient certificate file ÀØÜz
910895052:error:02001002:system library:fopen:no such file or directory:ALP$DKA0
:[UTILITY.SOURCE.OPENSSL.openssl-1_0_0d_64.crypto.bio]bss_file.c;3:403:fopen('ÀØ
Üz','r')
910895052:error:20074002:BIO routines:FILE_CTRL:system lib:ALP$DKA0:[UTILITY.SOU
RCE.OPENSSL.openssl-1_0_0d_64.crypto.bio]bss_file.c;3:405:
unable to load certificate

("argc_x" is my own decrementing "argc"-related variable, and when it
hit zero, it was time to stop.)

   Perhaps "argv" _should_ be NULL-terminated, but that seems not to be
the case here.  (Eventually, the programs are likely to run into enough
zero bytes to stop the scan, but who knows when?)  I have no idea how
much space is actually allocated in this case, so I'd be reluctant to
jam a NULL into argv[ argc], just so that this code will work reliably.

   One possible solution (work-around?) would be not to look for a
NULL terminator in "argv", but instead count the arguments, and quit
when the last argument has been processed.  For example, instead of
bumping that "args" pointer through the "argv" array (looking for that
NULL), one could use an integer argument counter, say, "arrghc", look at
"argv[ arrghc]" instead of "*args", and stop when "arrghc" gets up to
"argc".  Making the changes to all these files would be tedious, but
(not having tried it) I don't see any obvious problems.

   Using the known count ("argc") makes more sense to me than looking
for a terminating (NULL) value, but what do I know?

   Again, even if there's some Standard which demands a NULL after the
(known) end of "argv", I ain't got one, so it wouldn't help.

   Thoughts?

------------------------------------------------------------------------

   Steven M. Schweda               sms@antinode-info
   382 South Warwick Street        (+1) 651-699-9818
   Saint Paul  MN  55105-2547
______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
Development Mailing List                       [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: [openssl.org #2449] [BUG] openssl 1.0.0d warnings during build and ACCVIO on OpenVMS

Steven M. Schweda-2
In reply to this post by Steven M. Schweda-2
   I seem to have 1.0.0d builds working on VMS with 32- or 64-bit
pointers, but I'm still waiting for some guidance on how to make some of
the needed changes:

   1. I need a macro/typedef for an integer with the same size as a
pointer.  On non-VMS systems "intptr_t" might be suitable, but not on
VMS, so it appears to me that some new OpenSSL-specific thing must be
added.  I can get it defined appropriately on VMS.  Elsewhere, I don't
care; "size_t" (as is used now), "intptr_t", whatever.  But I'd be
happier (others, too, I'd guess) if someone else chose the name,
placement, and other details.

   I've identified two places where use of "size_t" caused problems on
VMS with 64-bit pointers, "crypto/bn/bn_mont.c" and
"crypto/bn/bn_nist.c".  It might be wise to scan the code for other
inappropriate uses of "size_t", as I haven't done that.

   2. The argv-using "apps/*.c" programs need reform to use "argc"
instead of looking for a NULL terminator at the end of "argv[]".
(Looking for a NULL makes sense for "envp[]", which is expected to be
NULL-terminated, but not so much for "argv[]", where it causes bad
behavior on VMS Alpha with 64-bit pointers, and where "argc" is
obviously available and suitable.)  I've converted "apps/cms.c" and
"apps/smime.c", because those were causing test failures.  My current
scheme changes code like this:

[...]
int MAIN(int argc, char **argv)
[...]
        char **args;
[...]
        args = argv + 1;
[...]
                while (*args)   [or similar]
[...]
                        args++;
[...]

to code like this:

[...]
int MAIN(int argc, char **argv)
[...]
        int argi;
        char **args;
[...]
        argi = 1;
        args = &argv[argi])[0];
[...]
                while (argi < argc)   [or similar]
[...]
                        args = &argv[++argi];
[...]

The maintainer(s) may prefer some other scheme/details, so I'm reluctant
to fiddle with the other, similar programs in that collection until
someone higher up nods, or complains, or something.  (Also, I wouldn't
bet that any other of those programs gets tested on VMS, so there's
probably some risk in my making the changes to them.)  For the ones I
have changed, I can supply patches or whole replacement files, if anyone
is interested.

   If some decision maker can settle these pending items, then I can
pack this batch of changes into a form which might be more easily
adopted into the main code base.


   Other changes:

   I've added a C macro, OPENSSL_NO_SETVBUF_IONBF, which is defined only
on VMS, and which is used to bracket instances of "setvbuf(..., _IONBF,
...)".  On VMS, this use of setvbuf() is unsupported, and is
incompatible with 64-bit pointers.  Everyone else can ignore this new
macro, and get the same behavior as before.

   I've modified the VMS builders to allow the user to specify a path to
a zlib object library (expecting "zlib.h" to be in the same directory).

   I've also done some tidying in the VMS builders (typography, lame
code reduction, ...).

   I've added a new VMS-specific header file, "crypto/vms_rms.h" to
reduce the code clutter involved with NAM versus NAML (RMS file name)
structures (in "crypto/LPdir_vms.c" and "crypto/dso/dso_vms.c").  I use
essentially similar stuff in many other projects.  You're welcome to it.
(I haven't added a copyright heading.)

------------------------------------------------------------------------

   Steven M. Schweda               sms@antinode-info
   382 South Warwick Street        (+1) 651-699-9818
   Saint Paul  MN  55105-2547
______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
Development Mailing List                       [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

RE: [openssl.org #2449] [BUG] openssl 1.0.0d warnings during build and ACCVIO on OpenVMS

Arpadffy Zoltan
Hello,

> I seem to have 1.0.0d builds working on VMS

Great news, I would be happy to try the build with your changes.
Could you please send some instructions where the patch can be found?
Or it would be the best to post the diffs here, making available for Richard to merge into the code.

Thank you, Steven.

Best regards,
Z

-----Original Message-----
From: Steven M. Schweda [mailto:[hidden email]]
Sent: den 28 februari 2011 07:28
To: [hidden email]
Subject: Re: [openssl.org #2449] [BUG] openssl 1.0.0d warnings during build and ACCVIO on OpenVMS

   I seem to have 1.0.0d builds working on VMS with 32- or 64-bit
pointers, but I'm still waiting for some guidance on how to make some of
the needed changes:

   1. I need a macro/typedef for an integer with the same size as a
pointer.  On non-VMS systems "intptr_t" might be suitable, but not on
VMS, so it appears to me that some new OpenSSL-specific thing must be
added.  I can get it defined appropriately on VMS.  Elsewhere, I don't
care; "size_t" (as is used now), "intptr_t", whatever.  But I'd be
happier (others, too, I'd guess) if someone else chose the name,
placement, and other details.

   I've identified two places where use of "size_t" caused problems on
VMS with 64-bit pointers, "crypto/bn/bn_mont.c" and
"crypto/bn/bn_nist.c".  It might be wise to scan the code for other
inappropriate uses of "size_t", as I haven't done that.

   2. The argv-using "apps/*.c" programs need reform to use "argc"
instead of looking for a NULL terminator at the end of "argv[]".
(Looking for a NULL makes sense for "envp[]", which is expected to be
NULL-terminated, but not so much for "argv[]", where it causes bad
behavior on VMS Alpha with 64-bit pointers, and where "argc" is
obviously available and suitable.)  I've converted "apps/cms.c" and
"apps/smime.c", because those were causing test failures.  My current
scheme changes code like this:

[...]
int MAIN(int argc, char **argv)
[...]
        char **args;
[...]
        args = argv + 1;
[...]
                while (*args)   [or similar]
[...]
                        args++;
[...]

to code like this:

[...]
int MAIN(int argc, char **argv)
[...]
        int argi;
        char **args;
[...]
        argi = 1;
        args = &argv[argi])[0];
[...]
                while (argi < argc)   [or similar]
[...]
                        args = &argv[++argi];
[...]

The maintainer(s) may prefer some other scheme/details, so I'm reluctant
to fiddle with the other, similar programs in that collection until
someone higher up nods, or complains, or something.  (Also, I wouldn't
bet that any other of those programs gets tested on VMS, so there's
probably some risk in my making the changes to them.)  For the ones I
have changed, I can supply patches or whole replacement files, if anyone
is interested.

   If some decision maker can settle these pending items, then I can
pack this batch of changes into a form which might be more easily
adopted into the main code base.


   Other changes:

   I've added a C macro, OPENSSL_NO_SETVBUF_IONBF, which is defined only
on VMS, and which is used to bracket instances of "setvbuf(..., _IONBF,
...)".  On VMS, this use of setvbuf() is unsupported, and is
incompatible with 64-bit pointers.  Everyone else can ignore this new
macro, and get the same behavior as before.

   I've modified the VMS builders to allow the user to specify a path to
a zlib object library (expecting "zlib.h" to be in the same directory).

   I've also done some tidying in the VMS builders (typography, lame
code reduction, ...).

   I've added a new VMS-specific header file, "crypto/vms_rms.h" to
reduce the code clutter involved with NAM versus NAML (RMS file name)
structures (in "crypto/LPdir_vms.c" and "crypto/dso/dso_vms.c").  I use
essentially similar stuff in many other projects.  You're welcome to it.
(I haven't added a copyright heading.)

------------------------------------------------------------------------

   Steven M. Schweda               sms@antinode-info
   382 South Warwick Street        (+1) 651-699-9818
   Saint Paul  MN  55105-2547
______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
Development Mailing List                       [hidden email]
Automated List Manager                           [hidden email]



______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
Development Mailing List                       [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: [openssl.org #2449] [BUG] openssl 1.0.0d warnings during build and ACCVIO on OpenVMS

Steven M. Schweda-2
In reply to this post by Steven M. Schweda-2
From: Arpadffy Zoltan <[hidden email]>

> > I seem to have 1.0.0d builds working on VMS
>
> Great news, I would be happy to try the build with your changes.
> Could you please send some instructions where the patch can be found?
> Or it would be the best to post the diffs here, making available for Richar=
> d to merge into the code.

   As I've explained, the stuff I have is not what I'd want in the
official code, so I haven't produced a real patch kit, but you might try
this:

      http://antinode.info/ftp/openssl/1_0_0d/openssl-1_0_0d_s1.zip

That's a "zip -w" ("store file version numbers") archive which should
include all the new or changed files.  If you have your source code in a
"[.openssl-1_0_0d...]" directory tree, then "unzip -V" ("retain VMS
version numbers") should put these files (with their higher version
numbers) over the originals without destroying anything.  (If your
directory name is different, then you'll need to move/rename or copy the
new files to where you want them.)

   Note that I didn't try to test all of the new explicit-32-bit
descriptor code.  If the normal tests hit it, then it's been tested.  If
not, then it hasn't.  It all looked plausible, and the compiler was
happy, but I don't promise anything.  Those changes should all be
invisible in a 32-bit-pointer build, so any new problems should be
confined to a 64-bit-pointer environment.

   If you try to use the new zlib stuff, then you'll need to have an
appropriate zlib object library available.  The VMS builders in the
standard zlib kit don't have a 64-bit-pointer option, so you'll need to
do some editing there.  I haven't tried using a LIBZ.OLB with the wrong
pointer size.  The OpenSSL builders trust the user, and don't try to
check anything.  I assume that something will explode, but I don't know
what or where.

   Wake me when it catches fire.

------------------------------------------------------------------------

   Steven M. Schweda               sms@antinode-info
   382 South Warwick Street        (+1) 651-699-9818
   Saint Paul  MN  55105-2547
______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
Development Mailing List                       [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: [openssl.org #2449] [BUG] openssl 1.0.0d warnings during build and ACCVIO on OpenVMS

Steven M. Schweda-2
In reply to this post by Steven M. Schweda-2
>       http://antinode.info/ftp/openssl/1_0_0d/openssl-1_0_0d_s1.zip

   A revised, possibly better, replacement file kit ("unzip -V") is now
available:

      http://antinode.info/ftp/openssl/1_0_0d/openssl-1_0_0d_s2.zip

   The builders should now be able to deal with both 32- and 64-bit
pointers in the same source kit directory tree.  That should include
"install.com" and "VMS/mkshared.com".  The object libraries and shared
images should now have HP-like names (with "$" -> "_"):

          32-bit                         64-bit
      SSL_LIBCRYPTO32.OLB            SSL_LIBCRYPTO.OLB
      SSL_LIBSSL32.OLB               SSL_LIBSSL.OLB
      SSL_LIBCRYPTO_SHR32.EXE        SSL_LIBCRYPTO_SHR.EXE
      SSL_LIBSSL_SHR32.EXE           SSL_LIBSSL_SHR.EXE

Among other advantages, this allows one installation directory tree (or
SYS$LIBRARY) to accomodate all of them.  (When run twice (for 32- and
64-bit builds) with one destination directory, "install.com" will copy
the header files twice, but these may be purged, as suggested.)

   Other than comments in the changed files, I haven't updated any
documentation.  (And some of the comments could still use some work.)

   My VAX is currently saturated building perl, so I haven't tried the
latest stuff there.  (But what could go wrong?  (And who'd care if it
did?))  If it finishes the perl job before summer, then I should be able
to check that again.

   For the morbidly curious, my do-everything build procedures (with
zlib support) look like these:

IT $ type [-]btsi_z.com
$ pipe show time ; -
   @ makevms.com ALL "" NODEBUG DECC TCPIP "" -
   utility_root:[source.zlib.zlib-1_2_5l] ; -
   show time
$ pipe show time ; @ [.test]tests.com ; show time
$ pipe show time ; @ [.vms]mkshared.com -
   utility_root:[source.zlib.zlib-1_2_5l] ; -
   show time
$ pipe show time ; @ install.com ""'p1'" ; show time

IT $ type [-]btsi_64z.com
$ pipe show time ; -
   @ makevms.com ALL 64 NODEBUG DECC TCPIP "" -
   utility_root:[source.zlib.zlib-1_2_5l]libz_64 ; -
   show time
$ pipe show time ; @ [.test]tests.com "" 64 ; show time
$ pipe show time ; @ [.vms]mkshared.com 64 -
   utility_root:[source.zlib.zlib-1_2_5l]libz_64 ; -
   show time
$ pipe show time ; @ install.com "''p1'" 64 ; show time

Note the "64" arguments on all the procedures for the 64-bit build.
Omit the "libz" path, if you don't have/want zlib support.

  As one might expect, I'm still awaiting some discussion of the pending
mysteries, so there's still work left to be done.

   Complaints are always welcome.

------------------------------------------------------------------------

   Steven M. Schweda               sms@antinode-info
   382 South Warwick Street        (+1) 651-699-9818
   Saint Paul  MN  55105-2547
______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
Development Mailing List                       [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: [openssl.org #2449] [BUG] openssl 1.0.0d warnings during build and ACCVIO on OpenVMS

Steven M. Schweda-2
In reply to this post by Steven M. Schweda-2
> IT $ type [-]btsi_z.com
> $ pipe show time ; -
>    @ makevms.com ALL "" NODEBUG DECC TCPIP "" -
>    utility_root:[source.zlib.zlib-1_2_5l] ; -
>    show time
> $ pipe show time ; @ [.test]tests.com ; show time
> $ pipe show time ; @ [.vms]mkshared.com -
>    utility_root:[source.zlib.zlib-1_2_5l] ; -
>    show time
> $ pipe show time ; @ install.com ""'p1'" ; show time

   Oops.  Lost a parameter ("") on "mkshared.com":

$ pipe show time ; -
   @ makevms.com ALL "" NODEBUG DECC TCPIP "" -
   utility_root:[source.zlib.zlib-1_2_5l] ; -
   show time
$ pipe show time ; @ [.test]tests.com ; show time
$ pipe show time ; @ [.vms]mkshared.com "" -
   utility_root:[source.zlib.zlib-1_2_5l] ; -
   show time
$ pipe show time ; @ install.com ""'p1'" ; show time

   I suppose that it could benefit from a 'That's not "64"!' message.
Perhaps in the next round.

   SMS.
______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
Development Mailing List                       [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: [openssl.org #2449] [BUG] openssl 1.0.0d warnings during build and ACCVIO on OpenVMS

Steven M. Schweda-2
In reply to this post by Steven M. Schweda-2
>       http://antinode.info/ftp/openssl/1_0_0d/openssl-1_0_0d_s2.zip

   For an "install.com" which works better (perhaps even properly):

      http://antinode.info/ftp/openssl/1_0_0d/openssl-1_0_0d_s2a.zip


> $ pipe show time ; @ install.com ""'p1'" ; show time

   That > ""'p1'" < should have been: > "''p1'" <, of course.

Sigh.  It must all be bug-free, now, though.

   SMS.
______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
Development Mailing List                       [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: [openssl.org #2449] [BUG] openssl 1.0.0d warnings during build and ACCVIO on OpenVMS

Steven M. Schweda-2
In reply to this post by Steven M. Schweda-2
>    2. The argv-using "apps/*.c" programs need reform to use "argc"
> instead of looking for a NULL terminator at the end of "argv[]".
> (Looking for a NULL makes sense for "envp[]", which is expected to be
> NULL-terminated, but not so much for "argv[]", where it causes bad
> behavior on VMS Alpha with 64-bit pointers, and where "argc" is
> obviously available and suitable.)  I've converted "apps/cms.c" and
> "apps/smime.c", because those were causing test failures.  [...]

   I've now also converted the other argv-using "apps/*.c" programs (and
fiddled a little with the previously converted "cms.c" and "smime.c"):

      http://antinode.info/ftp/openssl/1_0_0d/openssl-1_0_0d_s2b.zip

Again, while the compiler seems to be happy, I've done no testing other
than what happens in the usual test stream, so I wouldn't bet on a zero
new-bug count.  Code inspection and/or actual testing would be wise
before adopting these changes.

   SMS.
______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
Development Mailing List                       [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: [openssl.org #2449] [BUG] openssl 1.0.0d warnings during build and ACCVIO on OpenVMS

Steven M. Schweda-2
In reply to this post by Steven M. Schweda-2
   I apparently forgot to include the modified/new start-up procedures
("VMS/openssl_startup.com", "VMS/openssl_undo.com") in previous update
collections:

      http://antinode.info/ftp/openssl/1_0_0d/openssl-1_0_0d_s2c.zip

   SMS.
______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
Development Mailing List                       [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

RE: [openssl.org #2449] [BUG] openssl 1.0.0d warnings during build and ACCVIO on OpenVMS

Arpadffy Zoltan
Steven,

I am sorry, I could not test your changes yet - I'll do it, hopefully, during this week.
...but to be honest, I am more curious what is Richard's opinion about your changes.

Regards,
Z

________________________________________
From: Steven M. Schweda [[hidden email]]
Sent: Sunday, March 06, 2011 2:53 AM
To: [hidden email]
Subject: Re: [openssl.org #2449] [BUG] openssl 1.0.0d warnings during build and ACCVIO on OpenVMS

   I apparently forgot to include the modified/new start-up procedures
("VMS/openssl_startup.com", "VMS/openssl_undo.com") in previous update
collections:

      http://antinode.info/ftp/openssl/1_0_0d/openssl-1_0_0d_s2c.zip

   SMS.
______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
Development Mailing List                       [hidden email]
Automated List Manager                           [hidden email]



______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
Development Mailing List                       [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: [openssl.org #2449] [BUG] openssl 1.0.0d warnings during build and ACCVIO on OpenVMS

Steven M. Schweda-2
In reply to this post by Steven M. Schweda-2
From: Arpadffy Zoltan <[hidden email]>

> I am sorry, I could not test your changes yet - I'll do it, hopefully, duri=
> ng this week.
> ...but to be honest, I am more curious what is Richard's opinion about your=
>  changes.

   I'm curious about anyone's opinion, but you know as much as I do.
Please let me know if you see any problems.  (By the way, did you have
any actual need for the 64-bit-pointer stuff, or was that work done
mostly for fun?)

   Today's change set includes different names for the various
"[...]install.com" procedures, and better clean-up of the logical names
which each one defines.

      http://antinode.info/ftp/openssl/1_0_0d/openssl-1_0_0d_s2d.zip

Ideally, with these changes, no one should find any WRK_* logical names
left defined after running any of these procedures.  Also, with the new,
unique procedure names, the (new) "@@@ whatever.com" messages should
provide some useful information.  I think that I've exhausted my to-do
list, so, unless I find some new problems, I may be quiet for a while.

   My VAX is still working on a Perl build (which I was hoping to use
for OpenSSL testing), so I still haven't tried the latest version of
everything there.  (But what could go wrong?)

   I tried building the original and modified 1.0.0d code on an HP-UX
(11.31 ia64) system, and I didn't see any significant differences in the
test results.  So, either those command-line argc+argv changes were all
good, or else the tests are insufficient to show the new problems.

   Complaints are always welcome, of course.

   SMS.
______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
Development Mailing List                       [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: [openssl.org #2449] [BUG] openssl 1.0.0d warnings during build and ACCVIO on OpenVMS

Steven M. Schweda-2
In reply to this post by Steven M. Schweda-2
      http://antinode.info/ftp/openssl/1_0_0d/openssl-1_0_0d_s2e.zip

   Bad "crypto/crypto-lib.com".  (Minor error on VAX.)  Trust no one, I
always say.  (Especially me.)  It'll be a while before the VAX build
gets finished, so stay tuned.

   There seems to be some evidence that argv[] _should_ be
NULL-terminated, which would seem to shift the the blame for the
misbehavior of the "apps/" programs (on Alpha with 64-bit pointers) from
programs themselves to HP's C compiler team.

http://forums.itrc.hp.com/service/forums/questionanswer.do?threadId=1472203

   SMS.
______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
Development Mailing List                       [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: [openssl.org #2449] [BUG] openssl 1.0.0d warnings during build and ACCVIO on OpenVMS

Dr. Stephen Henson
On Tue, Mar 08, 2011, Steven M. Schweda wrote:

>    There seems to be some evidence that argv[] _should_ be
> NULL-terminated, which would seem to shift the the blame for the
> misbehavior of the "apps/" programs (on Alpha with 64-bit pointers) from
> programs themselves to HP's C compiler team.
>

Yes I'm pretty sure I read it somewhere. A simple workaround would be to copy
argv into a malloced copy that is properly NULL terminated if you haven't done
that already. As all the apps go through main() code in apps/openssl.c you
should only need to change one place.

Steve.
-
Dr Stephen N. Henson. OpenSSL project core developer.
Commercial tech support now available see: http://www.openssl.org
______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
Development Mailing List                       [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: [openssl.org #2449] [BUG] openssl 1.0.0d warnings during build and ACCVIO on OpenVMS

Steven M. Schweda-2
In reply to this post by Steven M. Schweda-2
From: "Dr. Stephen Henson" <[hidden email]>

> [...] A simple workaround would be to copy
> argv into a malloced copy that is properly NULL terminated if you haven't done
> that already. As all the apps go through main() code in apps/openssl.c you
> should only need to change one place.

   That probably makes (much) more sense than fiddling with all the
consumers.  With HP's newly restrictive patch (un)availability policy,
there's not much chance of being able to get away with doing nothing,
even if it is all their fault.

   While you're here, please don't forget that I'm still waiting for
wisdom on what to add where to get around the "crypto/bn" problems
caused by sizeof size_t != sizeof void*.

------------------------------------------------------------------------

   Steven M. Schweda               sms@antinode-info
   382 South Warwick Street        (+1) 651-699-9818
   Saint Paul  MN  55105-2547
______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
Development Mailing List                       [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: [openssl.org #2449] [BUG] openssl 1.0.0d warnings during build and ACCVIO on OpenVMS

Steven M. Schweda-2
In reply to this post by Steven M. Schweda-2
From: "Dr. Stephen Henson" <[hidden email]>

> [...] A simple workaround would be to copy
> argv into a malloced copy that is properly NULL terminated if you haven't done
> that already. As all the apps go through main() code in apps/openssl.c you
> should only need to change one place.

      http://antinode.info/ftp/openssl/1_0_0d/openssl-1_0_0d_s3.zip

Unless I've missed something, this should be a complete set of
replacement files to be laid over ("unzip -V", on VMS) a clean 1.0.0d
kit.  The new "apps/openssl.c" obviates most of the other replacement
"apps/*.c" files in previous kits.  It seems to work here, but
complaints are always welcome.

   Other than the still-pending mystery of how best to handle the
pointer-sized replacement for "size_t", these changes may now be ready
to be ignored.

   SMS.
______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
Development Mailing List                       [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: [openssl.org #2449] [BUG] openssl 1.0.0d warnings during build and ACCVIO on OpenVMS

Steven M. Schweda-2
In reply to this post by Steven M. Schweda-2
>       http://antinode.info/ftp/openssl/1_0_0d/openssl-1_0_0d_s3.zip
>
> Unless I've missed something, [...]

   Naturally, that was too good to be true.  I now have enough Perl on
my VAX system to run into a test failure:

[...]
---> TEST_CMS
CMS consistency test
CMS => PKCS#7 compatibility tests
signed content DER format, RSA key: OK
signed detached content DER format, RSA key: OK
signed content test streaming BER format, RSA: OK
signed content DER format, DSA key: OK
signed detached content DER format, DSA key: OK
signed detached content DER format, add RSA signer: OK
signed content test streaming BER format, DSA key: OK
%DCL-W-TKNOVF, command element is too long - shorten
%DCL-W-TKNOVF, command element is too long - shorten
 \mcr SYS$DISK:[-.VAX.EXE.APPS]openssl.exe cms -sign -in smcont.txt
 -outform "DER" -nodetach -signer smime-certs/smrsa1.pem -signer
 smime-certs/smrsa2.pem -signer smime-certs/smdsa1.pem -signer
 smime-certs/smdsa2.pem -stream -out test.cms 2> cms.err > cms
 \mcr SYS$DISK:[-.VAX.EXE.APPS]openssl.exe cms -sign -in smcont.txt
 -outform "DER" -nodetach -signer smime-certs/smrsa1.pem -signer
 smime-certs/smrsa2.pem -signer smime-certs/smdsa1.pem -signer
 smime-certs/smdsa2.pem -stream -out test.cms 2> cms.err > cms
signed content test streaming BER format, 2 DSA and 2 RSA keys: generation err
%SYSTEM-F-ABORT, abort
[...]


   If you don't work with it regularly, it's easy to forget how lame VMS
VAX V7.3 can be.

   A matched pair of changes to "test/cms-test.pl" and "test/tests.com"
sucks about twenty characters out of those command lines, which seems to
be good enough (for now).  Instead of making Perl expand a DCL symbol
("EXE_DIR" -> "SYS$DISK:[-.VAX.EXE.APPS]", in this case) to find
"openssl.exe", it's shorter (and cleaner-looking, and less work for
Perl) to define a logical name ("OSSLX"), and then use that.  And a
".exe" was redundant, too.  (Every byte counts, I always say.)

ALP $ gdiff tests.com;5 tests.com;
345,346c345,346
< $     ! The following makes perl include the DCL symbol table in the env.
< $     define/user perl_env_tables clisym_local,lnm$file_dev,ctrl_env
---
> $     ! Define the logical name used to find openssl.exe in the perl script.
> $     define /user_mode osslx 'exe_dir'

ALP $ gdiff cms-test.pl_orig cms-test.pl
59,60c59,60
< if ( $^O eq "VMS" && -f "$ENV{EXE_DIR}openssl.exe" ) {
<     $ossl_path = "pipe mcr $ENV{EXE_DIR}openssl.exe";
---
> if ( $^O eq "VMS" && -f "OSSLX:openssl.exe" ) {
>     $ossl_path = "pipe mcr OSSLX:openssl";


That should be turning:
      mcr SYS$DISK:[-.VAX.EXE.APPS]openssl.exe
into:
      mcr OSSLX:openssl

With any luck, we won't need to start abbreviating the file names soon.
"OSSLX" could be even shorter, but I thought that I'd leave a few fat
cells for the next fellow to trim.  Replacement files:

      http://antinode.info/ftp/openssl/1_0_0d/openssl-1_0_0d_s3a.zip

What could go wrong now?

   SMS.
______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
Development Mailing List                       [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: [openssl.org #2449] [BUG] openssl 1.0.0d warnings during build and ACCVIO on OpenVMS

Steven M. Schweda-2
In reply to this post by Steven M. Schweda-2
   Today's project: Restore command-line case preservation for
"openssl.exe" (on modern non-VAX systems, with Parse Style: Extended).
New: "apps/vms_decc_init.c".  Modified: "apps/makeapps.com", plus a
bunch of "test/*.com" files which depended on DCL+C command-line
downcasing (or were in danger of doing so).  (Except for a new
accomodation for 64-bit pointers, this is probably all old stuff which
got ignored in the past.)

   Replacement files:

      http://antinode.info/ftp/openssl/1_0_0d/openssl-1_0_0d_s3b.zip

   Bonus features:

   "apps/CA.com" now has a prompt which looks like a prompt instead of a
casual remark.

   "test/tests.com" now directs SYS$ERROR and SYS$OUTPUT in many places
to files instead of to the null device (NLA0:), allowing some diagnosis
after a test failure.  Addition of (what I hope are) well-placed "set
noon" and "set on" commands should avoid a variety of problems when a
test fails.  (I got hit by a definition of SYS$ERROR which persisted
after a premature exit from "test/tests.com", and messed up the next
build.)

   "test/clean_test.com" (new) deletes miscellaneous test results files.


   Question of the day:  Is "apps/openssl-vms.cnf" actually good for
anything?  Some of the tests use various "test/*.cnf" files, and these
seem to have no VMS-specific variants.

   SMS.
______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
Development Mailing List                       [hidden email]
Automated List Manager                           [hidden email]
123