Linux linking issues moving from 1.0.2t to 1.1.1c

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

Linux linking issues moving from 1.0.2t to 1.1.1c

Dan Heinz

Please bear with me as I am a Windows developer, and not too adept with Linux.  

 

Our library has been using the OpenSSL 1.0.2x branch, and we are moving to 1.1.1c.  I have the Windows build of our libraries working, and now I’ve moved to Linux.

Our library is built as a shared library as well as static library and both use the static OpenSSL library (we archive the OpenSSL static library with our static library).  

When I built our shared library I had some linker errors:
e/Debug32/mylib.so: undefined reference to `dlopen'

../../MyLib/Debug32/libMyLib.so: undefined reference to `pthread_rwlock_rdlock'

../../MyLib/Debug32/libMyLib.so: undefined reference to `pthread_rwlock_init'

../../MyLib/Debug32/libMyLib.so: undefined reference to `pthread_getspecific'

../../MyLib/Debug32/libMyLib.so: undefined reference to `pthread_rwlock_destroy'

../../MyLib/Debug32/libMyLib.so: undefined reference to `dlclose'

../../MyLib/Debug32/libMyLib.so: undefined reference to `pthread_key_create'

../../MyLib/Debug32/libMyLib.so: undefined reference to `pthread_rwlock_wrlock'

../../MyLib/Debug32/libMyLib.so: undefined reference to `pthread_once'

../../MyLib/Debug32/libMyLib.so: undefined reference to `pthread_atfork'

../../MyLib/Debug32/libMyLib.so: undefined reference to `pthread_setspecific'

../../MyLib/Debug32/libMyLib.so: undefined reference to `dlerror'

../../MyLib/Debug32/libMyLib.so: undefined reference to `pthread_rwlock_unlock'

../../MyLib/Debug32/libMyLib.so: undefined reference to `dlsym'

../../MyLib/Debug32/libMyLib.so: undefined reference to `pthread_key_delete'

../../MyLib/Debug32/libMyLib.so: undefined reference to `dladdr'

 

And the Configuration line from our build script:
./Configure linux-x86_64 --prefix=$install_path/openssl_64 -DPURIFY -DOPENSSL_NO_COMP no-shared no-dso no-sse2 no-idea no-mdc2 no-rc5 no-ssl3 no-zlib no-comp no-afalgeng no-pinshared

 

As our library does not use multiple threads, I can build the OpenSSL library with the no-threads configuration parameter to remove the pthread references.  Is there anything I should be aware of when doing so? 

 

For the dlopen, dlclose, dlerror,dlsym, and dladdr references, it seems I need to link libld using -ldl.  Isn’t the no-dso configuration option supposed to remove the need for this library?  Is there a way to do so with 1.1.1c?  While I can link this in our shared library, our static library will require anyone using our library to link libld.  I’d like to avoid this if possible, and it seems we could with the 1.0.2 branch. 

Am I missing something here?

 

Thanks in advance!

 

 

 

 

Reply | Threaded
Open this post in threaded view
|

Re: Linux linking issues moving from 1.0.2t to 1.1.1c

Richard Levitte - VMS Whacker-2
The no-dso is silently not valid in 1.1.1c.  That option didn't work
right, so it was unusable in practice anyway.  However, someone
recently fixed that up, unfortunately after the last 1.1.1 release.
The specific patch may be possible to find on github (unless that
branch has been deleted), otherwise you will have to cherry-pick the
appropriate commit.

Github PR: https://github.com/openssl/openssl/pull/9889
Commit ID: 8dcd57461972dceaaf014b71d173d0a8758e7054

Cheers,
Richard

On Mon, 07 Oct 2019 17:19:26 +0200,
Dan Heinz wrote:

>
>
> Please bear with me as I am a Windows developer, and not too adept with Linux.  
>
> Our library has been using the OpenSSL 1.0.2x branch, and we are moving to 1.1.1c.  I have the
> Windows build of our libraries working, and now I’ve moved to Linux.
>
> Our library is built as a shared library as well as static library and both use the static OpenSSL
> library (we archive the OpenSSL static library with our static library).  
>
> When I built our shared library I had some linker errors:
> e/Debug32/mylib.so: undefined reference to `dlopen'
>
> ../../MyLib/Debug32/libMyLib.so: undefined reference to `pthread_rwlock_rdlock'
>
> ../../MyLib/Debug32/libMyLib.so: undefined reference to `pthread_rwlock_init'
>
> ../../MyLib/Debug32/libMyLib.so: undefined reference to `pthread_getspecific'
>
> ../../MyLib/Debug32/libMyLib.so: undefined reference to `pthread_rwlock_destroy'
>
> ../../MyLib/Debug32/libMyLib.so: undefined reference to `dlclose'
>
> ../../MyLib/Debug32/libMyLib.so: undefined reference to `pthread_key_create'
>
> ../../MyLib/Debug32/libMyLib.so: undefined reference to `pthread_rwlock_wrlock'
>
> ../../MyLib/Debug32/libMyLib.so: undefined reference to `pthread_once'
>
> ../../MyLib/Debug32/libMyLib.so: undefined reference to `pthread_atfork'
>
> ../../MyLib/Debug32/libMyLib.so: undefined reference to `pthread_setspecific'
>
> ../../MyLib/Debug32/libMyLib.so: undefined reference to `dlerror'
>
> ../../MyLib/Debug32/libMyLib.so: undefined reference to `pthread_rwlock_unlock'
>
> ../../MyLib/Debug32/libMyLib.so: undefined reference to `dlsym'
>
> ../../MyLib/Debug32/libMyLib.so: undefined reference to `pthread_key_delete'
>
> ../../MyLib/Debug32/libMyLib.so: undefined reference to `dladdr'
>
> And the Configuration line from our build script:
> ./Configure linux-x86_64 --prefix=$install_path/openssl_64 -DPURIFY -DOPENSSL_NO_COMP no-shared
> no-dso no-sse2 no-idea no-mdc2 no-rc5 no-ssl3 no-zlib no-comp no-afalgeng no-pinshared
>
> As our library does not use multiple threads, I can build the OpenSSL library with the no-threads
> configuration parameter to remove the pthread references.  Is there anything I should be aware of
> when doing so?
>
> For the dlopen, dlclose, dlerror,dlsym, and dladdr references, it seems I need to link libld using
> -ldl.  Isn’t the no-dso configuration option supposed to remove the need for this library?  Is
> there a way to do so with 1.1.1c?  While I can link this in our shared library, our static library
> will require anyone using our library to link libld.  I’d like to avoid this if possible, and it
> seems we could with the 1.0.2 branch.
>
> Am I missing something here?
>
> Thanks in advance!
>
>
--
Richard Levitte         [hidden email]
OpenSSL Project         http://www.openssl.org/~levitte/
Reply | Threaded
Open this post in threaded view
|

RE: Linux linking issues moving from 1.0.2t to 1.1.1c

Dan Heinz
>The no-dso is silently not valid in 1.1.1c.  That option didn't work right, so it was unusable in practice anyway.  However, someone recently fixed that up, unfortunately after the last 1.1.1 release.
>The specific patch may be possible to find on github (unless that branch has been deleted), otherwise you will have to cherry-pick the appropriate commit.

>Github PR: https://github.com/openssl/openssl/pull/9889
>Commit ID: 8dcd57461972dceaaf014b71d173d0a8758e7054

>Cheers,
>Richard

Thanks for the info.  I did some more digging and you had actually posted a workaround in this thread:
https://github.com/openssl/openssl/issues/9036

I thought I would try it out.
I used your example and created my own config target in file named no_dos.conf.
(
    'my-linux-x86_64' => {
        inherit_from    => [ 'linux-x86_64' ],
        dso_scheme     => undef,
    }
);

./Configure --config ../no_dso.conf my-linux-x86_64  -m32 --prefix=$install_path/openssl_32 -DPURIFY -DOPENSSL_NO_COMP no-asm no-shared no-dso no-sse2 no-idea no-mdc2 no-rc5 no-ssl3 no-zlib no-comp no-afalgeng no-pinshared

But I'm getting this error from the script when Configure is run:
target already defined - ../no_dso.conf (offending arg: my-linux-x86_64)

What did I miss?
Reply | Threaded
Open this post in threaded view
|

Re: Linux linking issues moving from 1.0.2t to 1.1.1c

Richard Levitte - VMS Whacker-2
On Mon, 07 Oct 2019 21:55:50 +0200,
Dan Heinz wrote:

>
> >The no-dso is silently not valid in 1.1.1c.  That option didn't work right, so it was unusable in practice anyway.  However, someone recently fixed that up, unfortunately after the last 1.1.1 release.
> >The specific patch may be possible to find on github (unless that branch has been deleted), otherwise you will have to cherry-pick the appropriate commit.
>
> >Github PR: https://github.com/openssl/openssl/pull/9889
> >Commit ID: 8dcd57461972dceaaf014b71d173d0a8758e7054
>
> >Cheers,
> >Richard
>
> Thanks for the info.  I did some more digging and you had actually posted a workaround in this thread:
> https://github.com/openssl/openssl/issues/9036
>
> I thought I would try it out.
> I used your example and created my own config target in file named no_dos.conf.
> (
>     'my-linux-x86_64' => {
>         inherit_from    => [ 'linux-x86_64' ],
>         dso_scheme     => undef,
>     }
> );
>
> ./Configure --config ../no_dso.conf my-linux-x86_64  -m32 --prefix=$install_path/openssl_32 -DPURIFY -DOPENSSL_NO_COMP no-asm no-shared no-dso no-sse2 no-idea no-mdc2 no-rc5 no-ssl3 no-zlib no-comp no-afalgeng no-pinshared
>
> But I'm getting this error from the script when Configure is run:
> target already defined - ../no_dso.conf (offending arg: my-linux-x86_64)
>
> What did I miss?

You don't happen to have edited some Configurations/*.conf and added
that name already?  I'm otherwise unsure for the moment.

--
Richard Levitte         [hidden email]
OpenSSL Project         http://www.openssl.org/~levitte/
Reply | Threaded
Open this post in threaded view
|

Re: Linux linking issues moving from 1.0.2t to 1.1.1c

Richard Levitte - VMS Whacker-2
On Tue, 08 Oct 2019 14:59:23 +0200,
Richard Levitte wrote:

>
> On Mon, 07 Oct 2019 21:55:50 +0200,
> Dan Heinz wrote:
> >
> > >The no-dso is silently not valid in 1.1.1c.  That option didn't work right, so it was unusable in practice anyway.  However, someone recently fixed that up, unfortunately after the last 1.1.1 release.
> > >The specific patch may be possible to find on github (unless that branch has been deleted), otherwise you will have to cherry-pick the appropriate commit.
> >
> > >Github PR: https://github.com/openssl/openssl/pull/9889
> > >Commit ID: 8dcd57461972dceaaf014b71d173d0a8758e7054
> >
> > >Cheers,
> > >Richard
> >
> > Thanks for the info.  I did some more digging and you had actually posted a workaround in this thread:
> > https://github.com/openssl/openssl/issues/9036
> >
> > I thought I would try it out.
> > I used your example and created my own config target in file named no_dos.conf.
> > (
> >     'my-linux-x86_64' => {
> >         inherit_from    => [ 'linux-x86_64' ],
> >         dso_scheme     => undef,
> >     }
> > );
> >
> > ./Configure --config ../no_dso.conf my-linux-x86_64  -m32 --prefix=$install_path/openssl_32 -DPURIFY -DOPENSSL_NO_COMP no-asm no-shared no-dso no-sse2 no-idea no-mdc2 no-rc5 no-ssl3 no-zlib no-comp no-afalgeng no-pinshared
> >
> > But I'm getting this error from the script when Configure is run:
> > target already defined - ../no_dso.conf (offending arg: my-linux-x86_64)
> >
> > What did I miss?
>
> You don't happen to have edited some Configurations/*.conf and added
> that name already?  I'm otherwise unsure for the moment.

Figured it out.  Configure requires that '--config' be joined to its
value with an equal sign.  In other words, this slight variation
works:

./Configure --config=../no_dso.conf my-linux-x86_64  -m32 --prefix=$install_path/openssl_32 -DPURIFY -DOPENSSL_NO_COMP no-asm no-shared no-dso no-sse2 no-idea no-mdc2 no-rc5 no-ssl3 no-zlib no-comp no-afalgeng no-pinshared

Cheers,
Richard ( is just a messenger in this case, yeah? )

--
Richard Levitte         [hidden email]
OpenSSL Project         http://www.openssl.org/~levitte/
Reply | Threaded
Open this post in threaded view
|

RE: Linux linking issues moving from 1.0.2t to 1.1.1c

Dan Heinz
> > > >The no-dso is silently not valid in 1.1.1c.  That option didn't work right, so it was unusable in practice anyway.  However, someone recently fixed that up, unfortunately after the last 1.1.1 release.
> > > >The specific patch may be possible to find on github (unless that branch has been deleted), otherwise you will have to cherry-pick the appropriate commit.
> > >
> > > >Github PR: https://github.com/openssl/openssl/pull/9889
> > > >Commit ID: 8dcd57461972dceaaf014b71d173d0a8758e7054
> > >
> > > >Cheers,
> > > >Richard
> > >
> > > Thanks for the info.  I did some more digging and you had actually posted a workaround in this thread:
> > > https://github.com/openssl/openssl/issues/9036
> > >
> > > I thought I would try it out.
> > > I used your example and created my own config target in file named no_dos.conf.
> > > (
> > >     'my-linux-x86_64' => {
> > >         inherit_from    => [ 'linux-x86_64' ],
> > >         dso_scheme     => undef,
> > >     }
> > > );
> > >
> > > ./Configure --config ../no_dso.conf my-linux-x86_64  -m32
> > > --prefix=$install_path/openssl_32 -DPURIFY -DOPENSSL_NO_COMP no-asm
> > > no-shared no-dso no-sse2 no-idea no-mdc2 no-rc5 no-ssl3 no-zlib
> > > no-comp no-afalgeng no-pinshared
> > >
> > > But I'm getting this error from the script when Configure is run:
> > > target already defined - ../no_dso.conf (offending arg:
> > > my-linux-x86_64)
> > >
> > > What did I miss?
> >
> > You don't happen to have edited some Configurations/*.conf and added
> > that name already?  I'm otherwise unsure for the moment.
>
> Figured it out.  Configure requires that '--config' be joined to its value with an equal sign.  In other words, this slight variation
> works:
>
>./Configure --config=../no_dso.conf my-linux-x86_64  -m32 --prefix=$install_path/openssl_32 -DPURIFY -DOPENSSL_NO_COMP no-asm no-shared no-dso no-sse2 no-idea no-mdc2 no-rc5 no-ssl3 no-zlib no-comp no-afalgeng no-pinshared
>

Thank you!  That seems to have fixed things up nicely, and I no longer need to link libdl when linking my library.  

Another question is why I now need to link pthreads when I did not in the 1.0.2 version?  I've added no-threads to the configuration, but I'm curious why I didn't need to previously link it.  And I'd prefer not to change too many configuration parameters if possible.

Reply | Threaded
Open this post in threaded view
|

Re: Linux linking issues moving from 1.0.2t to 1.1.1c

Richard Levitte - VMS Whacker-2
On Tue, 08 Oct 2019 18:53:37 +0200,
Dan Heinz wrote:
>
> Another question is why I now need to link pthreads when I did not
> in the 1.0.2 version?  I've added no-threads to the configuration,
> but I'm curious why I didn't need to previously link it.  And I'd
> prefer not to change too many configuration parameters if possible.

We have radically changed how threads-related things are handled.
Previous to 1.1.0, we required the application to provide us with the
locking functionality in form of callbacks.  Since 1.1.0, these
matters are entirely internal, so libcrypto requires the presence of
the platform specific thread implementation of our choosing, which is
pthread on everything but Windows, where we use Windows thread calls.

So if you really really really want to have nothing to do with threads
in your applications, the 'no-threads' configuration option is the
right move.  However, if your applications do deal with threads,
directly or indirectly, disabling threads in libcrypto is of course a
bad move.

Cheers,
Richard

--
Richard Levitte         [hidden email]
OpenSSL Project         http://www.openssl.org/~levitte/