SSL_library_init - missing 36 bytes after cleanup

classic Classic list List threaded Threaded
15 messages Options
Reply | Threaded
Open this post in threaded view
|

SSL_library_init - missing 36 bytes after cleanup

Steffen Fiksdal
Hi!

I use valgrind to check my code, and I can't seem to be able to free up 36
bytes.

SSL_library_init() allocates 36 bytes that I am not able to free using the
regular cleanup functions.

The details:
SSL_library_init calls SSL_COMP_get_compression_methods() if
OPENSSL_NO_COMP is defined.

The SSL_COMP_get_compression_methods() calls load_builtin_compression()

The load_builtin_compression allocates sizeof(SSL_COMP) dynamically.

These are the 36 bytes that are taken away from me :-)
Is this a leak, or are they cleaned up in any sense by any of the cleanup
functions ?


Best Regards
Steffen Fiksdal
______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
User Support Mailing List                    [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

RE: SSL_library_init - missing 36 bytes after cleanup

JoelKatz

> I use valgrind to check my code, and I can't seem to be able to
> free up 36
> bytes.

        So what?

> SSL_library_init() allocates 36 bytes that I am not able to free
> using the
> regular cleanup functions.

        Correct.

> The details:
> SSL_library_init calls SSL_COMP_get_compression_methods() if
> OPENSSL_NO_COMP is defined.
>
> The SSL_COMP_get_compression_methods() calls load_builtin_compression()
>
> The load_builtin_compression allocates sizeof(SSL_COMP) dynamically.
>
> These are the 36 bytes that are taken away from me :-)

        Yep.

> Is this a leak,

        No.

> or are they cleaned up in any sense by any of the cleanup
> functions ?

        No.

        These are 36 bytes that are consumed and used by the library. They are not
leaked but they cannot be freed.

        DS


______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
User Support Mailing List                    [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

RE: SSL_library_init - missing 36 bytes after cleanup

Steven Reddie
David,

If 36 bytes are being dynamically allocated and not being freed how is it
not a leak?

Steven

-----Original Message-----
From: [hidden email]
[mailto:[hidden email]] On Behalf Of David Schwartz
Sent: Wednesday, 16 November 2005 10:09 AM
To: [hidden email]
Subject: RE: SSL_library_init - missing 36 bytes after cleanup


> I use valgrind to check my code, and I can't seem to be able to free
> up 36 bytes.

        So what?

> SSL_library_init() allocates 36 bytes that I am not able to free using
> the regular cleanup functions.

        Correct.

> The details:
> SSL_library_init calls SSL_COMP_get_compression_methods() if
> OPENSSL_NO_COMP is defined.
>
> The SSL_COMP_get_compression_methods() calls
> load_builtin_compression()
>
> The load_builtin_compression allocates sizeof(SSL_COMP) dynamically.
>
> These are the 36 bytes that are taken away from me :-)

        Yep.

> Is this a leak,

        No.

> or are they cleaned up in any sense by any of the cleanup functions ?

        No.

        These are 36 bytes that are consumed and used by the library. They
are not leaked but they cannot be freed.

        DS


______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
User Support Mailing List                    [hidden email]
Automated List Manager                           [hidden email]

______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
User Support Mailing List                    [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: SSL_library_init - missing 36 bytes after cleanup

Joshua Juran
On Nov 15, 2005, at 7:29 PM, Steven Reddie wrote:

> David,
>
> If 36 bytes are being dynamically allocated and not being freed how is
> it
> not a leak?
>
> Steven

Because it only happens once.

Imagine that when you shut off a faucet, water drips out for the next
ten seconds and then stops.  That's not a leak, and it's not a problem.
  What would be (both a problem and a leak) is water continually
dripping at a regular frequency.

The 36 bytes is not considered a leak because it's acquired only once
as a part of initialization, not proportionally to the use of your
application.  And it does get released -- when the process exits. :-)

If it's a problem, maybe you should invest in a RAM upgrade...  :-D

Josh

> -----Original Message-----
> [mailto:[hidden email]] On Behalf Of David Schwartz
> Sent: Wednesday, 16 November 2005 10:09 AM
>
> These are 36 bytes that are consumed and used by the library. They
> are not leaked but they cannot be freed.

______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
User Support Mailing List                    [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

RE: SSL_library_init - missing 36 bytes after cleanup

Steven Reddie
I understand about one-off leaks, but we're talking about a dynamically
loadable library when we're talking about OpenSSL.

What would happen if an application did something like this:

        for (int i=0; i<1000; i++)
        {
                hSSL = LoadLibrary("libssl.so")
                fn = GetSymbol(hSSL, SSL_COMP_get_compression_methods);
                fn();
                ReleaseLibrary(hSSL);
        }

Is it still a one-off leak?

Unfreed memory is not a problem in an application, but it's a whole
different story in a library.

Another example of such problems in libraries is atexit().  atexit() works
great in an application, but use it in libssl.so when used in the code above
and you'll most likely get a hard crash when the application terminates
because the handler is still registered with the C runtime library, but the
code has been unloaded.

Steven

-----Original Message-----
From: [hidden email]
[mailto:[hidden email]] On Behalf Of Joshua Juran
Sent: Wednesday, 16 November 2005 12:38 PM
To: [hidden email]
Subject: Re: SSL_library_init - missing 36 bytes after cleanup

On Nov 15, 2005, at 7:29 PM, Steven Reddie wrote:

> David,
>
> If 36 bytes are being dynamically allocated and not being freed how is
> it not a leak?
>
> Steven

Because it only happens once.

Imagine that when you shut off a faucet, water drips out for the next ten
seconds and then stops.  That's not a leak, and it's not a problem.
  What would be (both a problem and a leak) is water continually dripping at
a regular frequency.

The 36 bytes is not considered a leak because it's acquired only once as a
part of initialization, not proportionally to the use of your application.
And it does get released -- when the process exits. :-)

If it's a problem, maybe you should invest in a RAM upgrade...  :-D

Josh

> -----Original Message-----
> [mailto:[hidden email]] On Behalf Of David Schwartz
> Sent: Wednesday, 16 November 2005 10:09 AM
>
> These are 36 bytes that are consumed and used by the library. They
> are not leaked but they cannot be freed.

______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
User Support Mailing List                    [hidden email]
Automated List Manager                           [hidden email]

______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
User Support Mailing List                    [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: SSL_library_init - missing 36 bytes after cleanup

Joshua Juran
On Nov 15, 2005, at 9:00 PM, Steven Reddie wrote:

> I understand about one-off leaks, but we're talking about a dynamically
> loadable library when we're talking about OpenSSL.
>
> What would happen if an application did something like this:
>
> for (int i=0; i<1000; i++)
> {
> hSSL = LoadLibrary("libssl.so")
> fn = GetSymbol(hSSL, SSL_COMP_get_compression_methods);
> fn();
> ReleaseLibrary(hSSL);
> }
>
> Is it still a one-off leak?
>
> Unfreed memory is not a problem in an application, but it's a whole
> different story in a library.

I agree that the rules are different for shared libraries, and if ssl
hasn't cleaned up after itself completely when the call to
ReleaseLibrary() returns, it's a leak.

My experience with dynamically linked libraries is limited to Mac OS'
Code Fragment Manager, where I've avoided some issues by linking
plugins against the standard library statically, so each one gets its
own malloc pool.  I also generally work in C++, where I can use
statically allocated objects with constructors and destructors, such as
smart pointers.

> Another example of such problems in libraries is atexit().  atexit()
> works
> great in an application, but use it in libssl.so when used in the code
> above
> and you'll most likely get a hard crash when the application terminates
> because the handler is still registered with the C runtime library,
> but the
> code has been unloaded.

If OpenSSL claims to be usable when dynamically loaded, then it's a bug
in OpenSSL.  Otherwise, it's a bug in the code that's loading it.

Josh

> -----Original Message-----
> From: [hidden email]
> [mailto:[hidden email]] On Behalf Of Joshua Juran
> Sent: Wednesday, 16 November 2005 12:38 PM
> To: [hidden email]
> Subject: Re: SSL_library_init - missing 36 bytes after cleanup
>
> On Nov 15, 2005, at 7:29 PM, Steven Reddie wrote:
>
>> David,
>>
>> If 36 bytes are being dynamically allocated and not being freed how is
>> it not a leak?
>>
>> Steven
>
> Because it only happens once.
>
> Imagine that when you shut off a faucet, water drips out for the next
> ten
> seconds and then stops.  That's not a leak, and it's not a problem.
>   What would be (both a problem and a leak) is water continually
> dripping at
> a regular frequency.
>
> The 36 bytes is not considered a leak because it's acquired only once
> as a
> part of initialization, not proportionally to the use of your
> application.
> And it does get released -- when the process exits. :-)
>
> If it's a problem, maybe you should invest in a RAM upgrade...  :-D

______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
User Support Mailing List                    [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

RE: SSL_library_init - missing 36 bytes after cleanup

Steven Reddie
That is my point, that "one-off" leaks in DLLs are not one-off leaks at all.
While statically linking against the C runtime library certainly works
around the issue (at least on Windows, and as you say on Mac) it is a poor
resolution for a memory leak as it sacrifices the benefits of using a DLL.
Since OpenSSL doesn't statically link against the C runtime on Windows it's
a further indication that the issue raised by Steffen is in fact the leak
that he presumed.

Dismissing leaks as one-off's is a pet peeve of mine.  The notion of one-off
leaks in an executable is arguably passable, but becomes a plain old memory
leak just like any other when packaged as a library.

Regards,

Steven

-----Original Message-----
From: [hidden email]
[mailto:[hidden email]] On Behalf Of Joshua Juran
Sent: Wednesday, 16 November 2005 1:44 PM
To: [hidden email]
Subject: Re: SSL_library_init - missing 36 bytes after cleanup

On Nov 15, 2005, at 9:00 PM, Steven Reddie wrote:

> I understand about one-off leaks, but we're talking about a
> dynamically loadable library when we're talking about OpenSSL.
>
> What would happen if an application did something like this:
>
> for (int i=0; i<1000; i++)
> {
> hSSL = LoadLibrary("libssl.so")
> fn = GetSymbol(hSSL, SSL_COMP_get_compression_methods);
> fn();
> ReleaseLibrary(hSSL);
> }
>
> Is it still a one-off leak?
>
> Unfreed memory is not a problem in an application, but it's a whole
> different story in a library.

I agree that the rules are different for shared libraries, and if ssl hasn't
cleaned up after itself completely when the call to
ReleaseLibrary() returns, it's a leak.

My experience with dynamically linked libraries is limited to Mac OS'
Code Fragment Manager, where I've avoided some issues by linking plugins
against the standard library statically, so each one gets its own malloc
pool.  I also generally work in C++, where I can use statically allocated
objects with constructors and destructors, such as smart pointers.

> Another example of such problems in libraries is atexit().  atexit()
> works great in an application, but use it in libssl.so when used in
> the code above and you'll most likely get a hard crash when the
> application terminates because the handler is still registered with
> the C runtime library, but the code has been unloaded.

If OpenSSL claims to be usable when dynamically loaded, then it's a bug in
OpenSSL.  Otherwise, it's a bug in the code that's loading it.

Josh

> -----Original Message-----
> From: [hidden email]
> [mailto:[hidden email]] On Behalf Of Joshua Juran
> Sent: Wednesday, 16 November 2005 12:38 PM
> To: [hidden email]
> Subject: Re: SSL_library_init - missing 36 bytes after cleanup
>
> On Nov 15, 2005, at 7:29 PM, Steven Reddie wrote:
>
>> David,
>>
>> If 36 bytes are being dynamically allocated and not being freed how
>> is it not a leak?
>>
>> Steven
>
> Because it only happens once.
>
> Imagine that when you shut off a faucet, water drips out for the next
> ten seconds and then stops.  That's not a leak, and it's not a
> problem.
>   What would be (both a problem and a leak) is water continually
> dripping at a regular frequency.
>
> The 36 bytes is not considered a leak because it's acquired only once
> as a part of initialization, not proportionally to the use of your
> application.
> And it does get released -- when the process exits. :-)
>
> If it's a problem, maybe you should invest in a RAM upgrade...  :-D

______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
User Support Mailing List                    [hidden email]
Automated List Manager                           [hidden email]

______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
User Support Mailing List                    [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: SSL_library_init - missing 36 bytes after cleanup

Jonathon Green
In reply to this post by Joshua Juran
Hi All,

I agree with Steven and think that this is obviously a
leak.

I submitted a trivial patch for this a while back
which added a function that did a pop-free on the SSL
compression methods stack. It hasn't been adopted but
in my opinion it should be in order for the library to
be dynamically loadable without memory leaks.

Regards,

Jon.


--- Joshua Juran <[hidden email]> wrote:

> On Nov 15, 2005, at 9:00 PM, Steven Reddie wrote:
>
> > I understand about one-off leaks, but we're
> talking about a dynamically
> > loadable library when we're talking about OpenSSL.
> >
> > What would happen if an application did something
> like this:
> >
> > for (int i=0; i<1000; i++)
> > {
> > hSSL = LoadLibrary("libssl.so")
> > fn = GetSymbol(hSSL,
> SSL_COMP_get_compression_methods);
> > fn();
> > ReleaseLibrary(hSSL);
> > }
> >
> > Is it still a one-off leak?
> >
> > Unfreed memory is not a problem in an application,
> but it's a whole
> > different story in a library.
>
> I agree that the rules are different for shared
> libraries, and if ssl
> hasn't cleaned up after itself completely when the
> call to
> ReleaseLibrary() returns, it's a leak.
>
> My experience with dynamically linked libraries is
> limited to Mac OS'
> Code Fragment Manager, where I've avoided some
> issues by linking
> plugins against the standard library statically, so
> each one gets its
> own malloc pool.  I also generally work in C++,
> where I can use
> statically allocated objects with constructors and
> destructors, such as
> smart pointers.
>
> > Another example of such problems in libraries is
> atexit().  atexit()
> > works
> > great in an application, but use it in libssl.so
> when used in the code
> > above
> > and you'll most likely get a hard crash when the
> application terminates
> > because the handler is still registered with the C
> runtime library,
> > but the
> > code has been unloaded.
>
> If OpenSSL claims to be usable when dynamically
> loaded, then it's a bug
> in OpenSSL.  Otherwise, it's a bug in the code
> that's loading it.
>
> Josh
>
> > -----Original Message-----
> > From: [hidden email]
> > [mailto:[hidden email]] On Behalf
> Of Joshua Juran
> > Sent: Wednesday, 16 November 2005 12:38 PM
> > To: [hidden email]
> > Subject: Re: SSL_library_init - missing 36 bytes
> after cleanup
> >
> > On Nov 15, 2005, at 7:29 PM, Steven Reddie wrote:
> >
> >> David,
> >>
> >> If 36 bytes are being dynamically allocated and
> not being freed how is
> >> it not a leak?
> >>
> >> Steven
> >
> > Because it only happens once.
> >
> > Imagine that when you shut off a faucet, water
> drips out for the next
> > ten
> > seconds and then stops.  That's not a leak, and
> it's not a problem.
> >   What would be (both a problem and a leak) is
> water continually
> > dripping at
> > a regular frequency.
> >
> > The 36 bytes is not considered a leak because it's
> acquired only once
> > as a
> > part of initialization, not proportionally to the
> use of your
> > application.
> > And it does get released -- when the process
> exits. :-)
> >
> > If it's a problem, maybe you should invest in a
> RAM upgrade...  :-D
>
>
______________________________________________________________________
> OpenSSL Project                                
> http://www.openssl.org
> User Support Mailing List                  
> [hidden email]
> Automated List Manager                          
> [hidden email]
>



               
____________________________________________________
Do you Yahoo!?
The New Yahoo! Movies: Check out the Latest Trailers, Premiere Photos and full Actor Database.
http://au.movies.yahoo.com
______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
User Support Mailing List                    [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

RE: SSL_library_init - missing 36 bytes after cleanup

JoelKatz
In reply to this post by Steven Reddie

> I understand about one-off leaks, but we're talking about a dynamically
> loadable library when we're talking about OpenSSL.

> What would happen if an application did something like this:
>
> for (int i=0; i<1000; i++)
> {
> hSSL = LoadLibrary("libssl.so")
> fn = GetSymbol(hSSL, SSL_COMP_get_compression_methods);
> fn();
> ReleaseLibrary(hSSL);
> }

> Is it still a one-off leak?

        It is often effectively impossible to fully clean up without destroying the
process. It may be a good goal to aim for, but it is by no stretch of the
imagination a requirement.

        I would argue that an application that did something like that is broken.
If you plan to reuse a library soon, unloading it and reloading it is
broken.

> Unfreed memory is not a problem in an application, but it's a whole
> different story in a library.

        Nevertheless, perfectly freeing everything is not always possible and not
always worth the effort. In this particular case, it is a trivial bug.

> Another example of such problems in libraries is atexit().  atexit() works
> great in an application, but use it in libssl.so when used in the
> code above
> and you'll most likely get a hard crash when the application terminates
> because the handler is still registered with the C runtime
> library, but the
> code has been unloaded.

        It really depends upon what types of use the library is supposed to
support. Not every library claims to be suitable to use this way and for
some problems, it's simply prohibitively expensive to make them do so.

        DS


______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
User Support Mailing List                    [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

RE: SSL_library_init - missing 36 bytes after cleanup

JoelKatz
In reply to this post by Steven Reddie

> Dismissing leaks as one-off's is a pet peeve of mine.  The notion
> of one-off
> leaks in an executable is arguably passable, but becomes a plain
> old memory
> leak just like any other when packaged as a library.

        Not if the memory is reused if the library is unloaded and reloaded.

        How would you handle TSD, for example? There is no *portable* way to hook
the destruction of a thread.

        DS


______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
User Support Mailing List                    [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

RE: SSL_library_init - missing 36 bytes after cleanup

Steven Reddie
In reply to this post by JoelKatz
Sure, that's a contrived and extreme example, but the best way to illustrate
such a leak.  For long-running (24x7) servers that may consist of many
components and subcomponents this scenario can occur.  With large enterprise
server applications that are built with processes that value component
re-use it's effectively impossible to know how some low-level subcomponent
is implemented.  While excessively loading and unloading DLLs may be less
efficient, a 36 byte leak occuring frequently enough will bring down a
server in time.

I disagree with it being prohibitively expensive to eradicate such leaks.
Yes it takes some effort, but a library-lifetime structure to house such
"global" dynamically allocated data, or a cleanup callback registration
scheme, makes it trivial to release it all when appropriate.  I guess it
depends on how much you value leak-free code.  For me it's a
needle-in-a-haystack problem; that every "one-off" leak has the potential to
mask real leaks and make using leak detection tools such as Purify and
BoundsChecker less fruitful.

Steven

-----Original Message-----
From: [hidden email]
[mailto:[hidden email]] On Behalf Of David Schwartz
Sent: Wednesday, 16 November 2005 3:02 PM
To: [hidden email]
Subject: RE: SSL_library_init - missing 36 bytes after cleanup


> I understand about one-off leaks, but we're talking about a
> dynamically loadable library when we're talking about OpenSSL.

> What would happen if an application did something like this:
>
> for (int i=0; i<1000; i++)
> {
> hSSL = LoadLibrary("libssl.so")
> fn = GetSymbol(hSSL, SSL_COMP_get_compression_methods);
> fn();
> ReleaseLibrary(hSSL);
> }

> Is it still a one-off leak?

        It is often effectively impossible to fully clean up without
destroying the process. It may be a good goal to aim for, but it is by no
stretch of the imagination a requirement.

        I would argue that an application that did something like that is
broken.
If you plan to reuse a library soon, unloading it and reloading it is
broken.

> Unfreed memory is not a problem in an application, but it's a whole
> different story in a library.

        Nevertheless, perfectly freeing everything is not always possible
and not always worth the effort. In this particular case, it is a trivial
bug.

> Another example of such problems in libraries is atexit().  atexit()
> works great in an application, but use it in libssl.so when used in
> the code above and you'll most likely get a hard crash when the
> application terminates because the handler is still registered with
> the C runtime library, but the code has been unloaded.

        It really depends upon what types of use the library is supposed to
support. Not every library claims to be suitable to use this way and for
some problems, it's simply prohibitively expensive to make them do so.

        DS


______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
User Support Mailing List                    [hidden email]
Automated List Manager                           [hidden email]

______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
User Support Mailing List                    [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

RE: SSL_library_init - missing 36 bytes after cleanup

Steven Reddie
In reply to this post by JoelKatz
I don't understand how the memory could be reused since there is no
mechanism that I know of that the runtime library would or could use to
track this.  For a dynamically linked C runtime library the allocated memory
is held in the heap that remains after the library unloads, at which time
the pointer to it is lost.  When the library reloads there is no way, at
least that I know of, that this memory could be found again.

There may be no portable way to handle TSD cleanup, but there is no portable
way to do threading at all anyway, so I'm not sure what your argument is
there.  I'm not arguing that any of this needs to be done in a portable way,
and of course it cannot be since dynamically loadable modules (and
threading) are outside the scope of the C standard.  My original response
was to correct your reply to Steffen in which you stated that these 36 bytes
are not leaked.  The fact is that they are.  Perhaps it's not a serious
leak, but a leak nonetheless.

Regards,

Steven

-----Original Message-----
From: [hidden email]
[mailto:[hidden email]] On Behalf Of David Schwartz
Sent: Wednesday, 16 November 2005 3:03 PM
To: [hidden email]
Subject: RE: SSL_library_init - missing 36 bytes after cleanup


> Dismissing leaks as one-off's is a pet peeve of mine.  The notion of
> one-off leaks in an executable is arguably passable, but becomes a
> plain old memory leak just like any other when packaged as a library.

        Not if the memory is reused if the library is unloaded and reloaded.

        How would you handle TSD, for example? There is no *portable* way to
hook the destruction of a thread.

        DS


______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
User Support Mailing List                    [hidden email]
Automated List Manager                           [hidden email]

______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
User Support Mailing List                    [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

RE: SSL_library_init - missing 36 bytes after cleanup

JoelKatz

> There may be no portable way to handle TSD cleanup, but there is
> no portable
> way to do threading at all anyway, so I'm not sure what your argument is
> there.


        Huh? POSIX threads are portable. There are even support libraries for
WIN32.

> I'm not arguing that any of this needs to be done in a
> portable way,
> and of course it cannot be since dynamically loadable modules (and
> threading) are outside the scope of the C standard.

        Well, that's the problem. When you're trying to maintain a portable
library, doing things that can't be done portably gets a low priority. It is
often a better choice just not to do them.

> My original response
> was to correct your reply to Steffen in which you stated that
> these 36 bytes
> are not leaked.  The fact is that they are.  Perhaps it's not a serious
> leak, but a leak nonetheless.

        They are leaked only if a program repeatedly opens and closes the OpenSSL
library. If this is not supported, then they are only leaked if the
library's interface contract is broken.

        I don't know if OpenSSL is supposed to specifically support the unusual use
of being dynamically linked, and repeatedly opened and closed. A leak that
only occurs under unsupported usage is not a leak if only supported usage is
attempted.

        DS


______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
User Support Mailing List                    [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

RE: SSL_library_init - missing 36 bytes after cleanup

Steven Reddie
Yes, I use pthreads on Windows.  Since you stated "How would you handle TSD,
for example? There is no *portable* way to hook the destruction of a
thread." I figured you were dismissing pthreads as non-portable due to it
not being natively available on Windows, and accepted that you were only
interested in native threading.  You may have forgotten about:
        int pthread_key_create(pthread_key_t *key, void
(*destructor)(void*));

There are so many things that can't be done portably.
Daemons/Windows-services, sockets/winsock, drive/mount enumeration, GUIs(!),
etc.  I don't give these a low priority just because there are system
specifics involved.  Check out all of the #if's in OpenSSL; the crypto/rand
code for starters.  It would be nice to write pure POSIX code that works
across all platforms but in reality how often is this possible for anything
other than a trivial app?  Encapsulating system specifics is a perfectly
reasonable compromise, and the same can obviously be done for threading and
dynamic library management.  Pthread-win32 is of course such a library.

I don't know that repeatedly loading and unloading DLLs is unusal.  Sure,
doing so constantly would be weird, but I've already stated that the example
I presented was extreme in order to illustrate the point that we are talking
about a leak.  Good point, if OpenSSL was documented to not support being
dynamically loaded/unloaded/reloaded then my argument would be moot, but I
don't believe it is documented, so arguing over this is a religious war at
best.  It seems that you've not needed to dynamically load/unload/reload
DLLs, but believe me, there are scenarios where it needs to be done.  And
when you do, this leak quacks like a leak.

Steven

-----Original Message-----
From: [hidden email]
[mailto:[hidden email]] On Behalf Of David Schwartz
Sent: Wednesday, 16 November 2005 5:46 PM
To: [hidden email]
Subject: RE: SSL_library_init - missing 36 bytes after cleanup


> There may be no portable way to handle TSD cleanup, but there is no
> portable way to do threading at all anyway, so I'm not sure what your
> argument is there.


        Huh? POSIX threads are portable. There are even support libraries
for WIN32.

> I'm not arguing that any of this needs to be done in a portable way,
> and of course it cannot be since dynamically loadable modules (and
> threading) are outside the scope of the C standard.

        Well, that's the problem. When you're trying to maintain a portable
library, doing things that can't be done portably gets a low priority. It is
often a better choice just not to do them.

> My original response
> was to correct your reply to Steffen in which you stated that these 36
> bytes are not leaked.  The fact is that they are.  Perhaps it's not a
> serious leak, but a leak nonetheless.

        They are leaked only if a program repeatedly opens and closes the
OpenSSL library. If this is not supported, then they are only leaked if the
library's interface contract is broken.

        I don't know if OpenSSL is supposed to specifically support the
unusual use of being dynamically linked, and repeatedly opened and closed. A
leak that only occurs under unsupported usage is not a leak if only
supported usage is attempted.

        DS


______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
User Support Mailing List                    [hidden email]
Automated List Manager                           [hidden email]

______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
User Support Mailing List                    [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: SSL_library_init - missing 36 bytes after cleanup

Steffen Fiksdal
In reply to this post by Jonathon Green

I guess my 36 bytes are gone for good, and they will not be given back to
me :)

As far as I am concerned OpenSSL has a bug.
Off course is it more a theoretical matter than a practical one, but when
it is detected why not fix it?


Best Regards
Steffen Fiksdal


On Wed, 16 Nov 2005, Jonathon Green wrote:

> Hi All,
>
> I agree with Steven and think that this is obviously a
> leak.
>
> I submitted a trivial patch for this a while back
> which added a function that did a pop-free on the SSL
> compression methods stack. It hasn't been adopted but
> in my opinion it should be in order for the library to
> be dynamically loadable without memory leaks.
>
> Regards,
>
> Jon.
>
>
> --- Joshua Juran <[hidden email]> wrote:
>
>> On Nov 15, 2005, at 9:00 PM, Steven Reddie wrote:
>>
>>> I understand about one-off leaks, but we're
>> talking about a dynamically
>>> loadable library when we're talking about OpenSSL.
>>>
>>> What would happen if an application did something
>> like this:
>>>
>>> for (int i=0; i<1000; i++)
>>> {
>>> hSSL = LoadLibrary("libssl.so")
>>> fn = GetSymbol(hSSL,
>> SSL_COMP_get_compression_methods);
>>> fn();
>>> ReleaseLibrary(hSSL);
>>> }
>>>
>>> Is it still a one-off leak?
>>>
>>> Unfreed memory is not a problem in an application,
>> but it's a whole
>>> different story in a library.
>>
>> I agree that the rules are different for shared
>> libraries, and if ssl
>> hasn't cleaned up after itself completely when the
>> call to
>> ReleaseLibrary() returns, it's a leak.
>>
>> My experience with dynamically linked libraries is
>> limited to Mac OS'
>> Code Fragment Manager, where I've avoided some
>> issues by linking
>> plugins against the standard library statically, so
>> each one gets its
>> own malloc pool.  I also generally work in C++,
>> where I can use
>> statically allocated objects with constructors and
>> destructors, such as
>> smart pointers.
>>
>>> Another example of such problems in libraries is
>> atexit().  atexit()
>>> works
>>> great in an application, but use it in libssl.so
>> when used in the code
>>> above
>>> and you'll most likely get a hard crash when the
>> application terminates
>>> because the handler is still registered with the C
>> runtime library,
>>> but the
>>> code has been unloaded.
>>
>> If OpenSSL claims to be usable when dynamically
>> loaded, then it's a bug
>> in OpenSSL.  Otherwise, it's a bug in the code
>> that's loading it.
>>
>> Josh
>>
>>> -----Original Message-----
>>> From: [hidden email]
>>> [mailto:[hidden email]] On Behalf
>> Of Joshua Juran
>>> Sent: Wednesday, 16 November 2005 12:38 PM
>>> To: [hidden email]
>>> Subject: Re: SSL_library_init - missing 36 bytes
>> after cleanup
>>>
>>> On Nov 15, 2005, at 7:29 PM, Steven Reddie wrote:
>>>
>>>> David,
>>>>
>>>> If 36 bytes are being dynamically allocated and
>> not being freed how is
>>>> it not a leak?
>>>>
>>>> Steven
>>>
>>> Because it only happens once.
>>>
>>> Imagine that when you shut off a faucet, water
>> drips out for the next
>>> ten
>>> seconds and then stops.  That's not a leak, and
>> it's not a problem.
>>>   What would be (both a problem and a leak) is
>> water continually
>>> dripping at
>>> a regular frequency.
>>>
>>> The 36 bytes is not considered a leak because it's
>> acquired only once
>>> as a
>>> part of initialization, not proportionally to the
>> use of your
>>> application.
>>> And it does get released -- when the process
>> exits. :-)
>>>
>>> If it's a problem, maybe you should invest in a
>> RAM upgrade...  :-D
>>
>>
> ______________________________________________________________________
>> OpenSSL Project
>> http://www.openssl.org
>> User Support Mailing List
>> [hidden email]
>> Automated List Manager
>> [hidden email]
>>
>
>
>
>
> ____________________________________________________
> Do you Yahoo!?
> The New Yahoo! Movies: Check out the Latest Trailers, Premiere Photos and full Actor Database.
> http://au.movies.yahoo.com
> ______________________________________________________________________
> OpenSSL Project                                 http://www.openssl.org
> User Support Mailing List                    [hidden email]
> Automated List Manager                           [hidden email]
>
______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
User Support Mailing List                    [hidden email]
Automated List Manager                           [hidden email]