Win32 OPENSSL_USE_APPLINK usage

classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|

Win32 OPENSSL_USE_APPLINK usage

Patrice Guérin
Hello All,

First of all, I'm new in OpenSSL and I need your advices . Please, excuse my poor english.

If I have correctly understood the FAQ about the OPENSSL_applink. It applies only in the Win32 version:
  • If OPENSSL_USE_APPLINK is not defined at the compilation stage of OpenSSL, the snippet applink.c is not mandatory BUT I have to build a debug and a release version of OpenSSL DLLs.
    I don't know why but it seems to be related with FILE functions.
  • If OPENSSL_USE_APPLINK is defined, the snippet applink.c is mandatory in an application and the release version of OpenSSL DLLs can be used in any case.
The OPENSSL_applink function is searched only in the application module, so it's necessary to add or include the snippet with any application that use OpensSSL.
In the case of a unique DLL that use OpenSSL, what do think about the following way to define and use OPENSSL_applink
(I've tried this with success with 0.9.8.n but implement it in a brute force manner) :

ms/uplink.h
Added the following declaration next to extern void *OPENSSL_UplinkTable[]; :
extern void OPENSSL_SetApplink( void** (*foo)() );
ms/uplink.c
Added :
static    void** (*applink_foo)() = NULL;
void OPENSSL_SetApplink( void** (*foo)() )
{
    applink_foo = foo;
}
and modifiy OPENSSL_Uplink(). Priority is given to OPENSSL_Applink if it's found in the current module. If it's not, if applink_foo is not set, give the error message else use applink_foo.
        applink=(void**(*)())GetProcAddress(h,"OPENSSL_Applink");
        if (applink==NULL)
        {    if( applink_foo == NULL )
            {    apphandle=(HMODULE)-1;
                _tcscpy (msg+len,_T("no OPENSSL_Applink"));
                break;
            }
            else applink = applink_foo;
        }
In the user DLL that uses OpenSSL, add or include the snippet applink.c in the DLL project and
Add the following declaration (I don't know where to put it in OpenSSL include files)
    extern "C" {
        void **OPENSSL_Applink();
        void OPENSSL_SetApplink( void **(*foo)() );
        };
and add the following in the initialisation function or dllmain:
    CRYPTO_malloc_init();
    OPENSSL_SetApplink( OPENSSL_Applink );
    SSL_library_init(); /* load encryption & hash algorithms for SSL */               
    SSL_load_error_strings(); /* load the error strings for good error reporting */
This way, OPENSSL_Applink can be used inside DLLs, but I'm not sure of implications so I need your advices.

I want to point to your attention that the call ERR_print_errors_fp( stderr ) from the user DLL lead to an Access violation (0xc0000005). This situation arises in mixing debug and release versions and
is probably due to the fact that ERR_print_errors_fp( stderr ) calls fwrite directly. UP_fwrite should be used instead.

Thank you for reading
Best regards,
Patrice.

Reply | Threaded
Open this post in threaded view
|

Re: Win32 OPENSSL_USE_APPLINK usage

Phillip Hellewell-2
2010/4/16 Patrice Guérin <[hidden email]>
This way, OPENSSL_Applink can be used inside DLLs, but I'm not sure of implications so I need your advices.

I'm not an OpenSSL developer, but I think being able to set the Applink function pointer explicitly is an excellent idea!

Just two days ago I was banging my head against this very problem.  I have a C# app that calls into a C++ DLL that utilizes OpenSSL.  Adding applink.c to the DLL project didn't get rid of the error like I hoped it would, and after analyzing the OpenSSL code I discovered to my dismay that it always looks in the EXE for the OPENSSL_Applink function.  Since this app is a C# project, adding applink.c to it is pretty much impossible.

I actually ended up solving it by removing all uses of BIO_new_fp() in favor of my own custom BIO that I just finished writing earlier this week.  I hadn't thought of recompiling OpenSSL without OPENSSL_USE_APPLINK, but next time around I may do that.  But if your idea gets implemented, then I won't have to.

Phillip
Reply | Threaded
Open this post in threaded view
|

Re: Win32 OPENSSL_USE_APPLINK usage

Andy Polyakov
In reply to this post by Patrice Guérin
> If I have correctly understood the FAQ about the OPENSSL_applink. It
> applies only in the Win32 version:
>
>     * If OPENSSL_USE_APPLINK is not defined at the compilation stage of
>       OpenSSL, the snippet applink.c is not mandatory BUT I have to
>       build a debug and a release version of OpenSSL DLLs.
>       I don't know why but it seems to be related with FILE functions.
>     * If OPENSSL_USE_APPLINK is defined, the snippet applink.c is
>       mandatory in an application and the release version of OpenSSL
>       DLLs can be used in any case.

Well, OPENSSL_USE_APPLINK is always defined in Win32 builds. applink.c
is required if a) application uses certain subset of OpenSSL API and b)
application and DLL are compiled with different flags, debug vs.
non-debug, or different compilers.

> The OPENSSL_applink function is searched only in the application module,

Which is why it's called *app*link.

> so it's necessary to add or include the snippet with any application
> that use OpensSSL.

It should be noted that applink was introduced primarily in order to
facilitate migration for legacy applications. New code should rather
abstain from using above mentioned subset of OpenSSL API (whatever using
FILE *) and simply avoid applink.

> In the case of a unique DLL that use OpenSSL, what do think about the
> following way to define and use OPENSSL_applink
> (I've tried this with success with 0.9.8.n but implement it in a brute
> force manner) :
>
> ms/uplink.h
> Added the following declaration next to extern void
> *OPENSSL_UplinkTable[]; :
>
>     extern void OPENSSL_SetApplink( void** (*foo)() );
>
> ms/uplink.c
> Added :
>
>     static    void** (*applink_foo)() = NULL;
>     void OPENSSL_SetApplink( void** (*foo)() )
>     {
>         applink_foo = foo;
>     }
>
> and modifiy OPENSSL_Uplink(). Priority is given to OPENSSL_Applink if
> it's found in the current module. If it's not, if applink_foo is not
> set, give the error message else use applink_foo.
>
>             applink=(void**(*)())GetProcAddress(h,"OPENSSL_Applink");
>             if (applink==NULL)
>             {    if( applink_foo == NULL )
>                 {    apphandle=(HMODULE)-1;
>                     _tcscpy (msg+len,_T("no OPENSSL_Applink"));
>                     break;
>                 }
>                 else applink = applink_foo;
>             }
>
> In the user DLL that uses OpenSSL, add or include the snippet applink.c
> in the DLL project and
> Add the following declaration (I don't know where to put it in OpenSSL
> include files)
>
>         extern "C" {
>             void **OPENSSL_Applink();
>             void OPENSSL_SetApplink( void **(*foo)() );
>             };
>
> and add the following in the initialisation function or dllmain:
>
>         CRYPTO_malloc_init();
>         OPENSSL_SetApplink( OPENSSL_Applink );
>         SSL_library_init(); /* load encryption & hash algorithms for SSL
>     */              
>         SSL_load_error_strings(); /* load the error strings for good
>     error reporting */
>
> This way, OPENSSL_Applink can be used inside DLLs, but I'm not sure of
> implications so I need your advices.

As there is no way to ensure/know that there is only one DLL using
OpenSSL, it's not appropriate for main stream. If it helps to develop
and debug your application, there is nothing that prevents you from
doing so.

> I want to point to your attention that the call ERR_print_errors_fp(
> stderr ) from the user DLL lead to an Access violation (0xc0000005).
> This situation arises in mixing debug and release versions and
> is probably due to the fact that ERR_print_errors_fp( stderr ) calls
> fwrite directly. UP_fwrite should be used instead.

This is bug. It was fixed in development branch and is not present in
1.0.0. http://cvs.openssl.org/chngview?cn=19607 is committed even to
0.9.8. A.
______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
Development Mailing List                       [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: Win32 OPENSSL_USE_APPLINK usage

Andy Polyakov
In reply to this post by Phillip Hellewell-2
> Just two days ago I was banging my head against this very problem.  I
> have a C# app that calls into a C++ DLL that utilizes OpenSSL.  Adding
> applink.c to the DLL project didn't get rid of the error like I hoped it
> would, and after analyzing the OpenSSL code I discovered to my dismay
> that it always looks in the EXE for the OPENSSL_Applink function.  Since
> this app is a C# project, adding applink.c to it is pretty much impossible.
>
> I actually ended up solving it by removing all uses of BIO_new_fp() in
> favor of my own custom BIO  that I just finished writing earlier this
> week.

Why not BIO_new_file? A.
______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
Development Mailing List                       [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: Win32 OPENSSL_USE_APPLINK usage

Phillip Hellewell-2
On Tue, Apr 20, 2010 at 2:33 PM, Andy Polyakov <[hidden email]> wrote:
>>
>> I actually ended up solving it by removing all uses of BIO_new_fp() in
>> favor of my own custom BIO  that I just finished writing earlier this
>> week.
>
> Why not BIO_new_file? A.

Yeah, I discovered while analyzing the code that using BIO_new_file()
rather than BIO_new_fp() would disengage applink, however that was not
an option for me because BIO_new_file() can't open a file whose name
contains non-ANSI Unicode characters on Windows.  I have to open the
file myself using _wfopen().

Phillip
______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
Development Mailing List                       [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: Win32 OPENSSL_USE_APPLINK usage

Andy Polyakov
>>> I actually ended up solving it by removing all uses of BIO_new_fp() in
>>> favor of my own custom BIO  that I just finished writing earlier this
>>> week.
>> Why not BIO_new_file?
>
> Yeah, I discovered while analyzing the code that using BIO_new_file()
> rather than BIO_new_fp() would disengage applink, however that was not
> an option for me because BIO_new_file() can't open a file whose name
> contains non-ANSI Unicode characters on Windows.  I have to open the
> file myself using _wfopen().

There was suggestion to fall back to wfopen from a vmware engineer a
while ago, but he couldn't provide patch (not that it would be very
complex) and it was not followed up. Idea must have been something
similar to just committed http://cvs.openssl.org/chngview?cn=19610.

Please note that it's *not* like I disapprove practice of custom BIO and
suggest you to abandon it. It's *not* like that. On the contrary I can
appreciate advantage and even say that it's better approach than one
suggested in the beginning of the thread. A.
______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
Development Mailing List                       [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: Win32 OPENSSL_USE_APPLINK usage

Phillip Hellewell-2
> There was suggestion to fall back to wfopen from a vmware engineer a while
> ago, but he couldn't provide patch (not that it would be very complex) and
> it was not followed up. Idea must have been something similar to just
> committed http://cvs.openssl.org/chngview?cn=19610.

Sounds like a good idea, but perhaps an even better idea would be to
provide an alternate BIO_new_file() that takes a wchar_t*.  It would
call _wfopen() if supported by the compiler, and if not it would
convert the wchar_t* to a char* using wcstombs() and call fopen().

> Please note that it's *not* like I disapprove practice of custom BIO and
> suggest you to abandon it. It's *not* like that. On the contrary I can
> appreciate advantage and even say that it's better approach than one
> suggested in the beginning of the thread. A.

Cool, thanks :)  Yeah, I was quite happy with myself that I happened
to write a custom BIO just in time to avoid the whole applink mess,
even though that's not even the reason I wrote it.

Phillip
______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
Development Mailing List                       [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: Win32 OPENSSL_USE_APPLINK usage

Patrice Guérin
In reply to this post by Andy Polyakov
Hello Andy,

Thank you very much for answering

Andy Polyakov a écrit :

>> If I have correctly understood the FAQ about the OPENSSL_applink. It
>> applies only in the Win32 version:
>>
>>     * If OPENSSL_USE_APPLINK is not defined at the compilation stage of
>>       OpenSSL, the snippet applink.c is not mandatory BUT I have to
>>       build a debug and a release version of OpenSSL DLLs.
>>       I don't know why but it seems to be related with FILE functions.
>>     * If OPENSSL_USE_APPLINK is defined, the snippet applink.c is
>>       mandatory in an application and the release version of OpenSSL
>>       DLLs can be used in any case.
>
> Well, OPENSSL_USE_APPLINK is always defined in Win32 builds. applink.c
> is required if a) application uses certain subset of OpenSSL API and
> b) application and DLL are compiled with different flags, debug vs.
> non-debug, or different compilers.
I compile OpenSSL with Visual C++ 6, the same with which I build my DLL
and applications.
I've already used debug vs. non-debug, for example zlib.dll, I've never
had problems.
What is the obscure reason of this access violation ?

>
>> The OPENSSL_applink function is searched only in the application module,
>
> Which is why it's called *app*link.
>
>> so it's necessary to add or include the snippet with any application
>> that use OpensSSL.
>
> It should be noted that applink was introduced primarily in order to
> facilitate migration for legacy applications. New code should rather
> abstain from using above mentioned subset of OpenSSL API (whatever
> using FILE *) and simply avoid applink.
Do you mean that BIO API using FILE* is deprecated ? I have not seen
this mention in the documentation.
I find that this API is clean and coherent.
Only Windows sucks. Why BIO should suffer of this ?
If the only way to have a DLL that uses OpenSSL as a wrapper is to
compile OpenSSL without OPENSSL_USE_APPLINK define and provide a debug
and a release version... no problem... I will do so.

>
>> In the case of a unique DLL that use OpenSSL, what do think about the
>> following way to define and use OPENSSL_applink
>> (I've tried this with success with 0.9.8.n but implement it in a
>> brute force manner) :
>>
>> ms/uplink.h
>> Added the following declaration next to extern void
>> *OPENSSL_UplinkTable[]; :
>>
>>     extern void OPENSSL_SetApplink( void** (*foo)() );
>>
>> ms/uplink.c
>> Added :
>>
>>     static    void** (*applink_foo)() = NULL;
>>     void OPENSSL_SetApplink( void** (*foo)() )
>>     {
>>         applink_foo = foo;
>>     }
>>
>> and modifiy OPENSSL_Uplink(). Priority is given to OPENSSL_Applink if
>> it's found in the current module. If it's not, if applink_foo is not
>> set, give the error message else use applink_foo.
>>
>>             applink=(void**(*)())GetProcAddress(h,"OPENSSL_Applink");
>>             if (applink==NULL)
>>             {    if( applink_foo == NULL )
>>                 {    apphandle=(HMODULE)-1;
>>                     _tcscpy (msg+len,_T("no OPENSSL_Applink"));
>>                     break;
>>                 }
>>                 else applink = applink_foo;
>>             }
>>
>> In the user DLL that uses OpenSSL, add or include the snippet
>> applink.c in the DLL project and
>> Add the following declaration (I don't know where to put it in
>> OpenSSL include files)
>>
>>         extern "C" {
>>             void **OPENSSL_Applink();
>>             void OPENSSL_SetApplink( void **(*foo)() );
>>             };
>>
>> and add the following in the initialisation function or dllmain:
>>
>>         CRYPTO_malloc_init();
>>         OPENSSL_SetApplink( OPENSSL_Applink );
>>         SSL_library_init(); /* load encryption & hash algorithms for SSL
>>     */                       SSL_load_error_strings(); /* load the
>> error strings for good
>>     error reporting */
>>
>> This way, OPENSSL_Applink can be used inside DLLs, but I'm not sure
>> of implications so I need your advices.
>
> As there is no way to ensure/know that there is only one DLL using
> OpenSSL, it's not appropriate for main stream. If it helps to develop
> and debug your application, there is nothing that prevents you from
> doing so.
>
Of course ! but I don't especially want to do so !
This is the only way I' ve found to build a DLL using OpenSSL. I've
found impratical to add appilink.c in each application that uses the DLL.

>> I want to point to your attention that the call ERR_print_errors_fp(
>> stderr ) from the user DLL lead to an Access violation (0xc0000005).
>> This situation arises in mixing debug and release versions and
>> is probably due to the fact that ERR_print_errors_fp( stderr ) calls
>> fwrite directly. UP_fwrite should be used instead.
>
> This is bug. It was fixed in development branch and is not present in
> 1.0.0. http://cvs.openssl.org/chngview?cn=19607 is committed even to
> 0.9.8. A.
> ______________________________________________________________________
> OpenSSL Project                                 http://www.openssl.org
> Development Mailing List                       [hidden email]
> Automated List Manager                           [hidden email]
>
>

______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
Development Mailing List                       [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: Win32 OPENSSL_USE_APPLINK usage

Modem Man
In reply to this post by Andy Polyakov
Andy Polyakov schrieb:

>>>> I actually ended up solving it by removing all uses of BIO_new_fp() in
>>>> favor of my own custom BIO  that I just finished writing earlier this
>>>> week.
>>> Why not BIO_new_file?
>>
>> Yeah, I discovered while analyzing the code that using BIO_new_file()
>> rather than BIO_new_fp() would disengage applink, however that was not
>> an option for me because BIO_new_file() can't open a file whose name
>> contains non-ANSI Unicode characters on Windows.  I have to open the
>> file myself using _wfopen().
>
> There was suggestion to fall back to wfopen from a vmware engineer a
> while ago, but he couldn't provide patch (not that it would be very
> complex) and it was not followed up. Idea must have been something
> similar to just committed http://cvs.openssl.org/chngview?cn=19610.
why not adding the following to BIO_new_file()?

- BIO interface still uses char * (meaning latin ASCII 0x20..0x7F)
- BIO implementation calls UTF8_to_UCS16() on all platforms supporting
wfopen or _wfopen
- BIO implementation then calls wfopen / _wfopen with this UCS16 value
(sometimes known as WCHAR*)
- For Win32 and Win32_WinCE the conversion can be done with
FormatMessage() API. It's allways available.

... just my 5 cents.

The Modem Man

______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
Development Mailing List                       [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: Win32 OPENSSL_USE_APPLINK usage

Andy Polyakov
>>>>> I actually ended up solving it by removing all uses of BIO_new_fp() in
>>>>> favor of my own custom BIO  that I just finished writing earlier this
>>>>> week.
>>>> Why not BIO_new_file?
>>> Yeah, I discovered while analyzing the code that using BIO_new_file()
>>> rather than BIO_new_fp() would disengage applink, however that was not
>>> an option for me because BIO_new_file() can't open a file whose name
>>> contains non-ANSI Unicode characters on Windows.  I have to open the
>>> file myself using _wfopen().
>> There was suggestion to fall back to wfopen from a vmware engineer a
>> while ago, but he couldn't provide patch (not that it would be very
>> complex) and it was not followed up. Idea must have been something
>> similar to just committed http://cvs.openssl.org/chngview?cn=19610.
> why not adding the following to BIO_new_file()?
>
> - BIO interface still uses char * (meaning latin ASCII 0x20..0x7F)

This is incorrect assumption. BIO_new_file and fopen allow for
characters recognized as valid in current locale, which is not
necessarily limited to ASCII. In other words it's perfectly possible to
BIO_new_file as well as to fopen file with international characters in
its name, but the set of allowed non-ASCII characters varies.

> - BIO implementation calls UTF8_to_UCS16()

I.e. suggestion is to unconditionally assume UTF8 encoding of file name.
It's not safe assumption (see above).

> on all platforms supporting
> wfopen or _wfopen

What are the platforms supporting wfopen? Among those supported by
OpenSSL? It's Windows platforms.

> - BIO implementation then calls wfopen / _wfopen with this UCS16 value
> (sometimes known as WCHAR*)
> - For Win32 and Win32_WinCE the conversion can be done with
> FormatMessage() API. It's allways available.

??? So as MultiByteToWideChar... A.
______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
Development Mailing List                       [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: Win32 OPENSSL_USE_APPLINK usage

Andy Polyakov
In reply to this post by Patrice Guérin
>>> If I have correctly understood the FAQ about the OPENSSL_applink. It
>>> applies only in the Win32 version:
>>>
>>>     * If OPENSSL_USE_APPLINK is not defined at the compilation stage of
>>>       OpenSSL, the snippet applink.c is not mandatory BUT I have to
>>>       build a debug and a release version of OpenSSL DLLs.
>>>       I don't know why but it seems to be related with FILE functions.
>>>     * If OPENSSL_USE_APPLINK is defined, the snippet applink.c is
>>>       mandatory in an application and the release version of OpenSSL
>>>       DLLs can be used in any case.
>>
>> Well, OPENSSL_USE_APPLINK is always defined in Win32 builds. applink.c
>> is required if a) application uses certain subset of OpenSSL API and
>> b) application and DLL are compiled with different flags, debug vs.
>> non-debug, or different compilers.
> I compile OpenSSL with Visual C++ 6, the same with which I build my DLL
> and applications.
> I've already used debug vs. non-debug, for example zlib.dll, I've never
> had problems.

Because zlib.dll does not accept any FILE * arguments.

> What is the obscure reason of this access violation ?

Every MSVCRT dll has own copy of FILE structures and passing pointers to
them between dlls is not safe. Nor is it safe to pass descriptors
returned by open between those dlls. Nor is it safe to let one dll to
free memory malloc-ed by another dll. "Every MSVCRT dll" refers to
different versions, debug vs. non-debug version, etc.

>> It should be noted that applink was introduced primarily in order to
>> facilitate migration for legacy applications. New code should rather
>> abstain from using above mentioned subset of OpenSSL API (whatever
>> using FILE *) and simply avoid applink.
> Do you mean that BIO API using FILE* is deprecated ? I have not seen
> this mention in the documentation.

Well, not really. But if you want to ensure least head-ache in
cross-platform development you better avoid FILE* API. It's just state
of affairs. Basically I mean life is a bitch and it's not going to change:-)

> I find that this API is clean and coherent.
> Only Windows sucks.

There you go. It sucks, but it doesn't mean that you as cross-platform
application developer can dismiss it. You have to use it no matter how
much does it suck. Same applies other way. It doesn't matter how much
you appreciate some API, you might find yourself in situation when you
have to not use it.

> Why BIO should suffer of this ?

Because it just does.

> If the only way to have a DLL that uses OpenSSL as a wrapper is to
> compile OpenSSL without OPENSSL_USE_APPLINK define and provide a debug
> and a release version... no problem... I will do so.

If executable modules, be it .exe or .dll, passing FILE* and malloc-ed
pointers between them are compiled with same compiler and compiler flags
as OpenSSL, applink won't be engaged. It doesn't matter it you compile
OpenSSL without OPENSSL_USE_APPLINK defined. I mean if you do, you would
absolutely have to compile with same compiler and flags. And when you do
that, applink would have kept itself out of your way anyway.

>>> This way, OPENSSL_Applink can be used inside DLLs, but I'm not sure
>>> of implications so I need your advices.
>>
>> As there is no way to ensure/know that there is only one DLL using
>> OpenSSL, it's not appropriate for main stream. If it helps to develop
>> and debug your application, there is nothing that prevents you from
>> doing so.
>>
> Of course ! but I don't especially want to do so !
> This is the only way I' ve found to build a DLL using OpenSSL.

Depending on your situation you might find it appropriate to embed
OpenSSL into your dll. I.e. as opposite to having "your" dll and
"openssl" dll, you can have one dll with openssl in it. And as the other
subscriber pointed out, custom BIO would do the trick too.

> I've
> found impratical to add appilink.c in each application that uses the DLL.

Nobody said that you will like the advice:-) Cheers. A.
______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
Development Mailing List                       [hidden email]
Automated List Manager                           [hidden email]