OpenSSL › OpenSSL - User

private key not available for client_cert_cb

_‹ Previous Topic Next Topic _›

Classic

List

Threaded

30 messages Options

George

private key not available for client_cert_cb

Hi,

I'm new to OpenSSL and am trying to set up mutual authentication in a client. The client is setup with OpenSSL 1.0.2u. and the client's certificate + private key is stored on a Smart Card. When the client receives a certificate request from the server during the mutual authentication handshake, the OpenSSL client_cert_cb callback function is automatically invoked. The problem is that client_cert_cb requires a private key. Unfortunately, it is not possible to get a private key from a Smart Card. Is there a way to send a certificate to the server without needing the private key?

I'm setting up the callback function with:

void SSL_CTX_set_client_cert_cb(SSL_CTX *ctx, int (*client_cert_cb)(SSL *ssl, X509 **x509, EVP_PKEY **pkey));

Here is a sample of what my code looks like when I set this up:

SSL_CTX_set_client_cert_cb(context, openSSLClientAuthenticationCallBack);

int openSSLClientAuthenticationCallBack(SSL *ssl, X509 **x509, EVP_PKEY **pkey)
{
. . .
}

I can access the Smart Card using the PKCS#11 interface and I'm able to get the certificate and sign it, etc. However, I cannot get the actual private key from the Smart Card.

Does anyone know how I can get around this problem?

Thanks,
George

Jan Just Keijser-2

Re: private key not available for client_cert_cb

Hi,

On 14/12/20 08:08, George wrote:

Hi,

I'm new to OpenSSL and am trying to set up mutual authentication in a client. The client is setup with OpenSSL 1.0.2u. and the client's certificate + private key is stored on a Smart Card. When the client receives a certificate request from the server during the mutual authentication handshake, the OpenSSL client_cert_cb callback function is automatically invoked. The problem is that client_cert_cb requires a private key. Unfortunately, it is not possible to get a private key from a Smart Card. Is there a way to send a certificate to the server without needing the private key?

I'm setting up the callback function with:

void SSL_CTX_set_client_cert_cb(SSL_CTX *ctx, int (*client_cert_cb)(SSL *ssl, X509 **x509, EVP_PKEY **pkey));

Here is a sample of what my code looks like when I set this up:

SSL_CTX_set_client_cert_cb(context, openSSLClientAuthenticationCallBack);

int openSSLClientAuthenticationCallBack(SSL *ssl, X509 **x509, EVP_PKEY **pkey)
{
. . .
}

I can access the Smart Card using the PKCS#11 interface and I'm able to get the certificate and sign it, etc. However, I cannot get the actual private key from the Smart Card.

Does anyone know how I can get around this problem?

to use a pkcs#11 smartcard you normally use the OpenSSL pkcs11 engine ; you then do something like:

    engine_name = "pkcs11";
    ENGINE_register_all_complete();
    pkey_engine = ENGINE_by_id( "dynamic" );
    if (pkey_engine)
        {
            if (!ENGINE_ctrl_cmd_string(pkey_engine, "SO_PATH", engine_name, 0)
             || !ENGINE_ctrl_cmd_string(pkey_engine, "LOAD", NULL, 0))
            {
                warn( "EAP-TLS: Error loading dynamic engine '%s'", engine_name );
                log_ssl_errors();
                ENGINE_free(e);
                pkey_engine = NULL;
            }
        }
    }

    if (pkey_engine)
    {
        if(!ENGINE_set_default(pkey_engine, ENGINE_METHOD_ALL))
    }
    pkey_engine = eaptls_ssl_load_engine( "pkcs11" );
    pkey = ENGINE_load_private_key(pkey_engine, pkey_identifier, transfer_pin, &cb_data);
    SSL_CTX_use_PrivateKey(ctx, pkey);

where "transfer_pin" is a callback UI function to query the user for the pkcs11 device password.

More detailed code can be found in my pppd EAP-TLS patch, file eap-tls.c at
https://github.com/jjkeijser/ppp/blob/eap-tls/pppd/eap-tls.c

(and search for pkey_engine)

HTH,

JJK

George

Re: private key not available for client_cert_cb

Hi Jan,

Thanks for your response. It looks like I don't already have the PPP and PPPD. Do I need to download and install the following?
https://github.com/jjkeijser/ppp/tree/eap-tls

I am using OpenSSL in Windows 10 and compiled it with Visual Studio 2019. Will this EAP-TLS code compile/work with Visual Studio in Windows?

Are there any other ways to get the Smart Card to work without needing to install additional software?

Thanks!
George

On 2020-12-14 3:51 a.m., Jan Just Keijser wrote:

Hi,

On 14/12/20 08:08, George wrote:

Hi,

   I'm new to OpenSSL and am trying to set up mutual authentication in a client. The client is setup with OpenSSL 1.0.2u. and the client's certificate + private key is stored on a Smart Card. When the client receives a certificate request from the server during the mutual authentication handshake, the OpenSSL client_cert_cb callback function is automatically invoked. The problem is that client_cert_cb requires a private key. Unfortunately, it is not possible to get a private key from a Smart Card. Is there a way to send a certificate to the server without needing the private key?

I'm setting up the callback function with:

void SSL_CTX_set_client_cert_cb(SSL_CTX *ctx, int (*client_cert_cb)(SSL *ssl, X509 **x509, EVP_PKEY **pkey));

Here is a sample of what my code looks like when I set this up:

SSL_CTX_set_client_cert_cb(context, openSSLClientAuthenticationCallBack);

int openSSLClientAuthenticationCallBack(SSL *ssl, X509 **x509, EVP_PKEY **pkey)
{
. . .
}

I can access the Smart Card using the PKCS#11 interface and I'm able to get the certificate and sign it, etc. However, I cannot get the actual private key from the Smart Card.

Does anyone know how I can get around this problem?

to use a pkcs#11 smartcard you normally use the OpenSSL pkcs11 engine ; you then do something like:

    engine_name = "pkcs11";
    ENGINE_register_all_complete();
    pkey_engine = ENGINE_by_id( "dynamic" );
    if (pkey_engine)
        {
            if (!ENGINE_ctrl_cmd_string(pkey_engine, "SO_PATH", engine_name, 0)
             || !ENGINE_ctrl_cmd_string(pkey_engine, "LOAD", NULL, 0))
            {
                warn( "EAP-TLS: Error loading dynamic engine '%s'", engine_name );
                log_ssl_errors();
                ENGINE_free(e);
                pkey_engine = NULL;
            }
        }
    }

    if (pkey_engine)
    {
        if(!ENGINE_set_default(pkey_engine, ENGINE_METHOD_ALL))
    }
    pkey_engine = eaptls_ssl_load_engine( "pkcs11" );
    pkey = ENGINE_load_private_key(pkey_engine, pkey_identifier, transfer_pin, &cb_data);
    SSL_CTX_use_PrivateKey(ctx, pkey);

where "transfer_pin" is a callback UI function to query the user for the pkcs11 device password.

More detailed code can be found in my pppd EAP-TLS patch, file eap-tls.c at
https://github.com/jjkeijser/ppp/blob/eap-tls/pppd/eap-tls.c

(and search for pkey_engine)

HTH,

JJK

Michael Wojcik

RE: private key not available for client_cert_cb

> From: openssl-users <[hidden email]> On Behalf Of George
> Sent: Monday, 14 December, 2020 08:15

> Thanks for your response. It looks like I don't already have the PPP and PPPD.

You don't need PPP to use a smartcard or other PKCS#11 device. Jan just mentioned the source as a exemplar of the interactions your code will need to have with OpenSSL.

> Are there any other ways to get the Smart Card to work without needing to
> install additional software?

Probably not.

OpenSSL's PKCS#11 Engine implements the PKCS#11 API. That API needs a way to talk to the particular PKCS#11-compatible hardware you're using. That means it needs a driver, and generally some configuration as well.

It's been a few years since I last played around with this - I got OpenSSL working with a NitroKey as part of a code-signing spike - but you'll need to investigate PKCS#11 support for your particular device. There are Open Source projects such as OpenSC which may give you part or all of what you need to get OpenSSL's PKCS#11 Engine working with your hardware.

When I did it, it wasn't trivial. I spent a couple of days on investigation and experimenting before I got anything working, and a couple more days making sure I understood the entire process and documenting procedures that worked consistently. (With some applications I had persistent problems such as Windows insisting on prompting for the device PIN instead of letting me supply it programmatically, but I think that was only when using Microsoft APIs rather than going through OpenSSL.)

If the client certificate uses a public key that corresponds to a private key on the smartcard, though, that's what you'll have to do. You can't use a certificate as a proof of identity without the corresponding private key. (Some HSMs and other crypto devices have support for exporting private keys, often as multiple shares, for backup and cloning purposes. Using that to get the private key for direct use defeats the whole purpose of an HSM, of course, so that shouldn't be used to bypass the card.)

--
Michael Wojcik

George

Re: private key not available for client_cert_cb

Hi Michael,

I see what you mean. So once I have everything setup, i use the following to get the private key:
EVP_PKEY *pkey = ENGINE_load_private_key(pkey_engine, pkey_identifier, transfer_pin, &cb_data);

Will pkey actually contain the private key from the smart card? I thought it was not possible to get a private key from a smart card?

Once I have pkey, do I simply use it within the client_cert_cb callback function?

Thanks,
George

On 2020-12-14 10:58 a.m., Michael Wojcik wrote:

From: openssl-users [hidden email] On Behalf Of George
Sent: Monday, 14 December, 2020 08:15

  Thanks for your response. It looks like I don't already have the PPP and PPPD.

You don't need PPP to use a smartcard or other PKCS#11 device. Jan just mentioned the source as a exemplar of the interactions your code will need to have with OpenSSL.

Are there any other ways to get the Smart Card to work without needing to
install additional software?

Probably not.

OpenSSL's PKCS#11 Engine implements the PKCS#11 API. That API needs a way to talk to the particular PKCS#11-compatible hardware you're using. That means it needs a driver, and generally some configuration as well.

It's been a few years since I last played around with this - I got OpenSSL working with a NitroKey as part of a code-signing spike - but you'll need to investigate PKCS#11 support for your particular device. There are Open Source projects such as OpenSC which may give you part or all of what you need to get OpenSSL's PKCS#11 Engine working with your hardware.

When I did it, it wasn't trivial. I spent a couple of days on investigation and experimenting before I got anything working, and a couple more days making sure I understood the entire process and documenting procedures that worked consistently. (With some applications I had persistent problems such as Windows insisting on prompting for the device PIN instead of letting me supply it programmatically, but I think that was only when using Microsoft APIs rather than going through OpenSSL.)

If the client certificate uses a public key that corresponds to a private key on the smartcard, though, that's what you'll have to do. You can't use a certificate as a proof of identity without the corresponding private key. (Some HSMs and other crypto devices have support for exporting private keys, often as multiple shares, for backup and cloning purposes. Using that to get the private key for direct use defeats the whole purpose of an HSM, of course, so that shouldn't be used to bypass the card.)

--
Michael Wojcik

Paul Nelson

Re: private key not available for client_cert_cb

In reply to this post by George

How I did this:

1) You can make up your own EVP_PKEY that uses your own engine implementation and attach a data ptr to it

                        EVP_PKEY* returnPKey;

                        returnPKey = EVP_PKEY_new();

                        if( returnPKey )

{

        		    ENGINE* engine = ENGINE_by_id(YOUR_ENGINE_ID);

                            RSA* sc_rsa = RSA_new_method(engine);

                            if( sc_rsa )

{

				// attach a reference to a structure holding your smart card middleware info

                                RSA_set_ex_data(sc_rsa, ENGINE_smartcard_rsa_idx_middleware(),

                                                (void*)middleware->handle);

                                EVP_PKEY* pk = X509_get_pubkey( returnCert );

                                if( pk )

{

                                    sc_rsa->e = BN_new();

                                    sc_rsa->n = BN_new();

                                    BN_copy(sc_rsa->e, pk->pkey.rsa->e);

                                    BN_copy(sc_rsa->n, pk->pkey.rsa->n);

                                    EVP_PKEY_free(pk);

                                    EVP_PKEY_set1_RSA(returnPKey, sc_rsa);

                                    RSA_free(sc_rsa);

                                    *outCert = make this X509 from your smart card certificate;

                                    *outpkey = returnPKey;

}

                                else

                                    LogError("smartcards_fetch_identity can't get pubkey\n");

}

Then for your engine you will need some methods to configure it as follows:

void ENGINE_load_smartcard_keychain(void);

/*

 * ENGINE_tss_keychain_rsa_idx_middleware returns a ex_data index where engine user should store the

 *  pointer to the info needed to use the middleware

*/

int ENGINE_smartcard_rsa_idx_middleware(void);

Your ENGINE_load_smartcard_keychain method should set global values that get returned by ENGINE_smartcard_rsa_idx_middleware:

        gMiddlewareRSAIndex = RSA_get_ex_new_index(0x1234, NULL, NULL, NULL, NULL);

Configure your engine filling in an RSA_METHOD structure with what you will need. You don’t really need all the methods in RSA_METHOD structure, and if you don’t need them add a stub that returns a 0. I did not need either of the mod_exp method or the public key encrypt and decrypt methods. I also did not need the verify or keygen methods. Your init and finish methods just need to return 1.

I set the RSA_METHOD flags to RSA_FLAG_FIPS_METHOD|RSA_METHOD_FLAG_NO_CHECK|RSA_FLAG_CACHE_PUBLIC

This leaves the cipher methods for private key encrypt/decrypt and sign. The private key methods will be where all the work is done. Write a function to perform the smartcard ‘crypt’ method and use it in private encrypt/decrypt and the signing methods. You will need to pay attention to padding and make sure you know how to pad for PKCS1 type 1. The RSA_SSLV23_PADDING is not required and you can just return an error if you get called with this.

I handle the PIN entry requirement by having the engine return a specific error if the PIN is needed, then handle the PIN entry in the application. Once the PIN is entered and available to the middleware, I retry the connection.

The trick is to get a pointer to your middleware implementation from the private key engine methods like this:

    my_middleware_handle =

    	(my_middleware_handle)RSA_get_ex_data(rsa, gMiddlewareRSAIndex);

I found that writing the engine was more straightforward that attempting to use PKCS11.

On Dec 14, 2020, at 1:08 AM, George <[hidden email]> wrote:

Hi,

I'm new to OpenSSL and am trying to set up mutual authentication in a client. The client is setup with OpenSSL 1.0.2u. and the client's certificate + private key is stored on a Smart Card. When the client receives a certificate request from the server during the mutual authentication handshake, the OpenSSL client_cert_cb callback function is automatically invoked. The problem is that client_cert_cb requires a private key. Unfortunately, it is not possible to get a private key from a Smart Card. Is there a way to send a certificate to the server without needing the private key?

I'm setting up the callback function with:

void SSL_CTX_set_client_cert_cb(SSL_CTX *ctx, int (*client_cert_cb)(SSL *ssl, X509 **x509, EVP_PKEY **pkey));

Here is a sample of what my code looks like when I set this up:

SSL_CTX_set_client_cert_cb(context, openSSLClientAuthenticationCallBack);

int openSSLClientAuthenticationCallBack(SSL *ssl, X509 **x509, EVP_PKEY **pkey)
{
. . .
}

I can access the Smart Card using the PKCS#11 interface and I'm able to get the certificate and sign it, etc. However, I cannot get the actual private key from the Smart Card.

Does anyone know how I can get around this problem?

Thanks,
George

Michael Wojcik

RE: private key not available for client_cert_cb

In reply to this post by George

> From: openssl-users <[hidden email]> On Behalf Of George
> Sent: Monday, 14 December, 2020 09:36

> I see what you mean. So once I have everything setup, i use the following
> to get the private key:
> EVP_PKEY *pkey = ENGINE_load_private_key(...);
>
> Will pkey actually contain the private key from the smart card?

It had better not.

> I thought it was not possible to get a private key from a smart card?

That's the point of the smartcard (or other HSM), yes.

> Once I have pkey, do I simply use it within the client_cert_cb callback function?

You can't get the private key from the smartcard. Instead, you have to let the engine do the encryption. I don't know what ENGINE_load_private_key actually does - in my PKCS#11 work I didn't have to get into this - but I suspect it just puts a key identifier into pkey.

Then what ought to happen is that you pass that pkey to OpenSSL where you need an EVP_PKEY, and OpenSSL will call the engine's appropriate method for whatever it needs to do, and the engine will tell the smartcard "do this thing using the key with this identifier".

I suggest you refer to a example such as the PPP code that Jan cited to see how it does this sort of thing.

Or you can take the approach that Paul suggests in his reply of writing your own engine specifically for your hardware, if you don't need generic PKCS#11 support. Basically, PKCS#11 gives you support for more devices, and in principle should do some of the work for you; but as Paul suggests, the PKCS#11 API and its dependence on external drivers and libraries means it's not easy to work with. In some cases where you only need to support one type of device (or a family of devices that all use the same driver / library) it might well be easier to just write a simple engine that only supports the features you need. You can use the source for the existing engines in OpenSSL to get an idea of what that looks like.

A few years back I forked the OpenSSL CAPI engine to make some fixes and enhancements, and that was pretty straightforward.

So if you have a well-documented API for your particular smartcard, with handy functions like "do this to get an RSA signature of a blob of data with this key ID and these parameters", you may want to try Paul's route. Really depends on your requirements and what kind of support you already have for your device.

And all of this changes in 3.0 with the new "provider" architecture, so you'll get to take another crack at it soon.

--
Michael Wojcik

George

Re: private key not available for client_cert_cb

Ok, so I am not actually going to populate EVP_PKEY with a private key in the callback function:
int (*client_cert_cb)(SSL *ssl, X509 **x509, EVP_PKEY **pkey)?

Instead, I will call
EVP_PKEY *ENGINE_load_private_key(ENGINE *e, const char *key_id, UI_METHOD *ui_method, void *callback_data);
to get the EVP_PKEY, which will be used by OpenSSL to access the Smart Card.

Once I get the resulting EVP_PKEY using ENGINE_load_private_key(...), how do I assign it to pkey in the callback function?
If I had private key I would use something like
EVP_PKEY_assign_RSA(..)
Since I don't actually have a private key, should I use something else?

Thanks,
George

On 2020-12-14 12:59 p.m., Michael Wojcik wrote:

You can't get the private key from the smartcard. Instead, you have to let the engine do the encryption. I don't know what ENGINE_load_private_key actually does - in my PKCS#11 work I didn't have to get into this - but I suspect it just puts a key identifier into pkey.

Then what ought to happen is that you pass that pkey to OpenSSL where you need an EVP_PKEY, and OpenSSL will call the engine's appropriate method for whatever it needs to do, and the engine will tell the smartcard "do this thing using the key with this identifier".

I suggest you refer to a example such as the PPP code that Jan cited to see how it does this sort of thing.

Or you can take the approach that Paul suggests in his reply of writing your own engine specifically for your hardware, if you don't need generic PKCS#11 support. Basically, PKCS#11 gives you support for more devices, and in principle should do some of the work for you; but as Paul suggests, the PKCS#11 API and its dependence on external drivers and libraries means it's not easy to work with. In some cases where you only need to support one type of device (or a family of devices that all use the same driver / library) it might well be easier to just write a simple engine that only supports the features you need. You can use the source for the existing engines in OpenSSL to get an idea of what that looks like.

A few years back I forked the OpenSSL CAPI engine to make some fixes and enhancements, and that was pretty straightforward.

So if you have a well-documented API for your particular smartcard, with handy functions like "do this to get an RSA signature of a blob of data with this key ID and these parameters", you may want to try Paul's route. Really depends on your requirements and what kind of support you already have for your device.

And all of this changes in 3.0 with the new "provider" architecture, so you'll get to take another crack at it soon.

--
Michael Wojcik

Michael Wojcik

RE: private key not available for client_cert_cb

> From: openssl-users <[hidden email]> On Behalf Of George
> Sent: Monday, 14 December, 2020 13:01

> Once I get the resulting EVP_PKEY using ENGINE_load_private_key(...),
> how do I assign it to pkey in the callback function?

I don't know offhand. As I said in my other message, that's not an area I had to get into when I was working with PKCS#11 some years ago.

My advice is to look at existing examples, such as the code Jan pointed you to.

--
Michael Wojcik

Jan Just Keijser-2

Re: private key not available for client_cert_cb

In reply to this post by George

Hi,

On 14/12/20 21:01, George wrote:

Ok, so I am not actually going to populate EVP_PKEY with a private key in the callback function:
int (*client_cert_cb)(SSL *ssl, X509 **x509, EVP_PKEY **pkey)?

Instead, I will call
EVP_PKEY *ENGINE_load_private_key(ENGINE *e, const char *key_id, UI_METHOD *ui_method, void *callback_data);
to get the EVP_PKEY, which will be used by OpenSSL to access the Smart Card.

Once I get the resulting EVP_PKEY using ENGINE_load_private_key(...), how do I assign it to pkey in the callback function?
If I had private key I would use something like
EVP_PKEY_assign_RSA(..)
Since I don't actually have a private key, should I use something else?

like Michael pointed out, my eap-tls code is just an example of how you could handle a pkcs11 device; it does not us a callback at all, but my code loads the client cert+key upfront and avoids having to use a client callback altogether.

I guess you could also use a client callback for this (perhaps in combination with SSL_CTX_set_client_cert_engine()) . In that case you would get the (pseudo) key from the engine like this
EVP_PKEY *engine_key = ENGINE_load_private_key(ENGINE *e, const char *key_id, UI_METHOD *ui_method, void *callback_data);
and then set
pkey = &engine_key;
and see if that works.
Note that the ENGINE_load_private_key() function *does* return a EVP_PKEY struct but that does not mean the entire private key is contained in it; a private key consists of a modulus and a private part (exponent, prime1, prime2, exponent1, exponent2 etc). the ENGINE_load_private_key() call will return a struct containing the modulus but not the rest. You then use the engine to do the actual encryption and decryption.

HTH,

JJK

Thanks,
George

On 2020-12-14 12:59 p.m., Michael Wojcik wrote:

You can't get the private key from the smartcard. Instead, you have to let the engine do the encryption. I don't know what ENGINE_load_private_key actually does - in my PKCS#11 work I didn't have to get into this - but I suspect it just puts a key identifier into pkey.

Then what ought to happen is that you pass that pkey to OpenSSL where you need an EVP_PKEY, and OpenSSL will call the engine's appropriate method for whatever it needs to do, and the engine will tell the smartcard "do this thing using the key with this identifier".

I suggest you refer to a example such as the PPP code that Jan cited to see how it does this sort of thing.

Or you can take the approach that Paul suggests in his reply of writing your own engine specifically for your hardware, if you don't need generic PKCS#11 support. Basically, PKCS#11 gives you support for more devices, and in principle should do some of the work for you; but as Paul suggests, the PKCS#11 API and its dependence on external drivers and libraries means it's not easy to work with. In some cases where you only need to support one type of device (or a family of devices that all use the same driver / library) it might well be easier to just write a simple engine that only supports the features you need. You can use the source for the existing engines in OpenSSL to get an idea of what that looks like.

A few years back I forked the OpenSSL CAPI engine to make some fixes and enhancements, and that was pretty straightforward.

So if you have a well-documented API for your particular smartcard, with handy functions like "do this to get an RSA signature of a blob of data with this key ID and these parameters", you may want to try Paul's route. Really depends on your requirements and what kind of support you already have for your device.

And all of this changes in 3.0 with the new "provider" architecture, so you'll get to take another crack at it soon.

--
Michael Wojcik

George

Re: private key not available for client_cert_cb

Hi,

   I've been looking at the code in the pppd EAP-TLS patch, but I can't seem to load the engine with the pkcs11 DLL. It is failing with the error:

error:2507606A:DSO support routines:WIN32_BIND_FUNC:could not bind to the requested symbol name

I've verified the path is correct.

I am using OpenSSL1.0.2u with the FIPS Object Module 2.0.16 in Windows 10. Do I need to do anything special to allow loading of DLLs in OpenSSL?

Here is what I am trying to do:

    char* engine_name = "C:\\Users\\whipp\\junk4\\ActivClient\\acpkcs211.dll";
    ENGINE_load_builtin_engines();
    ENGINE_register_all_complete();
    ENGINE *pkey_engine = ENGINE_by_id("dynamic");
    ENGINE_ctrl_cmd_string(pkey_engine, "SO_PATH", engine_name, 0);
    ENGINE_ctrl_cmd_string(pkey_engine, "ID", "pkcs11", 0);
    ENGINE_ctrl_cmd_string(pkey_engine, "LOAD", NULL, 0);

Do you see anything wrong with this?

Thanks,
George

On 2020-12-15 4:38 a.m., Jan Just Keijser wrote:

Hi,

On 14/12/20 21:01, George wrote:

Ok, so I am not actually going to populate EVP_PKEY with a private key in the callback function:
int (*client_cert_cb)(SSL *ssl, X509 **x509, EVP_PKEY **pkey)?

Instead, I will call
EVP_PKEY *ENGINE_load_private_key(ENGINE *e, const char *key_id, UI_METHOD *ui_method, void *callback_data);
to get the EVP_PKEY, which will be used by OpenSSL to access the Smart Card.

Once I get the resulting EVP_PKEY using ENGINE_load_private_key(...), how do I assign it to pkey in the callback function?
If I had private key I would use something like
EVP_PKEY_assign_RSA(..)
Since I don't actually have a private key, should I use something else?

like Michael pointed out, my eap-tls code is just an example of how you could handle a pkcs11 device; it does not us a callback at all, but my code loads the client cert+key upfront and avoids having to use a client callback altogether.

I guess you could also use a client callback for this (perhaps in combination with SSL_CTX_set_client_cert_engine()) . In that case you would get the (pseudo) key from the engine like this
EVP_PKEY *engine_key = ENGINE_load_private_key(ENGINE *e, const char *key_id, UI_METHOD *ui_method, void *callback_data);
and then set
pkey = &engine_key;
and see if that works.
Note that the ENGINE_load_private_key() function *does* return a EVP_PKEY struct but that does not mean the entire private key is contained in it; a private key consists of a modulus and a private part (exponent, prime1, prime2, exponent1, exponent2 etc). the ENGINE_load_private_key() call will return a struct containing the modulus but not the rest. You then use the engine to do the actual encryption and decryption.

HTH,

JJK

Jan Just Keijser-2

Re: private key not available for client_cert_cb

Hi,

On 16/12/20 20:26, George wrote:

Hi,

   I've been looking at the code in the pppd EAP-TLS patch, but I can't seem to load the engine with the pkcs11 DLL. It is failing with the error:

error:2507606A:DSO support routines:WIN32_BIND_FUNC:could not bind to the requested symbol name

I've verified the path is correct.

I am using OpenSSL1.0.2u with the FIPS Object Module 2.0.16 in Windows 10. Do I need to do anything special to allow loading of DLLs in OpenSSL?

Here is what I am trying to do:

    char* engine_name = "C:\\Users\\whipp\\junk4\\ActivClient\\acpkcs211.dll";
    ENGINE_load_builtin_engines();
    ENGINE_register_all_complete();
    ENGINE *pkey_engine = ENGINE_by_id("dynamic");
    ENGINE_ctrl_cmd_string(pkey_engine, "SO_PATH", engine_name, 0);
    ENGINE_ctrl_cmd_string(pkey_engine, "ID", "pkcs11", 0);
    ENGINE_ctrl_cmd_string(pkey_engine, "LOAD", NULL, 0);

Do you see anything wrong with this?

I forgot to mention that loading a PKCS11 driver from within OpenSSL is a 2 stage rocket:

first stage: load the engine_pkcs11 module using

char* engine_name = "C:\\Windows\\System32\\pkcs11.dll"

This is a separate piece of code and is part of the libp11 project:
https://github.com/OpenSC/libp11

(it also has a nice wiki that explains how to do it on the command line using OPENSSL.EXE)

Then create an openssl.cnf section like this:

openssl_conf = openssl_init

[openssl_init]
engines = engine_section

[engine_section]
pkcs11 = pkcs11_section

[pkcs11_section]
engine_id = pkcs11
dynamic_path = "C:\Windows\System32\opensc-pkcs11.dll"
MODULE_path = "C:\Users\whipp\junk4\ActivClient\acpkcs211.dll"
PIN = "0001password"
init = 0

and load that (see the EAP-TLS code for an example or read
https://stackoverflow.com/questions/41119744/pkcs11-engine-for-openssl
for a similar question).

HTH,

JJK

On 2020-12-15 4:38 a.m., Jan Just Keijser wrote:

Hi,

On 14/12/20 21:01, George wrote:

Ok, so I am not actually going to populate EVP_PKEY with a private key in the callback function:
int (*client_cert_cb)(SSL *ssl, X509 **x509, EVP_PKEY **pkey)?

Instead, I will call
EVP_PKEY *ENGINE_load_private_key(ENGINE *e, const char *key_id, UI_METHOD *ui_method, void *callback_data);
to get the EVP_PKEY, which will be used by OpenSSL to access the Smart Card.

Once I get the resulting EVP_PKEY using ENGINE_load_private_key(...), how do I assign it to pkey in the callback function?
If I had private key I would use something like
EVP_PKEY_assign_RSA(..)
Since I don't actually have a private key, should I use something else?

like Michael pointed out, my eap-tls code is just an example of how you could handle a pkcs11 device; it does not us a callback at all, but my code loads the client cert+key upfront and avoids having to use a client callback altogether.

I guess you could also use a client callback for this (perhaps in combination with SSL_CTX_set_client_cert_engine()) . In that case you would get the (pseudo) key from the engine like this
EVP_PKEY *engine_key = ENGINE_load_private_key(ENGINE *e, const char *key_id, UI_METHOD *ui_method, void *callback_data);
and then set
pkey = &engine_key;
and see if that works.
Note that the ENGINE_load_private_key() function *does* return a EVP_PKEY struct but that does not mean the entire private key is contained in it; a private key consists of a modulus and a private part (exponent, prime1, prime2, exponent1, exponent2 etc). the ENGINE_load_private_key() call will return a struct containing the modulus but not the rest. You then use the engine to do the actual encryption and decryption.

HTH,

JJK

George

Re: private key not available for client_cert_cb

Ok. So I use the libp11 project DLL file for the SO_PATH and my smart card middleware DLL for the MODULE_PATH when setting up the OpenSSL Engine?

Thanks,
George

On 2020-12-17 3:22 a.m., Jan Just Keijser wrote:

Hi,

On 16/12/20 20:26, George wrote:

Hi,

   I've been looking at the code in the pppd EAP-TLS patch, but I can't seem to load the engine with the pkcs11 DLL. It is failing with the error:

error:2507606A:DSO support routines:WIN32_BIND_FUNC:could not bind to the requested symbol name

I've verified the path is correct.

I am using OpenSSL1.0.2u with the FIPS Object Module 2.0.16 in Windows 10. Do I need to do anything special to allow loading of DLLs in OpenSSL?

Here is what I am trying to do:

    char* engine_name = "C:\\Users\\whipp\\junk4\\ActivClient\\acpkcs211.dll";
    ENGINE_load_builtin_engines();
    ENGINE_register_all_complete();
    ENGINE *pkey_engine = ENGINE_by_id("dynamic");
    ENGINE_ctrl_cmd_string(pkey_engine, "SO_PATH", engine_name, 0);
    ENGINE_ctrl_cmd_string(pkey_engine, "ID", "pkcs11", 0);
    ENGINE_ctrl_cmd_string(pkey_engine, "LOAD", NULL, 0);

Do you see anything wrong with this?

I forgot to mention that loading a PKCS11 driver from within OpenSSL is a 2 stage rocket:

first stage: load the engine_pkcs11 module using

    char* engine_name = "C:\\Windows\\System32\\pkcs11.dll"

This is a separate piece of code and is part of the libp11 project:
https://github.com/OpenSC/libp11

(it also has a nice wiki that explains how to do it on the command line using OPENSSL.EXE)

Then create an openssl.cnf section like this:
openssl_conf = openssl_init

[openssl_init]
engines = engine_section

[engine_section]
pkcs11 = pkcs11_section

[pkcs11_section]
engine_id = pkcs11
dynamic_path = "C:\Windows\System32\opensc-pkcs11.dll"
MODULE_path = "C:\Users\whipp\junk4\ActivClient\acpkcs211.dll"
PIN = "0001password"
init = 0
and load that (see the EAP-TLS code for an example or read
https://stackoverflow.com/questions/41119744/pkcs11-engine-for-openssl
for a similar question).

HTH,

JJK

On 2020-12-15 4:38 a.m., Jan Just Keijser wrote:

Hi,

On 14/12/20 21:01, George wrote:

Ok, so I am not actually going to populate EVP_PKEY with a private key in the callback function:
int (*client_cert_cb)(SSL *ssl, X509 **x509, EVP_PKEY **pkey)?

Instead, I will call
EVP_PKEY *ENGINE_load_private_key(ENGINE *e, const char *key_id, UI_METHOD *ui_method, void *callback_data);
to get the EVP_PKEY, which will be used by OpenSSL to access the Smart Card.

Once I get the resulting EVP_PKEY using ENGINE_load_private_key(...), how do I assign it to pkey in the callback function?
If I had private key I would use something like
EVP_PKEY_assign_RSA(..)
Since I don't actually have a private key, should I use something else?

like Michael pointed out, my eap-tls code is just an example of how you could handle a pkcs11 device; it does not us a callback at all, but my code loads the client cert+key upfront and avoids having to use a client callback altogether.

I guess you could also use a client callback for this (perhaps in combination with SSL_CTX_set_client_cert_engine()) . In that case you would get the (pseudo) key from the engine like this
EVP_PKEY *engine_key = ENGINE_load_private_key(ENGINE *e, const char *key_id, UI_METHOD *ui_method, void *callback_data);
and then set
pkey = &engine_key;
and see if that works.
Note that the ENGINE_load_private_key() function *does* return a EVP_PKEY struct but that does not mean the entire private key is contained in it; a private key consists of a modulus and a private part (exponent, prime1, prime2, exponent1, exponent2 etc). the ENGINE_load_private_key() call will return a struct containing the modulus but not the rest. You then use the engine to do the actual encryption and decryption.

HTH,

JJK

Jan Just Keijser-2

Re: private key not available for client_cert_cb

On 17/12/20 14:55, George wrote:

Ok. So I use the libp11 project DLL file for the SO_PATH and my smart card middleware DLL for the MODULE_PATH when setting up the OpenSSL Engine?

yes just like in the example I posted below.

I would recommend the p11 wiki page to do it using the command line first - much easier to test & debug.

JJK

On 2020-12-17 3:22 a.m., Jan Just Keijser wrote:
Hi,

On 16/12/20 20:26, George wrote:

Hi,

   I've been looking at the code in the pppd EAP-TLS patch, but I can't seem to load the engine with the pkcs11 DLL. It is failing with the error:

error:2507606A:DSO support routines:WIN32_BIND_FUNC:could not bind to the requested symbol name

I've verified the path is correct.

I am using OpenSSL1.0.2u with the FIPS Object Module 2.0.16 in Windows 10. Do I need to do anything special to allow loading of DLLs in OpenSSL?

Here is what I am trying to do:

    char* engine_name = "C:\\Users\\whipp\\junk4\\ActivClient\\acpkcs211.dll";
    ENGINE_load_builtin_engines();
    ENGINE_register_all_complete();
    ENGINE *pkey_engine = ENGINE_by_id("dynamic");
    ENGINE_ctrl_cmd_string(pkey_engine, "SO_PATH", engine_name, 0);
    ENGINE_ctrl_cmd_string(pkey_engine, "ID", "pkcs11", 0);
    ENGINE_ctrl_cmd_string(pkey_engine, "LOAD", NULL, 0);

Do you see anything wrong with this?

I forgot to mention that loading a PKCS11 driver from within OpenSSL is a 2 stage rocket:

first stage: load the engine_pkcs11 module using

    char* engine_name = "C:\\Windows\\System32\\pkcs11.dll"

This is a separate piece of code and is part of the libp11 project:
https://github.com/OpenSC/libp11

(it also has a nice wiki that explains how to do it on the command line using OPENSSL.EXE)

Then create an openssl.cnf section like this:
openssl_conf = openssl_init

[openssl_init]
engines = engine_section

[engine_section]
pkcs11 = pkcs11_section

[pkcs11_section]
engine_id = pkcs11
dynamic_path = "C:\Windows\System32\opensc-pkcs11.dll"
MODULE_path = "C:\Users\whipp\junk4\ActivClient\acpkcs211.dll"
PIN = "0001password"
init = 0
and load that (see the EAP-TLS code for an example or read
https://stackoverflow.com/questions/41119744/pkcs11-engine-for-openssl
for a similar question).

HTH,

JJK

On 2020-12-15 4:38 a.m., Jan Just Keijser wrote:

Hi,

On 14/12/20 21:01, George wrote:

Ok, so I am not actually going to populate EVP_PKEY with a private key in the callback function:
int (*client_cert_cb)(SSL *ssl, X509 **x509, EVP_PKEY **pkey)?

Instead, I will call
EVP_PKEY *ENGINE_load_private_key(ENGINE *e, const char *key_id, UI_METHOD *ui_method, void *callback_data);
to get the EVP_PKEY, which will be used by OpenSSL to access the Smart Card.

Once I get the resulting EVP_PKEY using ENGINE_load_private_key(...), how do I assign it to pkey in the callback function?
If I had private key I would use something like
EVP_PKEY_assign_RSA(..)
Since I don't actually have a private key, should I use something else?

like Michael pointed out, my eap-tls code is just an example of how you could handle a pkcs11 device; it does not us a callback at all, but my code loads the client cert+key upfront and avoids having to use a client callback altogether.

I guess you could also use a client callback for this (perhaps in combination with SSL_CTX_set_client_cert_engine()) . In that case you would get the (pseudo) key from the engine like this
EVP_PKEY *engine_key = ENGINE_load_private_key(ENGINE *e, const char *key_id, UI_METHOD *ui_method, void *callback_data);
and then set
pkey = &engine_key;
and see if that works.
Note that the ENGINE_load_private_key() function *does* return a EVP_PKEY struct but that does not mean the entire private key is contained in it; a private key consists of a modulus and a private part (exponent, prime1, prime2, exponent1, exponent2 etc). the ENGINE_load_private_key() call will return a struct containing the modulus but not the rest. You then use the engine to do the actual encryption and decryption.

HTH,

JJK

George

Re: private key not available for client_cert_cb

Hi,

I'm able to setup the engine now, but as soon as I attempt to
execute the command
ENGINE_set_default(pkey_engine, ENGINE_METHOD_ALL);
,I see all kinds of middleware exceptions being generated:

Exception thrown at 0x773046D2 in GENCom.exe: Microsoft C++ exception:
unsigned long at memory location 0x07FCFA00.
Exception thrown at 0x773046D2 in GENCom.exe: Microsoft C++ exception:
AI::Middleware::CMWException at memory location 0x032FD2D0.
Exception thrown at 0x773046D2 in GENCom.exe: Microsoft C++ exception:
AI::Middleware::CMWException at memory location 0x032FD2D0.
Exception thrown at 0x773046D2 in GENCom.exe: Microsoft C++ exception:
AI::Middleware::CMWException at memory location 0x032FD2D0.
Exception thrown at 0x773046D2 in GENCom.exe: Microsoft C++ exception:
AI::Middleware::CMWException at memory location 0x032FD2D0.
Exception thrown at 0x773046D2 in GENCom.exe: Microsoft C++ exception:
AI::Middleware::CMWException at memory location 0x032FD2D0.
Exception thrown at 0x773046D2 in GENCom.exe: Microsoft C++ exception:
AI::Middleware::CMWException at memory location 0x032FD2D0.
Exception thrown at 0x773046D2 in GENCom.exe: Microsoft C++ exception:
AI::Middleware::CMWException at memory location 0x032FD2D0.
Exception thrown at 0x773046D2 in GENCom.exe: Microsoft C++ exception:
AI::Middleware::CMWException at memory location 0x032FD2D0.
.
.
.

Do you have any idea what is causing these errors? Am I missing
something in the configuration? When I use the OpenSSL command line
debugger, there are no errors:

OpenSSL> engine -t dynamic -pre
"SO_PATH:C:\\Users\\whipp\\junk4\\libp11-libp11-0.4.11\\src\\pkcs11.dll"
-pre ID:pkcs11 -pre LIST_ADD:1 -pre LOAD -pre "MODULE_PATH:C:\Program
Files (x86)\HID Global\ActivClient\\acpkcs211.dll"
(dynamic) Dynamic engine loading support
[Success]:
SO_PATH:C:\\Users\\whipp\\junk4\\libp11-libp11-0.4.11\\src\\pkcs11.dll
[Success]: ID:pkcs11
[Success]: LIST_ADD:1
[Success]: LOAD
[Success]: MODULE_PATH:C:\Program Files (x86)\HID
Global\ActivClient\\acpkcs211.dll
Loaded: (pkcs11) pkcs11 engine
[ available ]
OpenSSL>

Here is what my simplified code looks like:

char* enginePluginLibrary =
"C:\\Users\\whipp\\junk4\\libp11-libp11-0.4.11\\src\\pkcs11.dll";
char* pkcs11MiddlewareLibrary = "C:\\Program Files (x86)\\HID
Global\\ActivClient\\acpkcs211.dll";
ENGINE_load_builtin_engines();
ENGINE_register_all_complete();
ENGINE *pkey_engine = ENGINE_by_id("dynamic");

ENGINE_ctrl_cmd_string(pkey_engine, "SO_PATH", enginePluginLibrary, 0);
ENGINE_ctrl_cmd_string(pkey_engine, "ID", "pkcs11", 0);
ENGINE_ctrl_cmd_string(pkey_engine, "LIST_ADD", "1", 0);
ENGINE_ctrl_cmd_string(pkey_engine, "LOAD", NULL, 0);
ENGINE_ctrl_cmd_string(pkey_engine, "MODULE_PATH",
pkcs11MiddlewareLibrary, 0);
ENGINE_set_default(pkey_engine, ENGINE_METHOD_ALL);

Thanks!
George

On 2020-12-17 8:39 p.m., Jan Just Keijser wrote:

> On 17/12/20 14:55, George wrote:
>> Ok. So I use the libp11 project DLL file for the SO_PATH and my smart
>> card middleware DLL for the MODULE_PATH when setting up the OpenSSL
>> Engine?
>>
>>
> yes just like in the example I posted below.
>
> I would recommend the p11 wiki page to do it using the command line
> first - much easier to test & debug.
>
> JJK
>

Jan Just Keijser-2

Re: private key not available for client_cert_cb

Hi,

On 18/12/20 06:21, George wrote:

> Hi,
>
> I'm able to setup the engine now, but as soon as I attempt to
> execute the command
> ENGINE_set_default(pkey_engine, ENGINE_METHOD_ALL);
> ,I see all kinds of middleware exceptions being generated:
>
> Exception thrown at 0x773046D2 in GENCom.exe: Microsoft C++ exception:
> unsigned long at memory location 0x07FCFA00.
> Exception thrown at 0x773046D2 in GENCom.exe: Microsoft C++ exception:
> AI::Middleware::CMWException at memory location 0x032FD2D0.
> Exception thrown at 0x773046D2 in GENCom.exe: Microsoft C++ exception:
> AI::Middleware::CMWException at memory location 0x032FD2D0.
> Exception thrown at 0x773046D2 in GENCom.exe: Microsoft C++ exception:
> AI::Middleware::CMWException at memory location 0x032FD2D0.
> Exception thrown at 0x773046D2 in GENCom.exe: Microsoft C++ exception:
> AI::Middleware::CMWException at memory location 0x032FD2D0.
> Exception thrown at 0x773046D2 in GENCom.exe: Microsoft C++ exception:
> AI::Middleware::CMWException at memory location 0x032FD2D0.
> Exception thrown at 0x773046D2 in GENCom.exe: Microsoft C++ exception:
> AI::Middleware::CMWException at memory location 0x032FD2D0.
> Exception thrown at 0x773046D2 in GENCom.exe: Microsoft C++ exception:
> AI::Middleware::CMWException at memory location 0x032FD2D0.
> Exception thrown at 0x773046D2 in GENCom.exe: Microsoft C++ exception:
> AI::Middleware::CMWException at memory location 0x032FD2D0.
> .
> .
> .
>
>
> Do you have any idea what is causing these errors? Am I missing
> something in the configuration? When I use the OpenSSL command line
> debugger, there are no errors:
>
> OpenSSL> engine -t dynamic -pre
> "SO_PATH:C:\\Users\\whipp\\junk4\\libp11-libp11-0.4.11\\src\\pkcs11.dll"
> -pre ID:pkcs11 -pre LIST_ADD:1 -pre LOAD -pre "MODULE_PATH:C:\Program
> Files (x86)\HID Global\ActivClient\\acpkcs211.dll"
> (dynamic) Dynamic engine loading support
> [Success]:
> SO_PATH:C:\\Users\\whipp\\junk4\\libp11-libp11-0.4.11\\src\\pkcs11.dll
> [Success]: ID:pkcs11
> [Success]: LIST_ADD:1
> [Success]: LOAD
> [Success]: MODULE_PATH:C:\Program Files (x86)\HID
> Global\ActivClient\\acpkcs211.dll
> Loaded: (pkcs11) pkcs11 engine
> [ available ]
> OpenSSL>
>
>
> Here is what my simplified code looks like:
>
> char* enginePluginLibrary =
> "C:\\Users\\whipp\\junk4\\libp11-libp11-0.4.11\\src\\pkcs11.dll";
> char* pkcs11MiddlewareLibrary = "C:\\Program Files (x86)\\HID
> Global\\ActivClient\\acpkcs211.dll";
> ENGINE_load_builtin_engines();
> ENGINE_register_all_complete();
> ENGINE *pkey_engine = ENGINE_by_id("dynamic");
>
> ENGINE_ctrl_cmd_string(pkey_engine, "SO_PATH", enginePluginLibrary, 0);
> ENGINE_ctrl_cmd_string(pkey_engine, "ID", "pkcs11", 0);
> ENGINE_ctrl_cmd_string(pkey_engine, "LIST_ADD", "1", 0);
> ENGINE_ctrl_cmd_string(pkey_engine, "LOAD", NULL, 0);
> ENGINE_ctrl_cmd_string(pkey_engine, "MODULE_PATH",
> pkcs11MiddlewareLibrary, 0);
> ENGINE_set_default(pkey_engine, ENGINE_METHOD_ALL);
>
>

main difference between the OPENSSL.EXE example and your code is that
last call:

here's wat "ENGINE_set_default(pkey_engine, ENGINE_METHOD_ALL)" does:

int ENGINE_set_default(ENGINE *e, unsigned int flags)
{
     if ((flags & ENGINE_METHOD_CIPHERS) && !ENGINE_set_default_ciphers(e))
         return 0;
     if ((flags & ENGINE_METHOD_DIGESTS) && !ENGINE_set_default_digests(e))
         return 0;
#ifndef OPENSSL_NO_RSA
     if ((flags & ENGINE_METHOD_RSA) && !ENGINE_set_default_RSA(e))
         return 0;
#endif
#ifndef OPENSSL_NO_DSA
     if ((flags & ENGINE_METHOD_DSA) && !ENGINE_set_default_DSA(e))
         return 0;
#endif
#ifndef OPENSSL_NO_DH
     if ((flags & ENGINE_METHOD_DH) && !ENGINE_set_default_DH(e))
         return 0;
#endif
#ifndef OPENSSL_NO_ECDH
     if ((flags & ENGINE_METHOD_ECDH) && !ENGINE_set_default_ECDH(e))
         return 0;
#endif
#ifndef OPENSSL_NO_ECDSA
     if ((flags & ENGINE_METHOD_ECDSA) && !ENGINE_set_default_ECDSA(e))
         return 0;
#endif
     if ((flags & ENGINE_METHOD_RAND) && !ENGINE_set_default_RAND(e))
         return 0;
     if ((flags & ENGINE_METHOD_PKEY_METHS)
         && !ENGINE_set_default_pkey_meths(e))
         return 0;
     if ((flags & ENGINE_METHOD_PKEY_ASN1_METHS)
         && !ENGINE_set_default_pkey_asn1_meths(e))
         return 0;
     return 1;
}

(from the openssl 1.0.2 source tree)
It could be that one of those methods is not throwing the errors with
your smart card.
I'd advise you to test your smart card capabilities . It might also be
useful to do more command line testing with your smartcard using

   engine -vvvv -t dynamic -pre
"SO_PATH:C:\\Users\\whipp\\junk4\\libp11-libp11-0.4.11\\src\\pkcs11.dll"
-pre ID:pkcs11 -pre LIST_ADD:1 -pre LOAD -pre "MODULE_PATH:C:\Program
Files (x86)\HID Global\ActivClient\\acpkcs211.dll"

and then try out certain operations, like encrypt/decrypt or simply use
the command
   speed

and watch for any errors - that should give you a hint which method is
not supported by your smart card.

HTH,

JJK

>
> On 2020-12-17 8:39 p.m., Jan Just Keijser wrote:
>> On 17/12/20 14:55, George wrote:
>>> Ok. So I use the libp11 project DLL file for the SO_PATH and my
>>> smart card middleware DLL for the MODULE_PATH when setting up the
>>> OpenSSL Engine?
>>>
>>>
>> yes just like in the example I posted below.
>>
>> I would recommend the p11 wiki page to do it using the command line
>> first - much easier to test & debug.
>>
>> JJK
>>

George

Re: private key not available for client_cert_cb

Hi,

I narrowed the problem down to
ENGINE_set_default(pkey_engine, ENGINE_METHOD_ALL)

This causes the initial exception
Exception thrown at 0x757346D2 in GENCom.exe: Microsoft C++ exception: unsigned long at memory location 0x006FCD68.

It looks like some of the Engine methods cause an exception, but not all of them:

Works:
ENGINE_METHOD_CIPHERS
ENGINE_METHOD_DIGESTS
ENGINE_METHOD_DSA
ENGINE_METHOD_DH
ENGINE_METHOD_RAND
ENGINE_METHOD_PKEY_ASN1_METHS

Causes An Exception:
ENGINE_METHOD_RSA
ENGINE_METHOD_ECDH
ENGINE_METHOD_ECDSA
ENGINE_METHOD_PKEY_METHS

Is that normal behaviour, or is something wrong? Is there a way to find the supported engine methods to avoid triggering an exception?

It seems like alot of other smaple code I have looked at calls
ENGINE_init(pkey_engine);

Is the needed? When I call it, it always returns with "0". Should it be returning with "1"?

I did some testing in the OpenSSL command line, and here is what I found:

- The command line "speed" test appears to be fine:

OpenSSL> speed -engine pkcs11
engine "pkcs11" set.
Doing mdc2 for 3s on 16 size blocks: 2688737 mdc2's in 2.98s
Doing mdc2 for 3s on 64 size blocks: 880529 mdc2's in 3.00s
Doing mdc2 for 3s on 256 size blocks: 240916 mdc2's in 2.98s
Doing mdc2 for 3s on 1024 size blocks: 61287 mdc2's in 3.00s
Doing mdc2 for 3s on 8192 size blocks: 7774 mdc2's in 2.98s
.
.
.

- I also tried the following, which successfully created the PEM files:

OpenSSL> req -engine pkcs11 -new -key "pkcs11:object=Authentication - *;type=private;pin-value=123456" -keyform engine -out req2.pem -text -x509 -subj "/CN=*"
OpenSSL> x509 -engine pkcs11 -signkey "pkcs11:object=Authentication - *;type=private;pin-value=123456" -keyform engine -in req2.pem -out cert2.pem

Thanks,
George

On 2020-12-18 3:40 a.m., Jan Just Keijser wrote:

Hi,

On 18/12/20 06:21, George wrote:

Hi,

   I'm able to setup the engine now, but as soon as I attempt to execute the command
ENGINE_set_default(pkey_engine, ENGINE_METHOD_ALL);
,I see all kinds of middleware exceptions being generated:

Exception thrown at 0x773046D2 in GENCom.exe: Microsoft C++ exception: unsigned long at memory location 0x07FCFA00.
Exception thrown at 0x773046D2 in GENCom.exe: Microsoft C++ exception: AI::Middleware::CMWException at memory location 0x032FD2D0.
Exception thrown at 0x773046D2 in GENCom.exe: Microsoft C++ exception: AI::Middleware::CMWException at memory location 0x032FD2D0.
Exception thrown at 0x773046D2 in GENCom.exe: Microsoft C++ exception: AI::Middleware::CMWException at memory location 0x032FD2D0.
Exception thrown at 0x773046D2 in GENCom.exe: Microsoft C++ exception: AI::Middleware::CMWException at memory location 0x032FD2D0.
Exception thrown at 0x773046D2 in GENCom.exe: Microsoft C++ exception: AI::Middleware::CMWException at memory location 0x032FD2D0.
Exception thrown at 0x773046D2 in GENCom.exe: Microsoft C++ exception: AI::Middleware::CMWException at memory location 0x032FD2D0.
Exception thrown at 0x773046D2 in GENCom.exe: Microsoft C++ exception: AI::Middleware::CMWException at memory location 0x032FD2D0.
Exception thrown at 0x773046D2 in GENCom.exe: Microsoft C++ exception: AI::Middleware::CMWException at memory location 0x032FD2D0.
.
.
.

Do you have any idea what is causing these errors? Am I missing something in the configuration? When I use the OpenSSL command line debugger, there are no errors:

OpenSSL> engine -t dynamic -pre "SO_PATH:C:\\Users\\whipp\\junk4\\libp11-libp11-0.4.11\\src\\pkcs11.dll" -pre ID:pkcs11 -pre LIST_ADD:1 -pre LOAD -pre "MODULE_PATH:C:\Program Files (x86)\HID Global\ActivClient\\acpkcs211.dll"
(dynamic) Dynamic engine loading support
[Success]: SO_PATH:C:\\Users\\whipp\\junk4\\libp11-libp11-0.4.11\\src\\pkcs11.dll
[Success]: ID:pkcs11
[Success]: LIST_ADD:1
[Success]: LOAD
[Success]: MODULE_PATH:C:\Program Files (x86)\HID Global\ActivClient\\acpkcs211.dll
Loaded: (pkcs11) pkcs11 engine
     [ available ]
OpenSSL>

Here is what my simplified code looks like:

char* enginePluginLibrary = "C:\\Users\\whipp\\junk4\\libp11-libp11-0.4.11\\src\\pkcs11.dll";
char* pkcs11MiddlewareLibrary = "C:\\Program Files (x86)\\HID Global\\ActivClient\\acpkcs211.dll";
ENGINE_load_builtin_engines();
ENGINE_register_all_complete();
ENGINE *pkey_engine = ENGINE_by_id("dynamic");

ENGINE_ctrl_cmd_string(pkey_engine, "SO_PATH", enginePluginLibrary, 0);
ENGINE_ctrl_cmd_string(pkey_engine, "ID", "pkcs11", 0);
ENGINE_ctrl_cmd_string(pkey_engine, "LIST_ADD", "1", 0);
ENGINE_ctrl_cmd_string(pkey_engine, "LOAD", NULL, 0);
ENGINE_ctrl_cmd_string(pkey_engine, "MODULE_PATH", pkcs11MiddlewareLibrary, 0);
ENGINE_set_default(pkey_engine, ENGINE_METHOD_ALL);

main difference between the OPENSSL.EXE example and your code is that last call:

here's wat "ENGINE_set_default(pkey_engine, ENGINE_METHOD_ALL)" does:

int ENGINE_set_default(ENGINE *e, unsigned int flags)
{
    if ((flags & ENGINE_METHOD_CIPHERS) && !ENGINE_set_default_ciphers(e))
        return 0;
    if ((flags & ENGINE_METHOD_DIGESTS) && !ENGINE_set_default_digests(e))
        return 0;
#ifndef OPENSSL_NO_RSA
    if ((flags & ENGINE_METHOD_RSA) && !ENGINE_set_default_RSA(e))
        return 0;
#endif
#ifndef OPENSSL_NO_DSA
    if ((flags & ENGINE_METHOD_DSA) && !ENGINE_set_default_DSA(e))
        return 0;
#endif
#ifndef OPENSSL_NO_DH
    if ((flags & ENGINE_METHOD_DH) && !ENGINE_set_default_DH(e))
        return 0;
#endif
#ifndef OPENSSL_NO_ECDH
    if ((flags & ENGINE_METHOD_ECDH) && !ENGINE_set_default_ECDH(e))
        return 0;
#endif
#ifndef OPENSSL_NO_ECDSA
    if ((flags & ENGINE_METHOD_ECDSA) && !ENGINE_set_default_ECDSA(e))
        return 0;
#endif
    if ((flags & ENGINE_METHOD_RAND) && !ENGINE_set_default_RAND(e))
        return 0;
    if ((flags & ENGINE_METHOD_PKEY_METHS)
        && !ENGINE_set_default_pkey_meths(e))
        return 0;
    if ((flags & ENGINE_METHOD_PKEY_ASN1_METHS)
        && !ENGINE_set_default_pkey_asn1_meths(e))
        return 0;
    return 1;
}

(from the openssl 1.0.2 source tree)
It could be that one of those methods is not throwing the errors with your smart card.
I'd advise you to test your smart card capabilities . It might also be useful to do more command line testing with your smartcard using

engine -vvvv -t dynamic -pre "SO_PATH:C:\\Users\\whipp\\junk4\\libp11-libp11-0.4.11\\src\\pkcs11.dll" -pre ID:pkcs11 -pre LIST_ADD:1 -pre LOAD -pre "MODULE_PATH:C:\Program Files (x86)\HID Global\ActivClient\\acpkcs211.dll"

and then try out certain operations, like encrypt/decrypt or simply use the command
speed

and watch for any errors - that should give you a hint which method is not supported by your smart card.

HTH,

JJK

Jan Just Keijser-2

Re: private key not available for client_cert_cb

Hi,

On 19/12/20 04:48, George wrote:

Hi,

I narrowed the problem down to
ENGINE_set_default(pkey_engine, ENGINE_METHOD_ALL)

This causes the initial exception
Exception thrown at 0x757346D2 in GENCom.exe: Microsoft C++ exception: unsigned long at memory location 0x006FCD68.

It looks like some of the Engine methods cause an exception, but not all of them:

Works:
ENGINE_METHOD_CIPHERS
ENGINE_METHOD_DIGESTS
ENGINE_METHOD_DSA
ENGINE_METHOD_DH
ENGINE_METHOD_RAND
ENGINE_METHOD_PKEY_ASN1_METHS

Causes An Exception:
ENGINE_METHOD_RSA
ENGINE_METHOD_ECDH
ENGINE_METHOD_ECDSA
ENGINE_METHOD_PKEY_METHS

Is that normal behaviour, or is something wrong? Is there a way to find the supported engine methods to avoid triggering an exception?

I'd say no engine/pkcs11 module should trigger exceptions - that's an error in the pkcs11 module.

Something you can try is this:

run the 'openssl.exe' command:

openssl engine -t dynamic -pre "SO_PATH:C:\\Users\\whipp\\junk4\\libp11-libp11-0.4.11\\src\\pkcs11.dll" -pre ID:pkcs11 -pre LIST_ADD:1 -pre LOAD -pre "MODULE_PATH:C:\Program Files (x86)\HID Global\ActivClient\\acpkcs211.dll"

then on the OpenSSL prompt , try

s_client -keyform engine -key 0:<key-id> -cert "clientcert.pem" -connect remote_host:remote_port

that should start a TLS connection and use the pcks11 engine to ask for the key , identified by <key-id> in slot 0 (adjust the slot number if your smart card starts at number 1 etc.

HTH,

JJK

It seems like alot of other smaple code I have looked at calls
ENGINE_init(pkey_engine);

Is the needed? When I call it, it always returns with "0". Should it be returning with "1"?

I did some testing in the OpenSSL command line, and here is what I found:

- The command line "speed" test appears to be fine:

OpenSSL> speed -engine pkcs11
engine "pkcs11" set.
Doing mdc2 for 3s on 16 size blocks: 2688737 mdc2's in 2.98s
Doing mdc2 for 3s on 64 size blocks: 880529 mdc2's in 3.00s
Doing mdc2 for 3s on 256 size blocks: 240916 mdc2's in 2.98s
Doing mdc2 for 3s on 1024 size blocks: 61287 mdc2's in 3.00s
Doing mdc2 for 3s on 8192 size blocks: 7774 mdc2's in 2.98s
.
.
.

- I also tried the following, which successfully created the PEM files:

OpenSSL> req -engine pkcs11 -new -key "pkcs11:object=Authentication - *;type=private;pin-value=123456" -keyform engine -out req2.pem -text -x509 -subj "/CN=*"
OpenSSL> x509 -engine pkcs11 -signkey "pkcs11:object=Authentication - *;type=private;pin-value=123456" -keyform engine -in req2.pem -out cert2.pem

Thanks,
George

On 2020-12-18 3:40 a.m., Jan Just Keijser wrote:

Hi,

On 18/12/20 06:21, George wrote:

Hi,

   I'm able to setup the engine now, but as soon as I attempt to execute the command
ENGINE_set_default(pkey_engine, ENGINE_METHOD_ALL);
,I see all kinds of middleware exceptions being generated:

Exception thrown at 0x773046D2 in GENCom.exe: Microsoft C++ exception: unsigned long at memory location 0x07FCFA00.
Exception thrown at 0x773046D2 in GENCom.exe: Microsoft C++ exception: AI::Middleware::CMWException at memory location 0x032FD2D0.
Exception thrown at 0x773046D2 in GENCom.exe: Microsoft C++ exception: AI::Middleware::CMWException at memory location 0x032FD2D0.
Exception thrown at 0x773046D2 in GENCom.exe: Microsoft C++ exception: AI::Middleware::CMWException at memory location 0x032FD2D0.
Exception thrown at 0x773046D2 in GENCom.exe: Microsoft C++ exception: AI::Middleware::CMWException at memory location 0x032FD2D0.
Exception thrown at 0x773046D2 in GENCom.exe: Microsoft C++ exception: AI::Middleware::CMWException at memory location 0x032FD2D0.
Exception thrown at 0x773046D2 in GENCom.exe: Microsoft C++ exception: AI::Middleware::CMWException at memory location 0x032FD2D0.
Exception thrown at 0x773046D2 in GENCom.exe: Microsoft C++ exception: AI::Middleware::CMWException at memory location 0x032FD2D0.
Exception thrown at 0x773046D2 in GENCom.exe: Microsoft C++ exception: AI::Middleware::CMWException at memory location 0x032FD2D0.
.
.
.

Do you have any idea what is causing these errors? Am I missing something in the configuration? When I use the OpenSSL command line debugger, there are no errors:

OpenSSL> engine -t dynamic -pre "SO_PATH:C:\\Users\\whipp\\junk4\\libp11-libp11-0.4.11\\src\\pkcs11.dll" -pre ID:pkcs11 -pre LIST_ADD:1 -pre LOAD -pre "MODULE_PATH:C:\Program Files (x86)\HID Global\ActivClient\\acpkcs211.dll"
(dynamic) Dynamic engine loading support
[Success]: SO_PATH:C:\\Users\\whipp\\junk4\\libp11-libp11-0.4.11\\src\\pkcs11.dll
[Success]: ID:pkcs11
[Success]: LIST_ADD:1
[Success]: LOAD
[Success]: MODULE_PATH:C:\Program Files (x86)\HID Global\ActivClient\\acpkcs211.dll
Loaded: (pkcs11) pkcs11 engine
     [ available ]
OpenSSL>

Here is what my simplified code looks like:

char* enginePluginLibrary = "C:\\Users\\whipp\\junk4\\libp11-libp11-0.4.11\\src\\pkcs11.dll";
char* pkcs11MiddlewareLibrary = "C:\\Program Files (x86)\\HID Global\\ActivClient\\acpkcs211.dll";
ENGINE_load_builtin_engines();
ENGINE_register_all_complete();
ENGINE *pkey_engine = ENGINE_by_id("dynamic");

ENGINE_ctrl_cmd_string(pkey_engine, "SO_PATH", enginePluginLibrary, 0);
ENGINE_ctrl_cmd_string(pkey_engine, "ID", "pkcs11", 0);
ENGINE_ctrl_cmd_string(pkey_engine, "LIST_ADD", "1", 0);
ENGINE_ctrl_cmd_string(pkey_engine, "LOAD", NULL, 0);
ENGINE_ctrl_cmd_string(pkey_engine, "MODULE_PATH", pkcs11MiddlewareLibrary, 0);
ENGINE_set_default(pkey_engine, ENGINE_METHOD_ALL);

main difference between the OPENSSL.EXE example and your code is that last call:

here's wat "ENGINE_set_default(pkey_engine, ENGINE_METHOD_ALL)" does:

int ENGINE_set_default(ENGINE *e, unsigned int flags)
{
    if ((flags & ENGINE_METHOD_CIPHERS) && !ENGINE_set_default_ciphers(e))
        return 0;
    if ((flags & ENGINE_METHOD_DIGESTS) && !ENGINE_set_default_digests(e))
        return 0;
#ifndef OPENSSL_NO_RSA
    if ((flags & ENGINE_METHOD_RSA) && !ENGINE_set_default_RSA(e))
        return 0;
#endif
#ifndef OPENSSL_NO_DSA
    if ((flags & ENGINE_METHOD_DSA) && !ENGINE_set_default_DSA(e))
        return 0;
#endif
#ifndef OPENSSL_NO_DH
    if ((flags & ENGINE_METHOD_DH) && !ENGINE_set_default_DH(e))
        return 0;
#endif
#ifndef OPENSSL_NO_ECDH
    if ((flags & ENGINE_METHOD_ECDH) && !ENGINE_set_default_ECDH(e))
        return 0;
#endif
#ifndef OPENSSL_NO_ECDSA
    if ((flags & ENGINE_METHOD_ECDSA) && !ENGINE_set_default_ECDSA(e))
        return 0;
#endif
    if ((flags & ENGINE_METHOD_RAND) && !ENGINE_set_default_RAND(e))
        return 0;
    if ((flags & ENGINE_METHOD_PKEY_METHS)
        && !ENGINE_set_default_pkey_meths(e))
        return 0;
    if ((flags & ENGINE_METHOD_PKEY_ASN1_METHS)
        && !ENGINE_set_default_pkey_asn1_meths(e))
        return 0;
    return 1;
}

(from the openssl 1.0.2 source tree)
It could be that one of those methods is not throwing the errors with your smart card.
I'd advise you to test your smart card capabilities . It might also be useful to do more command line testing with your smartcard using

engine -vvvv -t dynamic -pre "SO_PATH:C:\\Users\\whipp\\junk4\\libp11-libp11-0.4.11\\src\\pkcs11.dll" -pre ID:pkcs11 -pre LIST_ADD:1 -pre LOAD -pre "MODULE_PATH:C:\Program Files (x86)\HID Global\ActivClient\\acpkcs211.dll"

and then try out certain operations, like encrypt/decrypt or simply use the command
speed

and watch for any errors - that should give you a hint which method is not supported by your smart card.

HTH,

JJK

George

Re: private key not available for client_cert_cb

Hi,

I tried running the "s_client" command and it appears to be working.

I guess there must be something wrong in my code.
My crash occurs when I call

ENGINE_init(pkey_engine);

I notice your code does not call this function. Is this needed needed? If so, when/where should it be called?

What exactly is the definition of "pkey_identifier" in

ENGINE_load_private_key(pkey_engine, pkey_identifier, transfer_pin, &cb_data) ?

I'm not clear on what this value should be. Can you give an example of what it would look like?

I have the following on my smart card:

Private Key Object; RSA
label:      Authentication - *
ID:         2b2586c684d69b670c0a805edf514e720f2b757d8e2faa0b3a7ff23d1ccfc7ba
Usage:      unwrap
Access:     sensitive, never extractable
Allowed mechanisms: RSA-PKCS,RSA-X-509

Would the pkey_identifier be the ID in the above?

What exactly is "prompt_info" in the structure PW_CB_DATA?
i.e.
typedef struct pw_cb_data {
const void* password;
const char* prompt_info;
} PW_CB_DATA;
Can you give an example of what it might look like?

Is the value of cb_data populated by the transfer_pin callback functions, or should it already contain a value when ENGINE_load_private_key is called?

Is there a way to skip the callback transfer_pin and use a hard coded pin for test purposes when calling ENGINE_load_private_key(...)?

Thanks!
George

On 2020-12-19 8:05 p.m., Jan Just Keijser wrote:

I'd say no engine/pkcs11 module should trigger exceptions - that's an error in the pkcs11 module.

Something you can try is this:

run the 'openssl.exe' command:

openssl engine -t dynamic -pre "SO_PATH:C:\\Users\\whipp\\junk4\\libp11-libp11-0.4.11\\src\\pkcs11.dll" -pre ID:pkcs11 -pre LIST_ADD:1 -pre LOAD -pre "MODULE_PATH:C:\Program Files (x86)\HID Global\ActivClient\\acpkcs211.dll"

then on the OpenSSL prompt , try

s_client -keyform engine -key 0:<key-id> -cert "clientcert.pem" -connect remote_host:remote_port

that should start a TLS connection and use the pcks11 engine to ask for the key , identified by <key-id> in slot 0 (adjust the slot number if your smart card starts at number 1 etc.

HTH,

JJK

Jan Just Keijser-2

Re: private key not available for client_cert_cb

Hi,

On 20/12/20 09:39, George wrote:

Hi,

I tried running the "s_client" command and it appears to be working.

I guess there must be something wrong in my code.

it is good news that the s_client command is working - it means there is something wrong with your code but you have everything at hand to fix it: download the openssl 1.0.2 tarball / zip file and look for the files
apps/s_client.c
apps/apps.c

that contains all of the code that the 's_client' command uses to make a connection and my bet is that is also does not call ENGINE_init

My crash occurs when I call

ENGINE_init(pkey_engine);

I notice your code does not call this function. Is this needed needed? If so, when/where should it be called?

tbh, I don't know - look through the openssl sources to see what it does, exactly.

What exactly is the definition of "pkey_identifier" in

ENGINE_load_private_key(pkey_engine, pkey_identifier, transfer_pin, &cb_data) ?

I'm not clear on what this value should be. Can you give an example of what it would look like?

I have the following on my smart card:

Private Key Object; RSA
label:      Authentication - *
ID:         2b2586c684d69b670c0a805edf514e720f2b757d8e2faa0b3a7ff23d1ccfc7ba
Usage:      unwrap
Access:     sensitive, never extractable
Allowed mechanisms: RSA-PKCS,RSA-X-509

Would the pkey_identifier be the ID in the above?

yes, although if you have multiple smartcards inserted at the same time then it helps to add the slot number, e.g.
0:<ID>

What exactly is "prompt_info" in the structure PW_CB_DATA?
i.e.
typedef struct pw_cb_data {
const void* password;
const char* prompt_info;
} PW_CB_DATA;
Can you give an example of what it might look like?

Is the value of cb_data populated by the transfer_pin callback functions, or should it already contain a value when ENGINE_load_private_key is called?

Is there a way to skip the callback transfer_pin and use a hard coded pin for test purposes when calling ENGINE_load_private_key(...)?

my eap-tls code does just that: if the password is specified in the ppp config file then the user is not prompted:

    if (pkey_engine)
    {
        EVP_PKEY   *pkey = NULL;
        PW_CB_DATA cb_data;
        UI_METHOD* transfer_pin = NULL;

        cb_data.password = passwd;
        cb_data.prompt_info = pkey_identifier;

HTH,

JJK

On 2020-12-19 8:05 p.m., Jan Just Keijser wrote:

I'd say no engine/pkcs11 module should trigger exceptions - that's an error in the pkcs11 module.

Something you can try is this:

run the 'openssl.exe' command:

openssl engine -t dynamic -pre "SO_PATH:C:\\Users\\whipp\\junk4\\libp11-libp11-0.4.11\\src\\pkcs11.dll" -pre ID:pkcs11 -pre LIST_ADD:1 -pre LOAD -pre "MODULE_PATH:C:\Program Files (x86)\HID Global\ActivClient\\acpkcs211.dll"

then on the OpenSSL prompt , try

s_client -keyform engine -key 0:<key-id> -cert "clientcert.pem" -connect remote_host:remote_port

that should start a TLS connection and use the pcks11 engine to ask for the key , identified by <key-id> in slot 0 (adjust the slot number if your smart card starts at number 1 etc.

HTH,

JJK