private key not available for client_cert_cb

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
30 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Re: private key not available for client_cert_cb

George
Hi,

    I was looking at the  code in https://github.com/jjkeijser/ppp/blob/eap-tls/pppd/eap-tls.c and realized I forgot to call ENGINE_ctrl_cmd(...) to setup "LOAD_CERT_CTRL". However, when I do this, the callback function is no longer being called during the mutual authentication handshake. I'm wondering if I have the parameter "cert_info.s_slot_cert_id" incorrectly configured. Here is what my code looks like:
   
struct
{
   const char* s_slot_cert_id;
   X509* cert;
} cert_info;
cert_info.s_slot_cert_id = "a9bee4d72100c52f77c3fc288d2be01a34b5d44f91b3b7ea3d349b8a25752c45";
cert_info.cert = NULL;

ENGINE_ctrl_cmd(engine, "LOAD_CERT_CTRL", 0, &cert_info, NULL, 0);
SSL_CTX_use_certificate(sslContext, cert_info.cert);

I tried manually using LOAD_CERT_CTRL in the openssl shell but I cannot seem to get it to work and cannot find any examples of how to use it.  Is the syntax for LOAD_CERT_CTRL correct? I am using "LOAD_CERT_CTRL:<certificate Object ID>".

OpenSSL> engine -vvvv -t dynamic -pre "SO_PATH:C:\\Users\\whipp\\junk4\\libp11-libp11-0.4.11\\src\\pkcs11.dll" -pre ID:pkcs11 -pre LIST_ADD:1 -pre LOAD -pre "MODULE_PATH:C:\Program Files (x86)\HID Global\ActivClient\\acpkcs211.dll" -pre PIN:123456 -pre FORCE_LOGIN -pre "LOAD_CERT_CTRL:a9bee4d72100c52f77c3fc288d2be01a34b5d44f91b3b7ea3d349b8a25752c45"

(dynamic) Dynamic engine loading support
[Success]: SO_PATH:C:\\Users\\whipp\\junk4\\libp11-libp11-0.4.11\\src\\pkcs11.dll
[Success]: ID:pkcs11
[Success]: LIST_ADD:1
[Success]: LOAD
[Success]: MODULE_PATH:C:\Program Files (x86)\HID Global\ActivClient\\acpkcs211.dll
[Success]: PIN:123456
[Success]: FORCE_LOGIN
[Failure]: LOAD_CERT_CTRL:a9bee4d72100c52f77c3fc288d2be01a34b5d44f91b3b7ea3d349b8a25752c45
4196:error:260AB086:engine routines:ENGINE_ctrl_cmd_string:cmd not executable:.\crypto\engine\eng_ctrl.c:316:
Loaded: (pkcs11) pkcs11 engine
     [ available ]
     SO_PATH: Specifies the path to the 'pkcs11' engine shared library
          (input flags): STRING
     MODULE_PATH: Specifies the path to the PKCS#11 module shared library
          (input flags): STRING
     PIN: Specifies the pin code
          (input flags): STRING
     VERBOSE: Print additional details
          (input flags): NO_INPUT
     QUIET: Remove additional details
          (input flags): NO_INPUT
     LOAD_CERT_CTRL: Get the certificate from card
          (input flags): [Internal]
     INIT_ARGS: Specifies additional initialization arguments to the PKCS#11 module
          (input flags): STRING
     SET_USER_INTERFACE: Set the global user interface (internal)
          (input flags): [Internal]
     SET_CALLBACK_DATA: Set the global user interface extra data (internal)
          (input flags): [Internal]
     FORCE_LOGIN: Force login to the PKCS#11 module
          (input flags): NO_INPUT
OpenSSL>

I'm using the certificate object ID "a9bee4d72100c52f77c3fc288d2be01a34b5d44f91b3b7ea3d349b8a25752c45" for LOAD_CERT_CTRL. Is this right? (I also tried adding "0:" in front of it to indicate slot 0, but that did not work either.

C:\Program Files\OpenSC Project\OpenSC\tools>pkcs11-tool --module="C:\Program Files\HID Global\ActivClient/acpkcs211.dll" -l -O
Using slot 0 with a present token (0x0)
.
.
.
Certificate Object; type = X.509 cert
  label:      Card Authentication - PIVKey E7F4FBE4644BA647ADDBE261BE596757
  subject:    DN: CN=PIVKey E7F4FBE4644BA647ADDBE261BE596757
  ID:         a9bee4d72100c52f77c3fc288d2be01a34b5d44f91b3b7ea3d349b8a25752c45



Thanks,
George


On 2020-12-23 6:00 a.m., Jan Just Keijser wrote:
Hi,

On 20/12/20 09:39, George wrote:
Hi,

   I tried running the "s_client" command and it appears to be working.

I guess there must be something wrong in my code.

it is good news that the s_client command is working - it means there is something wrong with your code but you have everything at hand to fix it: download the openssl 1.0.2 tarball / zip file and look for the files
  apps/s_client.c
  apps/apps.c 

that contains all of the code that the 's_client' command uses to make a connection and my bet is that is also does not call ENGINE_init
My crash occurs when I call
ENGINE_init(pkey_engine);
I notice your code does not call this function.  Is this needed needed? If so, when/where should it be called?

tbh,  I don't know - look through the openssl sources to see what it does, exactly.

What exactly is the definition of "pkey_identifier" in
ENGINE_load_private_key(pkey_engine, pkey_identifier, transfer_pin, &cb_data) ?

I'm not clear on what this value should be. Can you give an example of what it would look like?

I have the following on my smart card:
Private Key Object; RSA
  label:      Authentication - *
  ID:         2b2586c684d69b670c0a805edf514e720f2b757d8e2faa0b3a7ff23d1ccfc7ba
  Usage:      unwrap
  Access:     sensitive, never extractable
  Allowed mechanisms: RSA-PKCS,RSA-X-509

Would the pkey_identifier be the ID in the above?

yes, although if you have multiple smartcards inserted at the same time then it helps to add the slot number, e.g.
  0:<ID>



What exactly is "prompt_info" in the structure PW_CB_DATA?
i.e.
typedef struct pw_cb_data {
    const void* password;
    const char* prompt_info;
} PW_CB_DATA;

Can you give an example of what it might look like?

Is the value of cb_data populated by the transfer_pin callback functions, or should it already contain a value when  ENGINE_load_private_key is called?

Is there a way to skip the callback transfer_pin and use a hard coded pin for test purposes when calling ENGINE_load_private_key(...)?

my eap-tls code does just that: if the password is specified in  the ppp config file then the user is not prompted:

    if (pkey_engine)
    {   
        EVP_PKEY   *pkey = NULL;
        PW_CB_DATA  cb_data;
        UI_METHOD* transfer_pin = NULL;

        cb_data.password = passwd;
        cb_data.prompt_info = pkey_identifier;


HTH,

JJK

On 2020-12-19 8:05 p.m., Jan Just Keijser wrote:

I'd say no engine/pkcs11 module should trigger exceptions - that's an error in the pkcs11 module.

Something you can try is this:

run the 'openssl.exe' command:

openssl engine -t dynamic -pre "SO_PATH:C:\\Users\\whipp\\junk4\\libp11-libp11-0.4.11\\src\\pkcs11.dll" -pre ID:pkcs11 -pre LIST_ADD:1 -pre LOAD -pre "MODULE_PATH:C:\Program Files (x86)\HID Global\ActivClient\\acpkcs211.dll"

then on the OpenSSL prompt , try

  s_client -keyform engine -key 0:<key-id>  -cert  "clientcert.pem"  -connect remote_host:remote_port

that should start a TLS connection and use the pcks11 engine to ask for the key , identified by <key-id> in slot 0 (adjust the slot number if your smart card starts at number 1 etc.

HTH,

JJK





Reply | Threaded
Open this post in threaded view
|

Re: private key not available for client_cert_cb

Jan Just Keijser-2
Hi,

On 05/01/21 07:39, George wrote:
Hi,

    I was looking at the  code in https://github.com/jjkeijser/ppp/blob/eap-tls/pppd/eap-tls.c and realized I forgot to call ENGINE_ctrl_cmd(...) to setup "LOAD_CERT_CTRL". However, when I do this, the callback function is no longer being called during the mutual authentication handshake. I'm wondering if I have the parameter "cert_info.s_slot_cert_id" incorrectly configured. Here is what my code looks like:
   
struct
{
   const char* s_slot_cert_id;
   X509* cert;
} cert_info;
cert_info.s_slot_cert_id = "a9bee4d72100c52f77c3fc288d2be01a34b5d44f91b3b7ea3d349b8a25752c45";
cert_info.cert = NULL;

ENGINE_ctrl_cmd(engine, "LOAD_CERT_CTRL", 0, &cert_info, NULL, 0);
SSL_CTX_use_certificate(sslContext, cert_info.cert);

I tried manually using LOAD_CERT_CTRL in the openssl shell but I cannot seem to get it to work and cannot find any examples of how to use it.  Is the syntax for LOAD_CERT_CTRL correct? I am using "LOAD_CERT_CTRL:<certificate Object ID>".

OpenSSL> engine -vvvv -t dynamic -pre "SO_PATH:C:\\Users\\whipp\\junk4\\libp11-libp11-0.4.11\\src\\pkcs11.dll" -pre ID:pkcs11 -pre LIST_ADD:1 -pre LOAD -pre "MODULE_PATH:C:\Program Files (x86)\HID Global\ActivClient\\acpkcs211.dll" -pre PIN:123456 -pre FORCE_LOGIN -pre "LOAD_CERT_CTRL:a9bee4d72100c52f77c3fc288d2be01a34b5d44f91b3b7ea3d349b8a25752c45"

(dynamic) Dynamic engine loading support
[Success]: SO_PATH:C:\\Users\\whipp\\junk4\\libp11-libp11-0.4.11\\src\\pkcs11.dll
[Success]: ID:pkcs11
[Success]: LIST_ADD:1
[Success]: LOAD
[Success]: MODULE_PATH:C:\Program Files (x86)\HID Global\ActivClient\\acpkcs211.dll
[Success]: PIN:123456
[Success]: FORCE_LOGIN
[Failure]: LOAD_CERT_CTRL:a9bee4d72100c52f77c3fc288d2be01a34b5d44f91b3b7ea3d349b8a25752c45
4196:error:260AB086:engine routines:ENGINE_ctrl_cmd_string:cmd not executable:.\crypto\engine\eng_ctrl.c:316:
Loaded: (pkcs11) pkcs11 engine
     [ available ]
     SO_PATH: Specifies the path to the 'pkcs11' engine shared library
          (input flags): STRING
     MODULE_PATH: Specifies the path to the PKCS#11 module shared library
          (input flags): STRING
     PIN: Specifies the pin code
          (input flags): STRING
     VERBOSE: Print additional details
          (input flags): NO_INPUT
     QUIET: Remove additional details
          (input flags): NO_INPUT
     LOAD_CERT_CTRL: Get the certificate from card
          (input flags): [Internal]
     INIT_ARGS: Specifies additional initialization arguments to the PKCS#11 module
          (input flags): STRING
     SET_USER_INTERFACE: Set the global user interface (internal)
          (input flags): [Internal]
     SET_CALLBACK_DATA: Set the global user interface extra data (internal)
          (input flags): [Internal]
     FORCE_LOGIN: Force login to the PKCS#11 module
          (input flags): NO_INPUT
OpenSSL>

I'm using the certificate object ID "a9bee4d72100c52f77c3fc288d2be01a34b5d44f91b3b7ea3d349b8a25752c45" for LOAD_CERT_CTRL. Is this right? (I also tried adding "0:" in front of it to indicate slot 0, but that did not work either.


this has little to do with OpenSSL at the moment and more with libp11 - perhaps someone more knowledgable on the libp11 mailing list can help you.

I'd try to use
  -post LOAD_CERT_CTRL
instead of '-pre', as you want this done after the engine has been loaded.

The cert ID does look OK. Note that if you want to use the s_client command that you canNOT specify the certificate form '-certform engine' as the code does not grok that.

HTH,

JJK

Reply | Threaded
Open this post in threaded view
|

Re: private key not available for client_cert_cb

George
Hi,

   I have been trying to setup mutual authentication using a smart card but I can't seem to get the OpenSSL Engine to send a response back to the server containing client's certificate from the smart card.
  
I'm using the following to configure the certificate and private key:

    ENGINE_ctrl_cmd(engine, "LOAD_CERT_CTRL", 0, &cert_info, NULL, 0);
    SSL_CTX_use_certificate(sslContext, cert_info.cert);

    EVP_PKEY* privateKey = ENGINE_load_private_key(engine, "2b2586c684d69b670c0a805edf514e720f2b757d8e2faa0b3a7ff23d1ccfc7ba", transfer_pin, &cb_data);
    SSL_CTX_use_PrivateKey(sslContext, privateKey);
   
(I have been using the code in https://github.com/jjkeijser/ppp/blob/eap-tls/pppd/eap-tls.c  as a guide.)
   
This seems be successful. However, when I start the mutual authentication with
SSL_connect(ssl)
, the mutual authentications handshake fails. I can see the server requesting the certificate from the client and the client sends back an ACK for this message. However, the client does not send the certificate to the server.

I was looking through the OpenSSL code openssl-1.0.2u\ssl\ssl_rsa.c and noticed something interesting. The comment indicates that the flag RSA_METHOD_FLAG_NO_CHECK should be set for smart cards:

static int ssl_set_pkey(CERT *c, EVP_PKEY *pkey)
{
 . . .
#ifndef OPENSSL_NO_RSA
        /*
         * Don't check the public/private key, this is mostly for smart
         * cards.
         */
        if ((pkey->type == EVP_PKEY_RSA) &&
            (RSA_flags(pkey->pkey.rsa) & RSA_METHOD_FLAG_NO_CHECK)) ;
        else
#endif
. . .
}

However, it is not actually set when I use a debugger to inspect the flag. Does it need to be set? If so, how is this done? I could not find anything related to this in 
https://github.com/jjkeijser/ppp/blob/eap-tls/pppd/eap-tls.c




Thanks,
George


On 2021-01-05 11:51 a.m., Jan Just Keijser wrote:
Hi,

On 05/01/21 07:39, George wrote:
Hi,

    I was looking at the  code in https://github.com/jjkeijser/ppp/blob/eap-tls/pppd/eap-tls.c and realized I forgot to call ENGINE_ctrl_cmd(...) to setup "LOAD_CERT_CTRL". However, when I do this, the callback function is no longer being called during the mutual authentication handshake. I'm wondering if I have the parameter "cert_info.s_slot_cert_id" incorrectly configured. Here is what my code looks like:
   
struct
{
   const char* s_slot_cert_id;
   X509* cert;
} cert_info;
cert_info.s_slot_cert_id = "a9bee4d72100c52f77c3fc288d2be01a34b5d44f91b3b7ea3d349b8a25752c45";
cert_info.cert = NULL;

ENGINE_ctrl_cmd(engine, "LOAD_CERT_CTRL", 0, &cert_info, NULL, 0);
SSL_CTX_use_certificate(sslContext, cert_info.cert);

I tried manually using LOAD_CERT_CTRL in the openssl shell but I cannot seem to get it to work and cannot find any examples of how to use it.  Is the syntax for LOAD_CERT_CTRL correct? I am using "LOAD_CERT_CTRL:<certificate Object ID>".

OpenSSL> engine -vvvv -t dynamic -pre "SO_PATH:C:\\Users\\whipp\\junk4\\libp11-libp11-0.4.11\\src\\pkcs11.dll" -pre ID:pkcs11 -pre LIST_ADD:1 -pre LOAD -pre "MODULE_PATH:C:\Program Files (x86)\HID Global\ActivClient\\acpkcs211.dll" -pre PIN:123456 -pre FORCE_LOGIN -pre "LOAD_CERT_CTRL:a9bee4d72100c52f77c3fc288d2be01a34b5d44f91b3b7ea3d349b8a25752c45"

(dynamic) Dynamic engine loading support
[Success]: SO_PATH:C:\\Users\\whipp\\junk4\\libp11-libp11-0.4.11\\src\\pkcs11.dll
[Success]: ID:pkcs11
[Success]: LIST_ADD:1
[Success]: LOAD
[Success]: MODULE_PATH:C:\Program Files (x86)\HID Global\ActivClient\\acpkcs211.dll
[Success]: PIN:123456
[Success]: FORCE_LOGIN
[Failure]: LOAD_CERT_CTRL:a9bee4d72100c52f77c3fc288d2be01a34b5d44f91b3b7ea3d349b8a25752c45
4196:error:260AB086:engine routines:ENGINE_ctrl_cmd_string:cmd not executable:.\crypto\engine\eng_ctrl.c:316:
Loaded: (pkcs11) pkcs11 engine
     [ available ]
     SO_PATH: Specifies the path to the 'pkcs11' engine shared library
          (input flags): STRING
     MODULE_PATH: Specifies the path to the PKCS#11 module shared library
          (input flags): STRING
     PIN: Specifies the pin code
          (input flags): STRING
     VERBOSE: Print additional details
          (input flags): NO_INPUT
     QUIET: Remove additional details
          (input flags): NO_INPUT
     LOAD_CERT_CTRL: Get the certificate from card
          (input flags): [Internal]
     INIT_ARGS: Specifies additional initialization arguments to the PKCS#11 module
          (input flags): STRING
     SET_USER_INTERFACE: Set the global user interface (internal)
          (input flags): [Internal]
     SET_CALLBACK_DATA: Set the global user interface extra data (internal)
          (input flags): [Internal]
     FORCE_LOGIN: Force login to the PKCS#11 module
          (input flags): NO_INPUT
OpenSSL>

I'm using the certificate object ID "a9bee4d72100c52f77c3fc288d2be01a34b5d44f91b3b7ea3d349b8a25752c45" for LOAD_CERT_CTRL. Is this right? (I also tried adding "0:" in front of it to indicate slot 0, but that did not work either.


this has little to do with OpenSSL at the moment and more with libp11 - perhaps someone more knowledgable on the libp11 mailing list can help you.

I'd try to use
  -post LOAD_CERT_CTRL
instead of '-pre', as you want this done after the engine has been loaded.

The cert ID does look OK. Note that if you want to use the s_client command that you canNOT specify the certificate form '-certform engine' as the code does not grok that.

HTH,

JJK


Reply | Threaded
Open this post in threaded view
|

RE: private key not available for client_cert_cb

Michael Wojcik
> From: openssl-users <[hidden email]> On Behalf Of George
> Sent: Friday, 8 January, 2021 14:35

> The comment indicates that the flag RSA_METHOD_FLAG_NO_CHECK should be set
> for smart cards[...]

> However, it is not actually set when I use a debugger to inspect the flag.
> Does it need to be set? If so, how is this done?

If memory serves, the PKCS#11 implementation invoked by the pkcs11 engine is supposed to set it.

See for example this patch to OpenSC's pkcs11-helper library:

https://github.com/OpenSC/pkcs11-helper/commit/5198bb1e557dfd4109bea41c086825bf6ebdd9f3

(That patch actually is to set a different flag, but it shows the code in question.)

I know, that's probably not terribly helpful.

If you do a web search for something like

        pkcs11 "RSA_METHOD_FLAG_NO_CHECK"

you'll probably find a number of hits where other people ran into similar problems.

Isn't PKCS#11 grand? If you're bored with all the interoperability problems of X.509, PKIX, and TLS, we have good news!

--
Michael Wojcik
Reply | Threaded
Open this post in threaded view
|

Re: private key not available for client_cert_cb

George
Hi,

    I had a look at the pkcs11-helper and can see where the RSA_METHOD_FLAG_NO_CHECK is being set. It's using a session object called pkcs11h_openssl_session_t, which I do not see in the libp11 or openSC code.

Right now I am using the "libp11" DLL (i.e. libp11-libp11-0.4.11\src\pkcs11.dll) with my PKCS11 smart card middleware DLL. Should I be using the OpenSC pkcs11 DLL instead of my middleware DLL if I am using libp1?

Do you know if it is normal to see exceptions related to the PKCS11 function calls in the libp11 code? For example, I can see  the following function generate an exception on C_GetSlotList(...) multiple times but it eventually is successful.  Is this normal behaviour?

int pkcs11_enumerate_slots(PKCS11_CTX *ctx, PKCS11_SLOT **slotp, unsigned int *countp)
{
. . .
    rv = cpriv->method->C_GetSlotList(FALSE, NULL_PTR, &nslots);
. . .
}


Thanks,
George



On 2021-01-08 6:32 p.m., Michael Wojcik wrote:
From: openssl-users [hidden email] On Behalf Of George
Sent: Friday, 8 January, 2021 14:35

      
The comment indicates that the flag RSA_METHOD_FLAG_NO_CHECK should be set
for smart cards[...]

      
However, it is not actually set when I use a debugger to inspect the flag.
Does it need to be set? If so, how is this done?
If memory serves, the PKCS#11 implementation invoked by the pkcs11 engine is supposed to set it.

See for example this patch to OpenSC's pkcs11-helper library:

https://github.com/OpenSC/pkcs11-helper/commit/5198bb1e557dfd4109bea41c086825bf6ebdd9f3

(That patch actually is to set a different flag, but it shows the code in question.)

I know, that's probably not terribly helpful.

If you do a web search for something like

        pkcs11 "RSA_METHOD_FLAG_NO_CHECK"

you'll probably find a number of hits where other people ran into similar problems.

Isn't PKCS#11 grand? If you're bored with all the interoperability problems of X.509, PKIX, and TLS, we have good news!

--
Michael Wojcik

Reply | Threaded
Open this post in threaded view
|

RE: private key not available for client_cert_cb

Michael Wojcik
> From: openssl-users <[hidden email]> On Behalf Of George
> Sent: Sunday, 10 January, 2021 21:01

> Right now I am using the "libp11" DLL (i.e. libp11-libp11-0.4.11\src\pkcs11.dll)
> with my PKCS11 smart card middleware DLL. Should I be using the OpenSC pkcs11 DLL
> instead of my middleware DLL if I am using libp1?

Honestly, I have no idea. It's been years since I worked with PKCS#11, and then I was using a single piece of test hardware. I got it working with OpenSSL using the OpenSC modules, but that may have been specific to my case.

> Do you know if it is normal to see exceptions related to the PKCS11 function calls
> in the libp11 code? For example, I can see  the following function generate an
> exception on C_GetSlotList(...) multiple times but it eventually is successful.
> Is this normal behaviour?

What sort of "exception"? A Windows exception? UNIX signal? C++ exception?

My initial guess would be that this is a timing issue - maybe the device needs some time to become available, for example. But that's just a guess. Maybe someone with more experience with a variety of HSMs and PKCS#11 will weigh in.

--
Michael Wojcik
Reply | Threaded
Open this post in threaded view
|

Re: private key not available for client_cert_cb

Jan Just Keijser-2
In reply to this post by George
Hi,

On 08/01/21 22:35, George wrote:
Hi,

   I have been trying to setup mutual authentication using a smart card but I can't seem to get the OpenSSL Engine to send a response back to the server containing client's certificate from the smart card.
  
I'm using the following to configure the certificate and private key:

    ENGINE_ctrl_cmd(engine, "LOAD_CERT_CTRL", 0, &cert_info, NULL, 0);
    SSL_CTX_use_certificate(sslContext, cert_info.cert);

    EVP_PKEY* privateKey = ENGINE_load_private_key(engine, "2b2586c684d69b670c0a805edf514e720f2b757d8e2faa0b3a7ff23d1ccfc7ba", transfer_pin, &cb_data);
    SSL_CTX_use_PrivateKey(sslContext, privateKey);
   
(I have been using the code in https://github.com/jjkeijser/ppp/blob/eap-tls/pppd/eap-tls.c  as a guide.)
   
This seems be successful. However, when I start the mutual authentication with
SSL_connect(ssl)
, the mutual authentications handshake fails. I can see the server requesting the certificate from the client and the client sends back an ACK for this message. However, the client does not send the certificate to the server.

I was looking through the OpenSSL code openssl-1.0.2u\ssl\ssl_rsa.c and noticed something interesting. The comment indicates that the flag RSA_METHOD_FLAG_NO_CHECK should be set for smart cards:

static int ssl_set_pkey(CERT *c, EVP_PKEY *pkey)
{
 . . .
#ifndef OPENSSL_NO_RSA
        /*
         * Don't check the public/private key, this is mostly for smart
         * cards.
         */
        if ((pkey->type == EVP_PKEY_RSA) &&
            (RSA_flags(pkey->pkey.rsa) & RSA_METHOD_FLAG_NO_CHECK)) ;
        else
#endif
. . .
}

However, it is not actually set when I use a debugger to inspect the flag. Does it need to be set? If so, how is this done? I could not find anything related to this in 
https://github.com/jjkeijser/ppp/blob/eap-tls/pppd/eap-tls.c


if you read through the code blob that Michael pointed you to, you will find that this flag needs to be set *under certain circumstances* when using smartcards. It has to do mostly with the situation where
- private key is on the smart card
- the public key/certificate is NOT on the smart card
- you ask OpenSSL to verify the private key without explicitly providing a public key.

I've never run into this issue, but then again, I have not tested very often the case where the certificate was not present on the HSM/smart card but the private key is.  YMMV.

As for using pksc11helper versus using libp11: that is just a matter of taste. I used the engine_pkcs11 + libp11 route for the eap-tls code , mostly because it was the first "working" set of tools I found at the time.

You can also take the "pkcs11helper" route, which is what OpenVPN does (see https://github.com/openvpn).  Both methods have pro's and con's. 

Do you run into problems if you DO not set the RSA_METHOD_FLAG_NO_CHECK flag?  All that flag does is to stop OpenSSL from verifying that a public key/cert and private key match/belong together for RSA keys only; if your smartcard supports EC keys then this flag will do you no good.

HTH,

JJK

Reply | Threaded
Open this post in threaded view
|

Re: private key not available for client_cert_cb

George
I found out what my problem is! I'm running it in FIPS mode and this causes the PKCS11 engine to fail during mutual authentication. I eventually traced the problem to the following issue:
https://bugzilla.redhat.com/show_bug.cgi?id=1827535

It looks like there is a bug in libp11. Once I made the suggested workaround, it worked.
My original code, which is based on
https://github.com/jjkeijser/ppp/blob/eap-tls/pppd/eap-tls.c
worked perfectly after I added in the libp11 fix.  :)


Thanks!
George


On 2021-01-11 11:01 a.m., Jan Just Keijser wrote:
Hi,

On 08/01/21 22:35, George wrote:
Hi,

   I have been trying to setup mutual authentication using a smart card but I can't seem to get the OpenSSL Engine to send a response back to the server containing client's certificate from the smart card.
  
I'm using the following to configure the certificate and private key:

    ENGINE_ctrl_cmd(engine, "LOAD_CERT_CTRL", 0, &cert_info, NULL, 0);
    SSL_CTX_use_certificate(sslContext, cert_info.cert);

    EVP_PKEY* privateKey = ENGINE_load_private_key(engine, "2b2586c684d69b670c0a805edf514e720f2b757d8e2faa0b3a7ff23d1ccfc7ba", transfer_pin, &cb_data);
    SSL_CTX_use_PrivateKey(sslContext, privateKey);
   
(I have been using the code in https://github.com/jjkeijser/ppp/blob/eap-tls/pppd/eap-tls.c  as a guide.)
   
This seems be successful. However, when I start the mutual authentication with
SSL_connect(ssl)
, the mutual authentications handshake fails. I can see the server requesting the certificate from the client and the client sends back an ACK for this message. However, the client does not send the certificate to the server.

I was looking through the OpenSSL code openssl-1.0.2u\ssl\ssl_rsa.c and noticed something interesting. The comment indicates that the flag RSA_METHOD_FLAG_NO_CHECK should be set for smart cards:

static int ssl_set_pkey(CERT *c, EVP_PKEY *pkey)
{
 . . .
#ifndef OPENSSL_NO_RSA
        /*
         * Don't check the public/private key, this is mostly for smart
         * cards.
         */
        if ((pkey->type == EVP_PKEY_RSA) &&
            (RSA_flags(pkey->pkey.rsa) & RSA_METHOD_FLAG_NO_CHECK)) ;
        else
#endif
. . .
}

However, it is not actually set when I use a debugger to inspect the flag. Does it need to be set? If so, how is this done? I could not find anything related to this in 
https://github.com/jjkeijser/ppp/blob/eap-tls/pppd/eap-tls.c


if you read through the code blob that Michael pointed you to, you will find that this flag needs to be set *under certain circumstances* when using smartcards. It has to do mostly with the situation where
- private key is on the smart card
- the public key/certificate is NOT on the smart card
- you ask OpenSSL to verify the private key without explicitly providing a public key.

I've never run into this issue, but then again, I have not tested very often the case where the certificate was not present on the HSM/smart card but the private key is.  YMMV.

As for using pksc11helper versus using libp11: that is just a matter of taste. I used the engine_pkcs11 + libp11 route for the eap-tls code , mostly because it was the first "working" set of tools I found at the time.

You can also take the "pkcs11helper" route, which is what OpenVPN does (see https://github.com/openvpn).  Both methods have pro's and con's. 

Do you run into problems if you DO not set the RSA_METHOD_FLAG_NO_CHECK flag?  All that flag does is to stop OpenSSL from verifying that a public key/cert and private key match/belong together for RSA keys only; if your smartcard supports EC keys then this flag will do you no good.

HTH,

JJK


Reply | Threaded
Open this post in threaded view
|

Re: private key not available for client_cert_cb

George
In reply to this post by Michael Wojcik
I'm running this in Windows 10 and when I load the smart card middleware PKCS11 DLL, I see the exception:
Exception thrown at 0x773046D2 in GENCom.exe: Microsoft C++ exception: unsigned long at memory location 0x07FCFA00.

During mutual authentication, I also see alot of other exceptions such as:
Exception thrown at 0x773046D2 in GENCom.exe: Microsoft C++ exception: AI::Middleware::CMWException at memory location 0x032FD2D0.

I traced them down to various PKCS11 calls on the card in libp11.
e.g.
the function call to C_GetSlotList(...) in the file p11_slot.c triggers an exception:
int pkcs11_enumerate_slots(PKCS11_CTX *ctx, PKCS11_SLOT **slotp, unsigned int *countp)
{
. . .
rv = cpriv->method->C_GetSlotList(FALSE, NULL_PTR, &nslots);
. . .
}

It is interesting to note that this function seems to get called multiple times and it eventually works. I do not fully understand how/why the code does that and if this is the design intent. The exceptions don't seem to have any effect on the functionality, but I still need to understand why they are occurring. 

It looks like someone else using a smart card has also encountered similar problems in Windows but there is no real answer as to why they are occurring:
https://www.codeproject.com/Questions/1254182/Smart-card-apis-throw-first-chance-exceptions-but



Thanks,
George


On 2021-01-11 9:41 a.m., Michael Wojcik wrote:
From: openssl-users [hidden email] On Behalf Of George
Sent: Sunday, 10 January, 2021 21:01

      
Right now I am using the "libp11" DLL (i.e. libp11-libp11-0.4.11\src\pkcs11.dll)
with my PKCS11 smart card middleware DLL. Should I be using the OpenSC pkcs11 DLL
instead of my middleware DLL if I am using libp1?
Honestly, I have no idea. It's been years since I worked with PKCS#11, and then I was using a single piece of test hardware. I got it working with OpenSSL using the OpenSC modules, but that may have been specific to my case.

Do you know if it is normal to see exceptions related to the PKCS11 function calls
in the libp11 code? For example, I can see  the following function generate an
exception on C_GetSlotList(...) multiple times but it eventually is successful.
Is this normal behaviour?
What sort of "exception"? A Windows exception? UNIX signal? C++ exception?

My initial guess would be that this is a timing issue - maybe the device needs some time to become available, for example. But that's just a guess. Maybe someone with more experience with a variety of HSMs and PKCS#11 will weigh in.

--
Michael Wojcik

Reply | Threaded
Open this post in threaded view
|

RE: private key not available for client_cert_cb

Michael Wojcik
> From: openssl-users <[hidden email]> On Behalf Of George
> Sent: Tuesday, 12 January, 2021 00:18

> I'm running this in Windows 10 and when I load the smart card middleware
> PKCS11 DLL, I see the exception:
> Exception thrown at 0x773046D2 in GENCom.exe: Microsoft C++ exception:
> unsigned long at memory location 0x07FCFA00.

OK. If I were debugging libp11, it would be useful to know what the exception actually was, but as it is all I can say is that it seems to be a libp11 problem. As you noted further below:

> It looks like someone else using a smart card has also encountered similar
> problems in Windows but there is no real answer as to why they are occurring:
> https://www.codeproject.com/Questions/1254182/Smart-card-apis-throw-first-chance-
> exceptions-but

You'll probably have to just swallow the exceptions and retry until it works or your code decides to give up and return an error. Maybe one of the libp11 maintainers or someone else using the library will dig into it at some point.

--
Michael Wojcik
12