Engine with custom evp method callbacks

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Engine with custom evp method callbacks

Tobias.Wolf

Hi everbody,

 

I`m looking for a working example on how to implements a custom engine based on EVP methods callbacks. First I was implementing my custom engine based on RSA callbacks, but we found out that we cannot use this mechanism,

therefore I need to change to EVP, details are written here https://github.com/openssl/openssl/issues/7968.

 

                RSA_METHOD* rsa_method = RSA_meth_new("OpenSSL Custom RSA method", 0);

                const RSA_METHOD* ossl_rsa_meth = RSA_PKCS1_OpenSSL();

 

                rc = RSA_meth_set_priv_enc(rsa_method, gk_openssl_rsa_priv_enc);

               

                rc = ENGINE_set_RSA(e, rsa_method);

                if (rc != TRUE) {

                               return 0;

                }

 

                if (flags & ENGINE_METHOD_RSA) {

                               rc = ENGINE_register_RSA(e);

                                               if (rc != TRUE) {

                                                                               return 0;

                                               }

                }

 

 

Now I try with EVP the following source code but it’s not working:

 

EVP_PKEY_METHOD* engine_pkey_methods = EVP_PKEY_meth_new(EVP_PKEY_RSA_PSS, 0);

const EVP_PKEY_METHOD* ossl_pkey_methods = EVP_PKEY_meth_find(EVP_PKEY_RSA_PSS);

EVP_PKEY_meth_copy(engine_pkey_methods, ossl_pkey_methods);

 

// This shall be an equivalent to = RSA_PKCS1_OpenSSL();

const EVP_PKEY_METHOD* ossl_pkey_methods = EVP_PKEY_meth_find(EVP_PKEY_RSA_PSS);

 

But how to set the evp method the engine like RSA(e, rsa_method);?

This expects another callback, but I just want to set the method?!

 

int ENGINE_set_pkey_meths(ENGINE *e, ENGINE_PKEY_METHS_PTR f);

 

regards

Tobi

 

 

 

Reply | Threaded
Open this post in threaded view
|

Re: Engine with custom evp method callbacks

Dmitry Belyavsky-3
Dear Tobi, 

On Fri, Nov 22, 2019 at 12:27 PM <[hidden email]> wrote:

Hi everbody,

 

I`m looking for a working example on how to implements a custom engine based on EVP methods callbacks. First I was implementing my custom engine based on RSA callbacks, but we found out that we cannot use this mechanism,

therefore I need to change to EVP, details are written here https://github.com/openssl/openssl/issues/7968.

 

                RSA_METHOD* rsa_method = RSA_meth_new("OpenSSL Custom RSA method", 0);

                const RSA_METHOD* ossl_rsa_meth = RSA_PKCS1_OpenSSL();

 

                rc = RSA_meth_set_priv_enc(rsa_method, gk_openssl_rsa_priv_enc);

               

                rc = ENGINE_set_RSA(e, rsa_method);

                if (rc != TRUE) {

                               return 0;

                }

 

                if (flags & ENGINE_METHOD_RSA) {

                               rc = ENGINE_register_RSA(e);

                                               if (rc != TRUE) {

                                                                               return 0;

                                               }

                }

 

 

Now I try with EVP the following source code but it’s not working:

 

EVP_PKEY_METHOD* engine_pkey_methods = EVP_PKEY_meth_new(EVP_PKEY_RSA_PSS, 0);

const EVP_PKEY_METHOD* ossl_pkey_methods = EVP_PKEY_meth_find(EVP_PKEY_RSA_PSS);

EVP_PKEY_meth_copy(engine_pkey_methods, ossl_pkey_methods);

 

// This shall be an equivalent to = RSA_PKCS1_OpenSSL();

const EVP_PKEY_METHOD* ossl_pkey_methods = EVP_PKEY_meth_find(EVP_PKEY_RSA_PSS);

 

But how to set the evp method the engine like RSA(e, rsa_method);?

This expects another callback, but I just want to set the method?!

 

int ENGINE_set_pkey_meths(ENGINE *e, ENGINE_PKEY_METHS_PTR f);

 


I strongly suppose that you can't mix EVP_PKEY_METHOD and RSA_METHOD, but you should wrap the RSA_METHOD callbacks in the EVP_PKEY callbacks.
I suggest you look at the https://github.com/gost-engine/engine as an example of providing the EVP operations via the engine.

I also have an example of providing custom RSA_METHOD somewhere but it was designed to work with 1.0 and may be incompatible with the 1.1.* because of using the internal structures.

--
SY, Dmitry Belyavsky