Evp_Encrypt_Init Segfault

[Felix Dorner]

Hi,

the following code executes once, and does fine. Calling the function a
second time gives a segfault during the call marked by "-->"

unsigned char *encrypt_message(unsigned char *message, int inl, int *outl)
{
        EVP_CIPHER_CTX ctx;
        EVP_CIPHER_CTX_init(&ctx);
        -->EVP_EncryptInit(&ctx, EVP_bf_ecb(), NULL, NULL);
        EVP_CIPHER_CTX_set_key_length(&ctx, SHA_DIGEST_LENGTH);
        EVP_EncryptInit(&ctx, NULL, k, NULL);
        char *ret;
        int tmp, ol;
        ol = 0;
        ret = (char *)malloc(inl + EVP_CIPHER_CTX_block_size(&ctx));
        EVP_EncryptUpdate(&ctx, &ret[ol], &tmp, message, inl);
        ol = tmp;
        EVP_EncryptFinal(&ctx, &ret[ol], &tmp);
        *outl = ol+tmp;
        return ret;
}


Anything obvious that might lead to the segfault?

Thanks,
Felix
______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
User Support Mailing List                    [hidden email]
Automated List Manager                           [hidden email]

[Girish Venkatachalam]

Try calling EVP_CIPHER_CTX_cleanup(&ctx) at the end...

--- Felix Dorner <[hidden email]> wrote:

> Hi,
>
> the following code executes once, and does fine.
> Calling the function a
> second time gives a segfault during the call marked
> by "-->"
>
> unsigned char *encrypt_message(unsigned char
> *message, int inl, int *outl)
> {
>         EVP_CIPHER_CTX ctx;
>         EVP_CIPHER_CTX_init(&ctx);
>         -->EVP_EncryptInit(&ctx, EVP_bf_ecb(), NULL,
> NULL);
>         EVP_CIPHER_CTX_set_key_length(&ctx,
> SHA_DIGEST_LENGTH);
>         EVP_EncryptInit(&ctx, NULL, k, NULL);
>         char *ret;
>         int tmp, ol;
>         ol = 0;
>         ret = (char *)malloc(inl +
> EVP_CIPHER_CTX_block_size(&ctx));
>         EVP_EncryptUpdate(&ctx, &ret[ol], &tmp,
> message, inl);
>         ol = tmp;
>         EVP_EncryptFinal(&ctx, &ret[ol], &tmp);
>         *outl = ol+tmp;
>         return ret;
> }
>
>
> Anything obvious that might lead to the segfault?
>
> Thanks,
> Felix
>

______________________________________________________________________
> OpenSSL Project                                
> http://www.openssl.org
> User Support Mailing List                  
> [hidden email]
> Automated List Manager                          
> [hidden email]
>


__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around
http://mail.yahoo.com 
______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
User Support Mailing List                    [hidden email]
Automated List Manager                           [hidden email]

[Girish Venkatachalam]

Try calling EVP_CIPHER_CTX_cleanup(&ctx) at the end...

--- Felix Dorner <[hidden email]> wrote:

> Hi,
>
> the following code executes once, and does fine.
> Calling the function a
> second time gives a segfault during the call marked
> by "-->"
>
> unsigned char *encrypt_message(unsigned char
> *message, int inl, int *outl)
> {
>         EVP_CIPHER_CTX ctx;
>         EVP_CIPHER_CTX_init(&ctx);
>         -->EVP_EncryptInit(&ctx, EVP_bf_ecb(), NULL,
> NULL);
>         EVP_CIPHER_CTX_set_key_length(&ctx,
> SHA_DIGEST_LENGTH);
>         EVP_EncryptInit(&ctx, NULL, k, NULL);
>         char *ret;
>         int tmp, ol;
>         ol = 0;
>         ret = (char *)malloc(inl +
> EVP_CIPHER_CTX_block_size(&ctx));
>         EVP_EncryptUpdate(&ctx, &ret[ol], &tmp,
> message, inl);
>         ol = tmp;
>         EVP_EncryptFinal(&ctx, &ret[ol], &tmp);
>         *outl = ol+tmp;
>         return ret;
> }
>
>
> Anything obvious that might lead to the segfault?
>
> Thanks,
> Felix
>

______________________________________________________________________
> OpenSSL Project                                
> http://www.openssl.org
> User Support Mailing List                  
> [hidden email]
> Automated List Manager                          
> [hidden email]
>


__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around
http://mail.yahoo.com 
______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
User Support Mailing List                    [hidden email]
Automated List Manager                           [hidden email]

[Felix Dorner]

Girish Venkatachalam wrote:

>Try calling EVP_CIPHER_CTX_cleanup(&ctx) at the end...
>
>  
>

I have tried this, does not change the situation. gdb output is

200             EVP_EncryptInit(&ctx, EVP_bf_ecb(), NULL, NULL);
(gdb) step

Program received signal SIGSEGV, Segmentation fault.
0xb7df82fb in mallopt () from /lib/libc.so.6


I really dont know whats wrong here,

Felix
______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
User Support Mailing List                    [hidden email]
Automated List Manager                           [hidden email]

[Alain Damiral]

Have you tried with the EVP_EncryptInit_ex() family of functions ? I'm
not sure it would help much but it could be worth a try


Felix Dorner wrote:

>Girish Venkatachalam wrote:
>
>  
>
>>Try calling EVP_CIPHER_CTX_cleanup(&ctx) at the end...
>>
>>
>>
>>    
>>
>
>I have tried this, does not change the situation. gdb output is
>
>200             EVP_EncryptInit(&ctx, EVP_bf_ecb(), NULL, NULL);
>(gdb) step
>
>Program received signal SIGSEGV, Segmentation fault.
>0xb7df82fb in mallopt () from /lib/libc.so.6
>
>
>I really dont know whats wrong here,
>
>Felix
>______________________________________________________________________
>OpenSSL Project                                 http://www.openssl.org
>User Support Mailing List                    [hidden email]
>Automated List Manager                           [hidden email]
>  
>

--
Alain Damiral

______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
User Support Mailing List                    [hidden email]
Automated List Manager                           [hidden email]

[Mark-62]

Hi,

> the following code executes once, and does fine. Calling the
> function a
> second time gives a segfault during the call marked by "-->"

You may have inadvertantly corrupted the heap the first time
your code is executed.  I suggest you put several assert statements
to ensure that memory outside of that you allocated is not overwritten.

Regards,
Mark
______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
User Support Mailing List                    [hidden email]
Automated List Manager                           [hidden email]

[clarksom]

I'm not much of an expert with any of this, but you may want to look at
some of the return values of some of the functions to make sure
everything is good, such as on EVP_EncryptFinal.  Please take a look at
some code I did up last summer in C++ (but it is almost all C), located at
http://lunir.com/Encryption.cpp.  The Function in particular would be int
Encryption::encrypt(std::ifstream &istream, std::ofstream &ostream).  It
deals with streams but converts it all into cstrings, so it should
roughtly be the same.  Hope this helps.


--
Matthew Clarkson

On Mon, 30 Jan 2006, Felix Dorner wrote:

> Hi,
>
> the following code executes once, and does fine. Calling the function a
> second time gives a segfault during the call marked by "-->"
>
> unsigned char *encrypt_message(unsigned char *message, int inl, int *outl)
> {
>        EVP_CIPHER_CTX ctx;
>        EVP_CIPHER_CTX_init(&ctx);
>        -->EVP_EncryptInit(&ctx, EVP_bf_ecb(), NULL, NULL);
>        EVP_CIPHER_CTX_set_key_length(&ctx, SHA_DIGEST_LENGTH);
>        EVP_EncryptInit(&ctx, NULL, k, NULL);
>        char *ret;
>        int tmp, ol;
>        ol = 0;
>        ret = (char *)malloc(inl + EVP_CIPHER_CTX_block_size(&ctx));
>        EVP_EncryptUpdate(&ctx, &ret[ol], &tmp, message, inl);
>        ol = tmp;
>        EVP_EncryptFinal(&ctx, &ret[ol], &tmp);
>        *outl = ol+tmp;
>        return ret;
> }
>
>
> Anything obvious that might lead to the segfault?
>
> Thanks,
> Felix
> ______________________________________________________________________
> OpenSSL Project                                 http://www.openssl.org
> User Support Mailing List                    [hidden email]
> Automated List Manager                           [hidden email]
>

______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
User Support Mailing List                    [hidden email]
Automated List Manager                           [hidden email]

[Felix Dorner]

I somehow corrupted the ctx object by overshooting the malloced area as
Mark had pointed out.
The problem is solved now,

thanks, felix
______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
User Support Mailing List                    [hidden email]
Automated List Manager                           [hidden email]