EVP_DecryptFinal_ex fails on larger files

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

EVP_DecryptFinal_ex fails on larger files

clarksom
I am trying to make a C++ class to make encrytion and decrytion simple
using the blowfish algorithm with openssl.  Anyways, I have modelled my
program around
http://www.linuxgazette.com/issue87/misc/vinayak/sym_funcs.c.txt which is
a C program for reading and writing files, with encrytion using openssl.
Anyways, both programs seem to fail the same way on larger files (they
will pass on smaller files) and I can't seem to get EVP_DecryptFinal_ex to
work.
Here are my encrypt and decrypt functions with my defines as follows

IP_SIZE = 1024
OP_SIZE = 1032

int Encryption::encrypt(std::ifstream &istream, std::ofstream &ostream)
{
  unsigned char outbuf[OP_SIZE];
  int olen, tlen, n;
  char inbuff[IP_SIZE];
  EVP_CIPHER_CTX ctx;
  EVP_CIPHER_CTX_init (&ctx);
  EVP_EncryptInit_ex (&ctx, EVP_bf_cbc(), NULL, key, iv);

  olen = 0;
  tlen = 0;

  for (int i = 0;;i++)
  {
    bzero(&inbuff, IP_SIZE);

    if((n = istream.readsome(inbuff, IP_SIZE)) == -1)
    {
      std::cout << "read error\n";
      break;
    }

    else if(n == 0)
      break;

    if(EVP_EncryptUpdate(&ctx, outbuf, &olen, (unsigned char*)inbuff, n)
!= 1)
    {
      std::cout << "error in encrypt update\n";
      return 0;
    }
    std::cout << "before enc. fin. ||" << i << "||\n";
    std::cout << "olen: " << olen << "\n";
    //std::cout << "EVP_MAX_BLOCK_LENGTH: " << EVP_MAX_BLOCK_LENGTH << "\n";
    if(EVP_EncryptFinal_ex(&ctx, outbuf + olen, &tlen) != 1)
    {
      std::cout << "error in encrypt final\n";
      return 0;
    }
    else
    {
      std::cout << "Final success on: " << i << "\n";
      std::cout << "tlen: " << tlen << "\n";
    }

    olen += tlen;

    ostream.write((char*)outbuf, olen);
  }
  //EVP_CIPHER_CTX_cleanup (&ctx); not needed because of _ex's
  return 1;
}

int Encryption::decrypt(std::ifstream &istream, std::ofstream &ostream)
{
  unsigned char outbuf[IP_SIZE];
  int olen, tlen, n;
  char inbuff[OP_SIZE];
  EVP_CIPHER_CTX ctx;
  EVP_CIPHER_CTX_init (&ctx);
  EVP_DecryptInit_ex (&ctx, EVP_bf_cbc (), NULL, key, iv);

  tlen = 0;

  std::cout << "OP_SIZE: " << OP_SIZE << "\n";

  for (int i = 0;; i++)
  {
    bzero(&inbuff, OP_SIZE);

    if((n = istream.readsome(inbuff, OP_SIZE)) == -1)
    {
      std::cout << "read error\n";
      break;
    }

    else if(n == 0)
      break;

    if (EVP_DecryptUpdate(&ctx, outbuf, &olen, (unsigned char*)inbuff, n)
!= 1)
    {
      std::cout << "error in decrypt update\n";
      return 0;
    }
    std::cout << "before dec. fin. ||" << i << "||\n";
    std::cout << "inbuff size: " << n << "\n";
    std::cout << "olen size: " << olen << "\n";
    if (EVP_DecryptFinal_ex(&ctx, outbuf + olen, &tlen) != 1) //outbuf + olen
    {
      std::cout << "tlen: " << tlen << "\n";
      std::cout << "error in decrypt final\n";
      return 0;
    }
    else
    {
      //std::cout << "tlen: " << tlen << "\n";
    }
    olen += tlen;


    ostream.write((char*)outbuf, olen);
  }
  //EVP_CIPHER_CTX_cleanup (&ctx); //not needed because of _ex's
  return 1;
}

The output of my program during decryption is as follows

Key
166     234     103     213     202     55      168     35      155    
193     103 2        98      36      200     0
 ------
Initialization vector
88      240     255     191     16      116     255     183
 ------
OP_SIZE: 1032
before dec. fin. ||0||
inbuff size: 1032
olen size: 1024
DecryptFinal Success: 0
tlen: 0
before dec. fin. ||1||
inbuff size: 1032
olen size: 1032
DecryptFinal Success: 1
tlen: 0
before dec. fin. ||2||
inbuff size: 1032
olen size: 1032
DecryptFinal Success: 2
tlen: 0
before dec. fin. ||3||
inbuff size: 1032
olen size: 1032
DecryptFinal Success: 3
tlen: 0
before dec. fin. ||4||
inbuff size: 1032
olen size: 1032
DecryptFinal Success: 4
tlen: 0
before dec. fin. ||5||
inbuff size: 1032
olen size: 1032
DecryptFinal Success: 5
tlen: 0
before dec. fin. ||6||
inbuff size: 1032
olen size: 1032
DecryptFinal Success: 6
tlen: 0
before dec. fin. ||7||
inbuff size: 968
olen size: 968
tlen: 0
error in decrypt final
Segmentation fault

I have spent too long (about 20 hours) trying to debug this, and I just
can't get it, I'd really appreciate it if anyone could help me or even
give me a little advice on where I am going wrong.
if anyone would like more information (like my whole class), let me know,
I didn't want to post it since it's about 3x this size of relatively
useless stuff for this problem.

Thanks in advance.

--
Matt

______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
User Support Mailing List                    [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: EVP_DecryptFinal_ex fails on larger files

Dr. Stephen Henson
On Sun, Nov 13, 2005, [hidden email] wrote:

> I am trying to make a C++ class to make encrytion and decrytion simple
> using the blowfish algorithm with openssl.  Anyways, I have modelled my
> program around
> http://www.linuxgazette.com/issue87/misc/vinayak/sym_funcs.c.txt which is
> a C program for reading and writing files, with encrytion using openssl.
> Anyways, both programs seem to fail the same way on larger files (they
> will pass on smaller files) and I can't seem to get EVP_DecryptFinal_ex to
> work.
> Here are my encrypt and decrypt functions with my defines as follows
>

I suggest you rely on the documentation instead. The normal way to do things
is to call EVP_EncryptInit_ex() initially, then call EVP_EncryptUpdate() one
or more times on each chunk to be encrypted and finally call
EVP_EncryptFinal_ex() when all data has been handled.

That example seems to be calling EVP_EncryptFinal_ex() on each chunk.

Steve.
--
Dr Stephen N. Henson. Email, S/MIME and PGP keys: see homepage
OpenSSL project core developer and freelance consultant.
Funding needed! Details on homepage.
Homepage: http://www.drh-consultancy.demon.co.uk
______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
User Support Mailing List                    [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: EVP_DecryptFinal_ex fails on larger files

clarksom
Thanks a lot, I did read the doc's before, but was still confused.  I did
it your proposed way now, and everything is working perfectly.

Thanks again
--
Matthew Clarkson

On Mon, 14 Nov 2005, Dr. Stephen Henson wrote:

> On Sun, Nov 13, 2005, [hidden email] wrote:
>
>> I am trying to make a C++ class to make encrytion and decrytion simple
>> using the blowfish algorithm with openssl.  Anyways, I have modelled my
>> program around
>> http://www.linuxgazette.com/issue87/misc/vinayak/sym_funcs.c.txt which is
>> a C program for reading and writing files, with encrytion using openssl.
>> Anyways, both programs seem to fail the same way on larger files (they
>> will pass on smaller files) and I can't seem to get EVP_DecryptFinal_ex to
>> work.
>> Here are my encrypt and decrypt functions with my defines as follows
>>
>
> I suggest you rely on the documentation instead. The normal way to do things
> is to call EVP_EncryptInit_ex() initially, then call EVP_EncryptUpdate() one
> or more times on each chunk to be encrypted and finally call
> EVP_EncryptFinal_ex() when all data has been handled.
>
> That example seems to be calling EVP_EncryptFinal_ex() on each chunk.
>
> Steve.
> --
> Dr Stephen N. Henson. Email, S/MIME and PGP keys: see homepage
> OpenSSL project core developer and freelance consultant.
> Funding needed! Details on homepage.
> Homepage: http://www.drh-consultancy.demon.co.uk
> ______________________________________________________________________
> OpenSSL Project                                 http://www.openssl.org
> User Support Mailing List                    [hidden email]
> Automated List Manager                           [hidden email]
>
______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
User Support Mailing List                    [hidden email]
Automated List Manager                           [hidden email]