How can I pass data to a running instance of OpenSSL CLI on Windows within a batch file?

classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

How can I pass data to a running instance of OpenSSL CLI on Windows within a batch file?

Funnell, Leon

We have Windows application which passes data to OpenSSL.exe to encrypt as a Windows command, then scrapes the encrypted data back from the output.  The Windows app can call external Windows commands but we cannot call APIs or extend the functionality programmatically.   Functionally it works, but it doesn’t scale as each time you call OpenSSL.exe it takes about a second and spikes the CPU.  The application we are using is required to process 6000 records every hour. 

 

I have two tests set up:

1.       A batch file which runs 6000 times, repeatedly running the following command:

Openssl.exe aes-256-cbc -a -e -k eiccmkjd94jfgniw03ljkdlfutcnv320 –in test.txt

 

2.       A text file with the following line repeated 6000 times, which I paste into the OpenSSL CLI:

aes-256-cbc -a -e -k eiccmkjd94jfgniw03ljkdlfutcnv320 –in test.txt

 

When I use the batch file which invokes OpenSSL.exe 6000 times, it takes several hours to complete and spikes the CPU significantly.  It seems to be the initialisation of the OpenSSL.exe program rather than the encryption however, as if I paste in the text file to the OpenSSL.exe CLI it completes in several seconds and takes very little CPU.

 

What I need is a way of running OpenSSL.exe as a process which I can pass parameters to on STDIN, and output parameters to STDOUT.  I would like to be able to call another batch file or program with the unencrypted data as the input parameter which would then pass this to the running “service”, retrieve the  encrypted data result from this “service” and pass it as the output.

 

Can anyone enlighten me on a potential solution for this?

 

Thanks and Regards,

 

Leon Funnell


________________________________________________________
This e-mail is confidential and intended solely for the use of the individual(s) to whom it is addressed. If you are not the intended recipient, be advised that you have received this e-mail in error and that any use, dissemination, forwarding, printing, copying of, or any action taken in reliance upon it, is strictly prohibited and may be illegal.

Catlin Underwriting Agencies Limited and Catlin Insurance Company (UK) Ltd. are authorised and regulated by the Financial Services Authority.

The registered office of Catlin Underwriting Agencies Limited (incorporated and registered in England and Wales with company number 1815126) and Catlin Insurance Company (UK) Ltd. (incorporated and registered in England and Wales with company number 5328622) is 20 Gracechurch Street, London, EC3V 0BG.

Catlin Risk Solutions Limited is an Appointed Representative of Catlin Underwriting Agencies Limited.
________________________________________________________
Reply | Threaded
Open this post in threaded view
|

RE: How can I pass data to a running instance of OpenSSL CLI on Windows within a batch file?

J. J. Farrell-2

If you start openssl.exe, that's the mode it's in by default - waiting for commands from stdin, writing the output from those commands to stdout. Isn't that what you're looking for?

 

If you're looking for advice on the programming details of attaching to its stdin and stdout and sending/receiving that data from another program, you'd probably be better asking on a general Windows programming list where there'll be more people with that sort of expertise.

 

Regards,

                          jjf

 

From: Funnell, Leon [mailto:[hidden email]]
Sent: Monday, October 22, 2012 10:52 AM
To: [hidden email]
Subject: How can I pass data to a running instance of OpenSSL CLI on Windows within a batch file?

 

We have Windows application which passes data to OpenSSL.exe to encrypt as a Windows command, then scrapes the encrypted data back from the output.  The Windows app can call external Windows commands but we cannot call APIs or extend the functionality programmatically.   Functionally it works, but it doesn’t scale as each time you call OpenSSL.exe it takes about a second and spikes the CPU.  The application we are using is required to process 6000 records every hour. 

 

I have two tests set up:

1.       A batch file which runs 6000 times, repeatedly running the following command:

Openssl.exe aes-256-cbc -a -e -k eiccmkjd94jfgniw03ljkdlfutcnv320 –in test.txt

 

2.       A text file with the following line repeated 6000 times, which I paste into the OpenSSL CLI:

aes-256-cbc -a -e -k eiccmkjd94jfgniw03ljkdlfutcnv320 –in test.txt

 

When I use the batch file which invokes OpenSSL.exe 6000 times, it takes several hours to complete and spikes the CPU significantly.  It seems to be the initialisation of the OpenSSL.exe program rather than the encryption however, as if I paste in the text file to the OpenSSL.exe CLI it completes in several seconds and takes very little CPU.

 

What I need is a way of running OpenSSL.exe as a process which I can pass parameters to on STDIN, and output parameters to STDOUT.  I would like to be able to call another batch file or program with the unencrypted data as the input parameter which would then pass this to the running “service”, retrieve the  encrypted data result from this “service” and pass it as the output.

 

Can anyone enlighten me on a potential solution for this?

 

Thanks and Regards,

 

Leon Funnell


________________________________________________________
This e-mail is confidential and intended solely for the use of the individual(s) to whom it is addressed. If you are not the intended recipient, be advised that you have received this e-mail in error and that any use, dissemination, forwarding, printing, copying of, or any action taken in reliance upon it, is strictly prohibited and may be illegal.

Catlin Underwriting Agencies Limited and Catlin Insurance Company (UK) Ltd. are authorised and regulated by the Financial Services Authority.

The registered office of Catlin Underwriting Agencies Limited (incorporated and registered in England and Wales with company number 1815126) and Catlin Insurance Company (UK) Ltd. (incorporated and registered in England and Wales with company number 5328622) is 20 Gracechurch Street, London, EC3V 0BG.

Catlin Risk Solutions Limited is an Appointed Representative of Catlin Underwriting Agencies Limited.
________________________________________________________

Reply | Threaded
Open this post in threaded view
|

RE: How can I pass data to a running instance of OpenSSL CLI on Windows within a batch file?

Charles Mills
Msdn.com is excellent. Good advice, few flames.
--
Sent from my mobile phone. Please excuse my brevity.

Charles

Jeremy Farrell <[hidden email]> wrote:

If you start openssl.exe, that's the mode it's in by default - waiting for commands from stdin, writing the output from those commands to stdout. Isn't that what you're looking for?

 

If you're looking for advice on the programming details of attaching to its stdin and stdout and sending/receiving that data from another program, you'd probably be better asking on a general Windows programming list where there'll be more people with that sort of expertise.

 

Regards,

                          jjf

 

From: Funnell, Leon [mailto:[hidden email]]
Sent: Monday, October 22, 2012 10:52 AM
To: [hidden email]
Subject: How can I pass data to a running instance of OpenSSL CLI on Windows within a batch file?

 

We have Windows application which passes data to OpenSSL.exe to encrypt as a Windows command, then scrapes the encrypted data back from the output.  The Windows app can call external Windows commands but we cannot call APIs or extend the functionality programmatically.   Functionally it works, but it doesn’t scale as each time you call OpenSSL.exe it takes about a second and spikes the CPU.  The application we are using is required to process 6000 records every hour. 

 

I have two tests set up:

1.       A batch file which runs 6000 times, repeatedly running the following command:

Openssl.exe aes-256-cbc -a -e -k eiccmkjd94jfgniw03ljkdlfutcnv320 –in test.txt

 

2.       A text file with the following line repeated 6000 times, which I paste into the OpenSSL CLI:

aes-256-cbc -a -e -k eiccmkjd94jfgniw03ljkdlfutcnv320 –in test.txt

 

When I use the batch file which invokes OpenSSL.exe 6000 times, it takes several hours to complete and spikes the CPU significantly.  It seems to be the initialisation of the OpenSSL.exe program rather than the encryption however, as if I paste in the text file to the OpenSSL.exe CLI it completes in several seconds and takes very little CPU.

 

What I need is a way of running OpenSSL.exe as a process which I can pass parameters to on STDIN, and output parameters to STDOUT.  I would like to be able to call another batch file or program with the unencrypted data as the input parameter which would then pass this to the running “service”, retrieve the  encrypted data result from this “service” and pass it as the output.

 

Can anyone enlighten me on a potential solution for this?

 

Thanks and Regards,

 

Leon Funnell


________________________________________________________
This e-mail is confidential and intended solely for the use of the individual(s) to whom it is addressed. If you are not the intended recipient, be advised that you have received this e-mail in error and that any use, dissemination, forwarding, printing, copying of, or any action taken in reliance upon it, is strictly prohibited and may be illegal.

Catlin Underwriting Agencies Limited and Catlin Insurance Company (UK) Ltd. are authorised and regulated by the Financial Services Authority.

The registered office of Catlin Underwriting Agencies Limited (incorporated and registered in England and Wales with company number 1815126) and Catlin Insurance Company (UK) Ltd. (incorporated and registered in England and Wales with company number 5328622) is 20 Gracechurch Street, London, EC3V 0BG.

Catlin Risk Solutions Limited is an Appointed Representative of Catlin Underwriting Agencies Limited.
________________________________________________________

Reply | Threaded
Open this post in threaded view
|

RE: How can I pass data to a running instance of OpenSSL CLI on Windows within a batch file?

Funnell, Leon

Got it working (almost) in vbscript.  I have the following problem however:

 

If I run Openssl.exe on it’s own waiting for input, I can tell it to do one encryption only.  See the steps I have followed below:

1.       In Windows, run CMD.exe

2.       Cd to C:\OpenSSL-Win64\bin

3.       Invoke openssl.exe

4.       Type “aes-256-cbc -a -e -K 656963636D6B6A6439346A66676E697730336C6A6B646C667574636E76333230 -iv 0

5.       Enter text “12345678” and press Enter

6.       Press Ctrl-Z and enter

7.       Press Ctrl-Z and enter

8.       Outputs “6+gAsG2gj13Jsvujnfyasg==” – this is the encrypted value of “12345678<CR>” – need this without <CR>

9.       If I the type “aes-256-cbc -a -e -K 656963636D6B6A6439346A66676E697730336C6A6B646C667574636E76333230 -iv 0” a second time, I get “non-hex digit, invalid hex iv value, error in aes-256-cbc”

10.   Then there is no way to exit openssl except pressing CTRL-C

 

If I follow the same above process in vbscript using oExec.StdIn.Write, I can get it to work without the <CR>, as it seems to understand the EOT (character 26) marker on the end of the line.  The problem is, it returns to the Openssl> prompt, but again wont work a following time due to the error “non-hex digit, invalid hex iv value, error in aes-256-cbc”

 

Any ideas?  This looks like a bug with the CLI, unless you need to change the syntax second time round.

 

Leon

 

 

 

From: [hidden email] [mailto:[hidden email]] On Behalf Of Charles Mills
Sent: Tuesday, October 23, 2012 12:12 AM
To: [hidden email]; [hidden email]
Subject: RE: How can I pass data to a running instance of OpenSSL CLI on Windows within a batch file?

 

Msdn.com is excellent. Good advice, few flames.
--
Sent from my mobile phone. Please excuse my brevity.

Charles

Jeremy Farrell <[hidden email]> wrote:

If you start openssl.exe, that's the mode it's in by default - waiting for commands from stdin, writing the output from those commands to stdout. Isn't that what you're looking for?

 

If you're looking for advice on the programming details of attaching to its stdin and stdout and sending/receiving that data from another program, you'd probably be better asking on a general Windows programming list where there'll be more people with that sort of expertise.

 

Regards,

                          jjf

 

From: Funnell, Leon [hidden email]
Sent: Monday, October 22, 2012 10:52 AM
To: [hidden email]
Subject: How can I pass data to a running instance of OpenSSL CLI on Windows within a batch file?

 

We have Windows application which passes data to OpenSSL.exe to encrypt as a Windows command, then scrapes the encrypted data back from the output.  The Windows app can call external Windows commands but we cannot call APIs or extend the functionality programmatically.   Functionally it works, but it doesn’t scale as each time you call OpenSSL.exe it takes about a second and spikes the CPU.  The application we are using is required to process 6000 records every hour. 

 

I have two tests set up:

1.       A batch file which runs 6000 times, repeatedly running the following command:

Openssl.exe aes-256-cbc -a -e -k eiccmkjd94jfgniw03ljkdlfutcnv320 –in test.txt

 

2.       A text file with the following line repeated 6000 times, which I paste into the OpenSSL CLI:

aes-256-cbc -a -e -k eiccmkjd94jfgniw03ljkdlfutcnv320 –in test.txt

 

When I use the batch file which invokes OpenSSL.exe 6000 times, it takes several hours to complete and spikes the CPU significantly.  It seems to be the initialisation of the OpenSSL.exe program rather than the encryption however, as if I paste in the text file to the OpenSSL.exe CLI it completes in several seconds and takes very little CPU.

 

What I need is a way of running OpenSSL.exe as a process which I can pass parameters to on STDIN, and output parameters to STDOUT.  I would like to be able to call another batch file or program with the unencrypted data as the input parameter which would then pass this to the running “service”, retrieve the  encrypted data result from this “service” and pass it as the output.

 

Can anyone enlighten me on a potential solution for this?

 

Thanks and Regards,

 

Leon Funnell


________________________________________________________
This e-mail is confidential and intended solely for the use of the individual(s) to whom it is addressed. If you are not the intended recipient, be advised that you have received this e-mail in error and that any use, dissemination, forwarding, printing, copying of, or any action taken in reliance upon it, is strictly prohibited and may be illegal.

Catlin Underwriting Agencies Limited and Catlin Insurance Company (UK) Ltd. are authorised and regulated by the Financial Services Authority.

The registered office of Catlin Underwriting Agencies Limited (incorporated and registered in England and Wales with company number 1815126) and Catlin Insurance Company (UK) Ltd. (incorporated and registered in England and Wales with company number 5328622) is 20 Gracechurch Street, London, EC3V 0BG.

Catlin Risk Solutions Limited is an Appointed Representative of Catlin Underwriting Agencies Limited.
________________________________________________________


________________________________________________________
This e-mail is confidential and intended solely for the use of the individual(s) to whom it is addressed. If you are not the intended recipient, be advised that you have received this e-mail in error and that any use, dissemination, forwarding, printing, copying of, or any action taken in reliance upon it, is strictly prohibited and may be illegal.

Catlin Underwriting Agencies Limited and Catlin Insurance Company (UK) Ltd. are authorised and regulated by the Financial Services Authority.

The registered office of Catlin Underwriting Agencies Limited (incorporated and registered in England and Wales with company number 1815126) and Catlin Insurance Company (UK) Ltd. (incorporated and registered in England and Wales with company number 5328622) is 20 Gracechurch Street, London, EC3V 0BG.

Catlin Risk Solutions Limited is an Appointed Representative of Catlin Underwriting Agencies Limited.
________________________________________________________
Reply | Threaded
Open this post in threaded view
|

RE: How can I pass data to a running instance of OpenSSL CLI on Windows within a batch file?

Carl Young

Do have really have to use OpenSSL.exe or could you create/use a modified version of that tool that does exactly what you expect?

 

Your "scaling" problem is because of the entropy gathering each time OpenSSL is launched. This takes a significant amount of time, especially compared to the actual encryption operation. I would think that creating a service process that responded to a CLI client tool would be more efficacious than trying to scrape the OpenSSL output, but that's just my opinion. Without knowing your exact restrictions, it's hard to suggest what to do.

 

If you don't want the overhead of installing services, and everything will be run from the same window session, you could have the first instance of your process register a global object and stay resident waiting for LRPC calls from secondary instances (or even use files and events - whatever floats your boat).

 

Carl

 


From: [hidden email] [[hidden email]] on behalf of Funnell, Leon [[hidden email]]
Sent: 12 November 2012 11:18
To: [hidden email]
Subject: RE: How can I pass data to a running instance of OpenSSL CLI on Windows within a batch file?

Got it working (almost) in vbscript.  I have the following problem however:

 

If I run Openssl.exe on it’s own waiting for input, I can tell it to do one encryption only.  See the steps I have followed below:

1.       In Windows, run CMD.exe

2.       Cd to C:\OpenSSL-Win64\bin

3.       Invoke openssl.exe

4.       Type “aes-256-cbc -a -e -K 656963636D6B6A6439346A66676E697730336C6A6B646C667574636E76333230 -iv 0

5.       Enter text “12345678” and press Enter

6.       Press Ctrl-Z and enter

7.       Press Ctrl-Z and enter

8.       Outputs “6+gAsG2gj13Jsvujnfyasg==” – this is the encrypted value of “12345678<CR>” – need this without <CR>

9.       If I the type “aes-256-cbc -a -e -K 656963636D6B6A6439346A66676E697730336C6A6B646C667574636E76333230 -iv 0” a second time, I get “non-hex digit, invalid hex iv value, error in aes-256-cbc”

10.   Then there is no way to exit openssl except pressing CTRL-C

 

If I follow the same above process in vbscript using oExec.StdIn.Write, I can get it to work without the <CR>, as it seems to understand the EOT (character 26) marker on the end of the line.  The problem is, it returns to the Openssl> prompt, but again wont work a following time due to the error “non-hex digit, invalid hex iv value, error in aes-256-cbc”

 

Any ideas?  This looks like a bug with the CLI, unless you need to change the syntax second time round.

 

Leon

 

 

 

From: [hidden email] [mailto:[hidden email]] On Behalf Of Charles Mills
Sent: Tuesday, October 23, 2012 12:12 AM
To: [hidden email]; [hidden email]
Subject: RE: How can I pass data to a running instance of OpenSSL CLI on Windows within a batch file?

 

Msdn.com is excellent. Good advice, few flames.
--
Sent from my mobile phone. Please excuse my brevity.

Charles

Jeremy Farrell <[hidden email]> wrote:

If you start openssl.exe, that's the mode it's in by default - waiting for commands from stdin, writing the output from those commands to stdout. Isn't that what you're looking for?

 

If you're looking for advice on the programming details of attaching to its stdin and stdout and sending/receiving that data from another program, you'd probably be better asking on a general Windows programming list where there'll be more people with that sort of expertise.

 

Regards,

                          jjf

 

From: Funnell, Leon [hidden email]
Sent: Monday, October 22, 2012 10:52 AM
To: [hidden email]
Subject: How can I pass data to a running instance of OpenSSL CLI on Windows within a batch file?

 

We have Windows application which passes data to OpenSSL.exe to encrypt as a Windows command, then scrapes the encrypted data back from the output.  The Windows app can call external Windows commands but we cannot call APIs or extend the functionality programmatically.   Functionally it works, but it doesn’t scale as each time you call OpenSSL.exe it takes about a second and spikes the CPU.  The application we are using is required to process 6000 records every hour. 

 

I have two tests set up:

1.       A batch file which runs 6000 times, repeatedly running the following command:

Openssl.exe aes-256-cbc -a -e -k eiccmkjd94jfgniw03ljkdlfutcnv320 –in test.txt

 

2.       A text file with the following line repeated 6000 times, which I paste into the OpenSSL CLI:

aes-256-cbc -a -e -k eiccmkjd94jfgniw03ljkdlfutcnv320 –in test.txt

 

When I use the batch file which invokes OpenSSL.exe 6000 times, it takes several hours to complete and spikes the CPU significantly.  It seems to be the initialisation of the OpenSSL.exe program rather than the encryption however, as if I paste in the text file to the OpenSSL.exe CLI it completes in several seconds and takes very little CPU.

 

What I need is a way of running OpenSSL.exe as a process which I can pass parameters to on STDIN, and output parameters to STDOUT.  I would like to be able to call another batch file or program with the unencrypted data as the input parameter which would then pass this to the running “service”, retrieve the  encrypted data result from this “service” and pass it as the output.

 

Can anyone enlighten me on a potential solution for this?

 

Thanks and Regards,

 

Leon Funnell


________________________________________________________
This e-mail is confidential and intended solely for the use of the individual(s) to whom it is addressed. If you are not the intended recipient, be advised that you have received this e-mail in error and that any use, dissemination, forwarding, printing, copying of, or any action taken in reliance upon it, is strictly prohibited and may be illegal.

Catlin Underwriting Agencies Limited and Catlin Insurance Company (UK) Ltd. are authorised and regulated by the Financial Services Authority.

The registered office of Catlin Underwriting Agencies Limited (incorporated and registered in England and Wales with company number 1815126) and Catlin Insurance Company (UK) Ltd. (incorporated and registered in England and Wales with company number 5328622) is 20 Gracechurch Street, London, EC3V 0BG.

Catlin Risk Solutions Limited is an Appointed Representative of Catlin Underwriting Agencies Limited.
________________________________________________________


________________________________________________________
This e-mail is confidential and intended solely for the use of the individual(s) to whom it is addressed. If you are not the intended recipient, be advised that you have received this e-mail in error and that any use, dissemination, forwarding, printing, copying of, or any action taken in reliance upon it, is strictly prohibited and may be illegal.

Catlin Underwriting Agencies Limited and Catlin Insurance Company (UK) Ltd. are authorised and regulated by the Financial Services Authority.

The registered office of Catlin Underwriting Agencies Limited (incorporated and registered in England and Wales with company number 1815126) and Catlin Insurance Company (UK) Ltd. (incorporated and registered in England and Wales with company number 5328622) is 20 Gracechurch Street, London, EC3V 0BG.

Catlin Risk Solutions Limited is an Appointed Representative of Catlin Underwriting Agencies Limited.
________________________________________________________
Reply | Threaded
Open this post in threaded view
|

RE: How can I pass data to a running instance of OpenSSL CLI on Windows within a batch file?

Funnell, Leon

Carl,

 

Thanks for your reply.  I would love to have a service which I could call, unfortunately my Windows development skills don’t stretch that far as I am an infrastructure person with some basic VBscripting skills.

 

Do you (or another member of this list as CC’ed) have something that could be used for this purpose? 

 

Below are my requirements:

1.       I do actually need to call SOMETHING from the command line and use STDIN and STDOUT as that is all my windows application supports

2.       I would suggest that I call a small wrapper application with the text to be encrypted and the key as STDIN, which then messages a running service with the required text and key, receives back the encrypted text, then passes this back to the command line as STDOUT

3.       The wrapper application would need to be written to be completely self-contained and lightweight.  Using a Windows SDK would likely incur similar startup issues, so it might need to be written in C or something similar which is fully linked

 

Another option – could Openssl.exe be made to store it’s entropy information in a file or registry location which could be read in every time rather than recreated?  Is this information different every time?  I don’t really need randomness  or salt as I am using it only for symmetric encryption which is the same every time.  The far end (a third party) for the data synchronisation only supports symmetric encryption and no salt.

 

Thanks and Regards,

 

Leon

 

 

 

From: Carl Young [mailto:[hidden email]]
Sent: Tuesday, November 13, 2012 4:00 PM
To: Funnell, Leon
Subject: RE: How can I pass data to a running instance of OpenSSL CLI on Windows within a batch file?

 

Sorry - I meant to add "my reply was either discarded from the group list or is still waiting for an OK"

 

Carl


From: Carl Young
Sent: 13 November 2012 15:59
To: [hidden email]
Subject: FW: How can I pass data to a running instance of OpenSSL CLI on Windows within a batch file?

 


From: Carl Young
Sent: 12 November 2012 11:43
To: [hidden email]
Subject: RE: How can I pass data to a running instance of OpenSSL CLI on Windows within a batch file?

Do have really have to use OpenSSL.exe or could you create/use a modified version of that tool that does exactly what you expect?

 

Your "scaling" problem is because of the entropy gathering each time OpenSSL is launched. This takes a significant amount of time, especially compared to the actual encryption operation. I would think that creating a service process that responded to a CLI client tool would be more efficacious than trying to scrape the OpenSSL output, but that's just my opinion. Without knowing your exact restrictions, it's hard to suggest what to do.

 

If you don't want the overhead of installing services, and everything will be run from the same window session, you could have the first instance of your process register a global object and stay resident waiting for LRPC calls from secondary instances (or even use files and events - whatever floats your boat).

 

Carl

 


From: [hidden email] [[hidden email]] on behalf of Funnell, Leon [[hidden email]]
Sent: 12 November 2012 11:18
To: [hidden email]
Subject: RE: How can I pass data to a running instance of OpenSSL CLI on Windows within a batch file?

Got it working (almost) in vbscript.  I have the following problem however:

 

If I run Openssl.exe on it’s own waiting for input, I can tell it to do one encryption only.  See the steps I have followed below:

1.       In Windows, run CMD.exe

2.       Cd to C:\OpenSSL-Win64\bin

3.       Invoke openssl.exe

4.       Type “aes-256-cbc -a -e -K 656963636D6B6A6439346A66676E697730336C6A6B646C667574636E76333230 -iv 0

5.       Enter text “12345678” and press Enter

6.       Press Ctrl-Z and enter

7.       Press Ctrl-Z and enter

8.       Outputs “6+gAsG2gj13Jsvujnfyasg==” – this is the encrypted value of “12345678<CR>” – need this without <CR>

9.       If I the type “aes-256-cbc -a -e -K 656963636D6B6A6439346A66676E697730336C6A6B646C667574636E76333230 -iv 0” a second time, I get “non-hex digit, invalid hex iv value, error in aes-256-cbc”

10.   Then there is no way to exit openssl except pressing CTRL-C

 

If I follow the same above process in vbscript using oExec.StdIn.Write, I can get it to work without the <CR>, as it seems to understand the EOT (character 26) marker on the end of the line.  The problem is, it returns to the Openssl> prompt, but again wont work a following time due to the error “non-hex digit, invalid hex iv value, error in aes-256-cbc”

 

Any ideas?  This looks like a bug with the CLI, unless you need to change the syntax second time round.

 

Leon

 

 

 

From: [hidden email] [hidden email] On Behalf Of Charles Mills
Sent: Tuesday, October 23, 2012 12:12 AM
To: [hidden email]; [hidden email]
Subject: RE: How can I pass data to a running instance of OpenSSL CLI on Windows within a batch file?

 

Msdn.com is excellent. Good advice, few flames.
--
Sent from my mobile phone. Please excuse my brevity.

Charles

Jeremy Farrell <[hidden email]> wrote:

If you start openssl.exe, that's the mode it's in by default - waiting for commands from stdin, writing the output from those commands to stdout. Isn't that what you're looking for?

 

If you're looking for advice on the programming details of attaching to its stdin and stdout and sending/receiving that data from another program, you'd probably be better asking on a general Windows programming list where there'll be more people with that sort of expertise.

 

Regards,

                          jjf

 

From: Funnell, Leon [hidden email]
Sent: Monday, October 22, 2012 10:52 AM
To: [hidden email]
Subject: How can I pass data to a running instance of OpenSSL CLI on Windows within a batch file?

 

We have Windows application which passes data to OpenSSL.exe to encrypt as a Windows command, then scrapes the encrypted data back from the output.  The Windows app can call external Windows commands but we cannot call APIs or extend the functionality programmatically.   Functionally it works, but it doesn’t scale as each time you call OpenSSL.exe it takes about a second and spikes the CPU.  The application we are using is required to process 6000 records every hour. 

 

I have two tests set up:

1.       A batch file which runs 6000 times, repeatedly running the following command:

Openssl.exe aes-256-cbc -a -e -k eiccmkjd94jfgniw03ljkdlfutcnv320 –in test.txt

 

2.       A text file with the following line repeated 6000 times, which I paste into the OpenSSL CLI:

aes-256-cbc -a -e -k eiccmkjd94jfgniw03ljkdlfutcnv320 –in test.txt

 

When I use the batch file which invokes OpenSSL.exe 6000 times, it takes several hours to complete and spikes the CPU significantly.  It seems to be the initialisation of the OpenSSL.exe program rather than the encryption however, as if I paste in the text file to the OpenSSL.exe CLI it completes in several seconds and takes very little CPU.

 

What I need is a way of running OpenSSL.exe as a process which I can pass parameters to on STDIN, and output parameters to STDOUT.  I would like to be able to call another batch file or program with the unencrypted data as the input parameter which would then pass this to the running “service”, retrieve the  encrypted data result from this “service” and pass it as the output.

 

Can anyone enlighten me on a potential solution for this?

 

Thanks and Regards,

 

Leon Funnell


________________________________________________________
This e-mail is confidential and intended solely for the use of the individual(s) to whom it is addressed. If you are not the intended recipient, be advised that you have received this e-mail in error and that any use, dissemination, forwarding, printing, copying of, or any action taken in reliance upon it, is strictly prohibited and may be illegal.

Catlin Underwriting Agencies Limited and Catlin Insurance Company (UK) Ltd. are authorised and regulated by the Financial Services Authority.

The registered office of Catlin Underwriting Agencies Limited (incorporated and registered in England and Wales with company number 1815126) and Catlin Insurance Company (UK) Ltd. (incorporated and registered in England and Wales with company number 5328622) is 20 Gracechurch Street, London, EC3V 0BG.

Catlin Risk Solutions Limited is an Appointed Representative of Catlin Underwriting Agencies Limited.
________________________________________________________


________________________________________________________
This e-mail is confidential and intended solely for the use of the individual(s) to whom it is addressed. If you are not the intended recipient, be advised that you have received this e-mail in error and that any use, dissemination, forwarding, printing, copying of, or any action taken in reliance upon it, is strictly prohibited and may be illegal.

Catlin Underwriting Agencies Limited and Catlin Insurance Company (UK) Ltd. are authorised and regulated by the Financial Services Authority.

The registered office of Catlin Underwriting Agencies Limited (incorporated and registered in England and Wales with company number 1815126) and Catlin Insurance Company (UK) Ltd. (incorporated and registered in England and Wales with company number 5328622) is 20 Gracechurch Street, London, EC3V 0BG.

Catlin Risk Solutions Limited is an Appointed Representative of Catlin Underwriting Agencies Limited.
________________________________________________________


________________________________________________________
This e-mail is confidential and intended solely for the use of the individual(s) to whom it is addressed. If you are not the intended recipient, be advised that you have received this e-mail in error and that any use, dissemination, forwarding, printing, copying of, or any action taken in reliance upon it, is strictly prohibited and may be illegal.

Catlin Underwriting Agencies Limited and Catlin Insurance Company (UK) Ltd. are authorised and regulated by the Financial Services Authority.

The registered office of Catlin Underwriting Agencies Limited (incorporated and registered in England and Wales with company number 1815126) and Catlin Insurance Company (UK) Ltd. (incorporated and registered in England and Wales with company number 5328622) is 20 Gracechurch Street, London, EC3V 0BG.

Catlin Risk Solutions Limited is an Appointed Representative of Catlin Underwriting Agencies Limited.
________________________________________________________
Reply | Threaded
Open this post in threaded view
|

Re: How can I pass data to a running instance of OpenSSL CLI on Windows within a batch file?

John Zavgren
In reply to this post by Funnell, Leon
Leon:
I suggest that you write a program that uses file descriptors for IO? I'd write it in C.


On Mon, Oct 22, 2012 at 5:51 AM, Funnell, Leon <[hidden email]> wrote:

We have Windows application which passes data to OpenSSL.exe to encrypt as a Windows command, then scrapes the encrypted data back from the output.  The Windows app can call external Windows commands but we cannot call APIs or extend the functionality programmatically.   Functionally it works, but it doesn’t scale as each time you call OpenSSL.exe it takes about a second and spikes the CPU.  The application we are using is required to process 6000 records every hour. 

 

I have two tests set up:

1.       A batch file which runs 6000 times, repeatedly running the following command:

Openssl.exe aes-256-cbc -a -e -k eiccmkjd94jfgniw03ljkdlfutcnv320 –in test.txt

 

2.       A text file with the following line repeated 6000 times, which I paste into the OpenSSL CLI:

aes-256-cbc -a -e -k eiccmkjd94jfgniw03ljkdlfutcnv320 –in test.txt

 

When I use the batch file which invokes OpenSSL.exe 6000 times, it takes several hours to complete and spikes the CPU significantly.  It seems to be the initialisation of the OpenSSL.exe program rather than the encryption however, as if I paste in the text file to the OpenSSL.exe CLI it completes in several seconds and takes very little CPU.

 

What I need is a way of running OpenSSL.exe as a process which I can pass parameters to on STDIN, and output parameters to STDOUT.  I would like to be able to call another batch file or program with the unencrypted data as the input parameter which would then pass this to the running “service”, retrieve the  encrypted data result from this “service” and pass it as the output.

 

Can anyone enlighten me on a potential solution for this?

 

Thanks and Regards,

 

Leon Funnell


________________________________________________________
This e-mail is confidential and intended solely for the use of the individual(s) to whom it is addressed. If you are not the intended recipient, be advised that you have received this e-mail in error and that any use, dissemination, forwarding, printing, copying of, or any action taken in reliance upon it, is strictly prohibited and may be illegal.

Catlin Underwriting Agencies Limited and Catlin Insurance Company (UK) Ltd. are authorised and regulated by the Financial Services Authority.

The registered office of Catlin Underwriting Agencies Limited (incorporated and registered in England and Wales with company number 1815126) and Catlin Insurance Company (UK) Ltd. (incorporated and registered in England and Wales with company number 5328622) is 20 Gracechurch Street, London, EC3V 0BG.

Catlin Risk Solutions Limited is an Appointed Representative of Catlin Underwriting Agencies Limited.
________________________________________________________



--
No amount of believing makes something a fact. James Randi


John Zavgren
603-371-0513 (home)
603-801-2094 (cell)
johnzavgren (skype)
603-821-0904 (skype)
[hidden email]

Reply | Threaded
Open this post in threaded view
|

RE: How can I pass data to a running instance of OpenSSL CLI on Windows within a batch file?

Carl Young
I would hope that one of us could provide you something given a week or so... very busy with work currently but  I'm sure I could do something in time.

Carl

From: [hidden email] [[hidden email]] on behalf of John Zavgren [[hidden email]]
Sent: 14 November 2012 12:08
To: [hidden email]
Subject: Re: How can I pass data to a running instance of OpenSSL CLI on Windows within a batch file?


Leon:
I suggest that you write a program that uses file descriptors for IO? I'd write it in C.



On Mon, Oct 22, 2012 at 5:51 AM, Funnell, Leon <[hidden email]> wrote:

We have Windows application which passes data to OpenSSL.exe to encrypt as a Windows command, then scrapes the encrypted data back from the output.  The Windows app can call external Windows commands but we cannot call APIs or extend the functionality programmatically.   Functionally it works, but it doesn’t scale as each time you call OpenSSL.exe it takes about a second and spikes the CPU.  The application we are using is required to process 6000 records every hour.  

I have two tests set up:
1.       A batch file which runs 6000 times, repeatedly running the following command:
Openssl.exe aes-256-cbc -a -e -k eiccmkjd94jfgniw03ljkdlfutcnv320 –in test.txt

2.       A text file with the following line repeated 6000 times, which I paste into the OpenSSL CLI:
aes-256-cbc -a -e -k eiccmkjd94jfgniw03ljkdlfutcnv320 –in test.txt

When I use the batch file which invokes OpenSSL.exe 6000 times, it takes several hours to complete and spikes the CPU significantly.  It seems to be the initialisation of the OpenSSL.exe program rather than the encryption however, as if I paste in the text file to the OpenSSL.exe CLI it completes in several seconds and takes very little CPU.

What I need is a way of running OpenSSL.exe as a process which I can pass parameters to on STDIN, and output parameters to STDOUT.  I would like to be able to call another batch file or program with the unencrypted data as the input parameter which would then pass this to the running “service”, retrieve the  encrypted data result from this “service” and pass it as the output.

Can anyone enlighten me on a potential solution for this?

Thanks and Regards,

Leon Funnell

________________________________________________________
This e-mail is confidential and intended solely for the use of the individual(s) to whom it is addressed. If you are not the intended recipient, be advised that you have received this e-mail in error and that any use, dissemination, forwarding, printing, copying of, or any action taken in reliance upon it, is strictly prohibited and may be illegal.

Catlin Underwriting Agencies Limited and Catlin Insurance Company (UK) Ltd. are authorised and regulated by the Financial Services Authority.

The registered office of Catlin Underwriting Agencies Limited (incorporated and registered in England and Wales with company number 1815126) and Catlin Insurance Company (UK) Ltd. (incorporated and registered in England and Wales with company number 5328622) is 20 Gracechurch Street, London, EC3V 0BG.

Catlin Risk Solutions Limited is an Appointed Representative of Catlin Underwriting Agencies Limited.
________________________________________________________






--

No amount of believing makes something a fact. James Randi




John Zavgren
603-371-0513 (home)
603-801-2094 (cell)
johnzavgren (skype)
603-821-0904 (skype)
john@zavgren.com______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
User Support Mailing List                    [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: How can I pass data to a running instance of OpenSSL CLI on Windows within a batch file?

Jakob Bohm-7
In reply to this post by Funnell, Leon
(Top posting to keep this thread consistent)

Hi,

As for encrypting/decrypting a file via stdin/stdout, the openssl.exe
program can already do that (it is almost the default behavior for those
commands that encrypt/decrypt things, you may need to add the "-passin"
option to indicate if the password will be passed in via STDIN or via
an extra handle that you have opened in the parent process, marked as
"inheritable" with DuplicateHandle() and given a Microsoft C file
handle number (the mapping from C file handles such as "3" to Windows
file handles such as 0x00001234 is passed to child processes via the
"lpReserved2" field in the STARTUPINFO structure, the -passin option
uses C file handles).

Actually, using a Windows (Win32) SDK is kind of mandatory, as the C
libraries are written on top of the Windows SDK anyway.  Using the
Windows (.NET) SDK is 100% overhead on the other hand, so we really
need to check which one we are given when MS offers us a "Windows SDK".

On many platforms (ncluding Windows), openssl.exe has a bad habit of
storing its entropy in a file between invocations.  The file name is
usually ".rnd" and there might be a way to control its location.

On 11/14/2012 10:51 AM, Funnell, Leon wrote:

> Carl,
>
> Thanks for your reply.  I would love to have a service which I could
> call, unfortunately my Windows development skills don’t stretch that far
> as I am an infrastructure person with some basic VBscripting skills.
>
> Do you (or another member of this list as CC’ed) have something that
> could be used for this purpose?
>
> Below are my requirements:
>
> 1.I do actually need to call SOMETHING from the command line and use
> STDIN and STDOUT as that is all my windows application supports
>
> 2.I would suggest that I call a small wrapper application with the text
> to be encrypted and the key as STDIN, which then messages a running
> service with the required text and key, receives back the encrypted
> text, then passes this back to the command line as STDOUT
>
> 3.The wrapper application would need to be written to be completely
> self-contained and lightweight.  Using a Windows SDK would likely incur
> similar startup issues, so it might need to be written in C or something
> similar which is fully linked
>
> Another option – could Openssl.exe be made to store it’s entropy
> information in a file or registry location which could be read in every
> time rather than recreated?  Is this information different every time?
> I don’t really need randomness  or salt as I am using it only for
> symmetric encryption which is the same every time.  The far end (a third
> party) for the data synchronisation only supports symmetric encryption
> and no salt.
>
>
>> *From:*Carl Young [mailto:[hidden email]]
>> *Sent:* Tuesday, November 13, 2012 4:00 PM
>> *To:* Funnell, Leon
>> *Subject:* RE: How can I pass data to a running instance of OpenSSL CLI
>> on Windows within a batch file?
>>
>> Sorry - I meant to add "my reply was either discarded from the group
>> list or is still waiting for an OK"
>>
> > ------------------------------------------------------------------------
>>> *From:*Carl Young
>>> *Sent:* 13 November 2012 15:59
>>> *To:* [hidden email] <mailto:[hidden email]>
>>> *Subject:* FW: How can I pass data to a running instance of OpenSSL CLI
>>> on Windows within a batch file?
>>>
>>> ------------------------------------------------------------------------
>>> > *From:*Carl Young
>>>> *Sent:* 12 November 2012 11:43
>>>> *To:* <[hidden email]>
>>>> *Subject:* RE: How can I pass data to a running instance of OpenSSL CLI
>>>> on Windows within a batch file?
>>>>
>>>> Do have really /have/ to use OpenSSL.exe or could you create/use a
>>>> modified version of that tool that does exactly what you expect?
>>>>
>>>> Your "scaling" problem is because of the entropy gathering each time
>>>> OpenSSL is launched. This takes a significant amount of time, especially
>>>> compared to the actual encryption operation. I would think that creating
>>>> a service process that responded to a CLI client tool would be more
>>>> efficacious than trying to scrape the OpenSSL output, but that's just my
>>>> opinion. Without knowing your exact restrictions, it's hard to suggest
>>>> what to do.
>>>>
>>>> If you don't want the overhead of installing services, and everything
>>>> will be run from the same window session, you could have the first
>>>> instance of your process register a global object and stay resident
>>>> waiting for LRPC calls from secondary instances (or even use files and
>>>> events - whatever floats your boat).
>>>>
>>>> ------------------------------------------------------------------------
>>>>> *From:* Funnell, Leon  <[hidden email]>
>>>>> *Sent:* 12 November 2012 11:18
>>>>> *To:* <[hidden email]>
>>>>> *Subject:* RE: How can I pass data to a running instance of OpenSSL CLI
>>>>> on Windows within a batch file?
>>>>>
>>>>> Got it working (almost) in vbscript.  I have the following problem however:
>>>>>
>>>>> If I run Openssl.exe on it’s own waiting for input, I can tell it to do
>>>>> one encryption only.  See the steps I have followed below:
>>>>>
>>>>> 1.In Windows, run CMD.exe
>>>>>
>>>>> 2.Cd to C:\OpenSSL-Win64\bin
>>>>>
>>>>> 3.Invoke openssl.exe
>>>>>
>>>>> 4.Type “aes-256-cbc -a -e -K
>>>>> 656963636D6B6A6439346A66676E697730336C6A6B646C667574636E76333230 -iv 0”
>>>>>
>>>>> 5.Enter text “12345678” and press Enter
>>>>>
>>>>> 6.Press Ctrl-Z and enter
>>>>>
>>>>> 7.Press Ctrl-Z and enter
>>>>>
>>>>> 8.Outputs “6+gAsG2gj13Jsvujnfyasg==” – this is the encrypted value of
>>>>> “12345678<CR>” – need this without <CR>
>>>>>
>>>>> 9.If I the type “aes-256-cbc -a -e -K
>>>>> 656963636D6B6A6439346A66676E697730336C6A6B646C667574636E76333230 -iv 0”
>>>>> a second time, I get “non-hex digit, invalid hex iv value, error in
>>>>> aes-256-cbc”
>>>>>
>>>>> 10.Then there is no way to exit openssl except pressing CTRL-C
>>>>>
>>>>> If I follow the same above process in vbscript using oExec.StdIn.Write,
>>>>> I can get it to work without the <CR>, as it seems to understand the EOT
>>>>> (character 26) marker on the end of the line.  The problem is, it
>>>>> returns to the Openssl> prompt, but again wont work a following time due
>>>>> to the error “non-hex digit, invalid hex iv value, error in aes-256-cbc”
>>>>>
>>>>> Any ideas?  This looks like a bug with the CLI, unless you need to
>>>>> change the syntax second time round.
>>>>>
>>>>>> *From:* Charles Mills
>>>>>> *Sent:* Tuesday, October 23, 2012 12:12 AM
>>>>>> *To:* <[hidden email]>;
>>>>>> *Subject:* RE: How can I pass data to a running instance of OpenSSL CLI
>>>>>> on Windows within a batch file?
>>>>>>
>>>>>> Msdn.com <http://Msdn.com> is excellent. Good advice, few flames.
>>>>>>
>>>>>> Charles
>>>>>>
>>>>>> Jeremy Farrell <[hidden email]> wrote:
>>>>>>>
>>>>>>> If you start openssl.exe, that's the mode it's in by default - waiting
>>>>>>> for commands from stdin, writing the output from those commands to
>>>>>>> stdout. Isn't that what you're looking for?
>>>>>>>
>>>>>>> If you're looking for advice on the programming details of attaching to
>>>>>>> its stdin and stdout and sending/receiving that data from another
>>>>>>> program, you'd probably be better asking on a general Windows
>>>>>>> programming list where there'll be more people with that sort of expertise.
>>>>>>>
>>>>>> >
>>>>>>>> *From:* Funnell, Leon <[hidden email]>
>>>>>>>> *Sent:* Monday, October 22, 2012 10:52 AM
>>>>>>>> *To:* <[hidden email]>
>>>>>>>> *Subject:* How can I pass data to a running instance of OpenSSL CLI on
>>>>>>>> Windows within a batch file?
>>>>>>>>
>>>>>>>> We have Windows application which passes data to OpenSSL.exe to encrypt
>>>>>>>> as a Windows command, then scrapes the encrypted data back from the
>>>>>>>> output.  The Windows app can call external Windows commands but we
>>>>>>>> cannot call APIs or extend the functionality programmatically.
>>>>>>>> Functionally it works, but it doesn’t scale as each time you call
>>>>>>>> OpenSSL.exe it takes about a second and spikes the CPU.  The application
>>>>>>>> we are using is required to process 6000 records every hour.
>>>>>>>>
>>>>>>>> I have two tests set up:
>>>>>>>>
>>>>>>>> 1.A batch file which runs 6000 times, repeatedly running the following
>>>>>>>> command:
>>>>>>>>
>>>>>>>> *Openssl.exe aes-256-cbc -a -e -k eiccmkjd94jfgniw03ljkdlfutcnv320 –in
>>>>>>>> test.txt*
>>>>>>>>
>>>>>>>> 2.A text file with the following line repeated 6000 times, which I paste
>>>>>>>> into the OpenSSL CLI:
>>>>>>>>
>>>>>>>> *aes-256-cbc -a -e -k eiccmkjd94jfgniw03ljkdlfutcnv320 –in test.txt*
>>>>>>>>
>>>>>>>> When I use the batch file which invokes OpenSSL.exe 6000 times, it takes
>>>>>>>> several hours to complete and spikes the CPU significantly.  It seems to
>>>>>>>> be the initialisation of the OpenSSL.exe program rather than the
>>>>>>>> encryption however, as if I paste in the text file to the OpenSSL.exe
>>>>>>>> CLI it completes in several seconds and takes very little CPU.
>>>>>>>>
>>>>>>>> What I need is a way of running OpenSSL.exe as a process which I can
>>>>>>>> pass parameters to on STDIN, and output parameters to STDOUT.  I would
>>>>>>>> like to be able to call another batch file or program with the
>>>>>>>> unencrypted data as the input parameter which would then pass this to
>>>>>>>> the running “service”, retrieve the  encrypted data result from this
>>>>>>>> “service” and pass it as the output.
>>>>>>>>
>>>>>>>> Can anyone enlighten me on a potential solution for this?
>>>>>>>>


Enjoy

Jakob
--
Jakob Bohm, CIO, Partner, WiseMo A/S.  http://www.wisemo.com
Transformervej 29, 2730 Herlev, Denmark.  Direct +45 31 13 16 10
This public discussion message is non-binding and may contain errors.
WiseMo - Remote Service Management for PCs, Phones and Embedded
______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
User Support Mailing List                    [hidden email]
Automated List Manager                           [hidden email]