having a lot of troubles trying to get AES-NI working

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
13 messages Options
Reply | Threaded
Open this post in threaded view
|

having a lot of troubles trying to get AES-NI working

Ewen Chan
So this is a partial continuation from the discussion thread that I
started yesterday in regards to using AES-CBC.

I've got an Intel Core i7 3930K that supports AES-NI and I spent the
greater part of last night trying to get openssl to work or at least
recognize it, but it doesn't seem to want to do that.

I've tried with Cygwin 1.5-something (I forget) and the latest cygwin
(tried upgrading just the openssl package - didn't work; so I ended up
uninstalling my old cygwin; installing the new and it still didn't
work.)

I've also tried Ubuntu 12.04 LTS and Ubuntu 12.10 (and it said that it
downloaded the update to it and applied it, but it still didn't work
for either). I've even tried redownloading the source from
www.openssl.org/source (taking it up to 1.0.1e) and that also still
didn't work.

(i.e. "didn't work" means that when I type "openssl engine" - the
aesni doesn't show up as an option).

I then tried to modify the initramfs config file to add aes_ni and
then running and update-initramfs; and that didn't work either.

All of the probing that I tried to do showed that the AES-NI kernel
module wasn't loaded (but - for example in Solaris 11 that I've got
running in a VM; when I type "openssl engine"; it will show (aesni)
AES-NI engine (no aesni)); which leads me to think that on the Linux
side, something similiar should happen (that openssl aesni engine
should still be available but then there'd be a comment if the AES-NI
kernel module wasn't loaded).

And now I am trying to install Solaris 11 on the host system itself
and it's having it's own set of issues (hardware compatibility; which
I might have to set up a PXE boot server so that I can patch in/update
drivers for the Solaris 11 install - but that's a different story for
some other list).

And I also tried SuSE Linux Enterprise Server 11 SP1 (I think) - same thing.

cat /proc/cpuinfo shows that aes is available.
cat /proc/crypto does not.

And people (via more googling) have said that they can load kernel
modules post-boot, but I don't know how to do that either.

Any help on any recommended OS would be greatly appreciated. (It
doesn't matter so much to me which OS is used so long as the openssl
using the aesni engine works.) Thanks.
______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
User Support Mailing List                    [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: having a lot of troubles trying to get AES-NI working

Ewen Chan
But even if it isn't enabled in the BIOS, shouldn't the output be
something like this when you probe it (even if it isn't available or
enabled in BIOS, but openssl itself supports it)?

$ openssl engine
(aesni) Intel AES-NI engine (no-aesni)
(dynamic) Dynamic engine loading support


(I haven't been able to find the option in the BIOS, so I'm going to
have to send a support request to ASUS about that or maybe try and see
if I can find another tool to see whether it's there; and whether or
not it's active or not - again, different questions for different
points).

On Thu, Mar 14, 2013 at 11:28 AM, Matthew Hall <[hidden email]> wrote:

> In many cases you have to explicitly enable it in the BIOS first before it will work right.
> --
> Sent from my mobile device.
>
> Ewen Chan <[hidden email]> wrote:
>
>>So this is a partial continuation from the discussion thread that I
>>started yesterday in regards to using AES-CBC.
>>
>>I've got an Intel Core i7 3930K that supports AES-NI and I spent the
>>greater part of last night trying to get openssl to work or at least
>>recognize it, but it doesn't seem to want to do that.
>>
>>I've tried with Cygwin 1.5-something (I forget) and the latest cygwin
>>(tried upgrading just the openssl package - didn't work; so I ended up
>>uninstalling my old cygwin; installing the new and it still didn't
>>work.)
>>
>>I've also tried Ubuntu 12.04 LTS and Ubuntu 12.10 (and it said that it
>>downloaded the update to it and applied it, but it still didn't work
>>for either). I've even tried redownloading the source from
>>www.openssl.org/source (taking it up to 1.0.1e) and that also still
>>didn't work.
>>
>>(i.e. "didn't work" means that when I type "openssl engine" - the
>>aesni doesn't show up as an option).
>>
>>I then tried to modify the initramfs config file to add aes_ni and
>>then running and update-initramfs; and that didn't work either.
>>
>>All of the probing that I tried to do showed that the AES-NI kernel
>>module wasn't loaded (but - for example in Solaris 11 that I've got
>>running in a VM; when I type "openssl engine"; it will show (aesni)
>>AES-NI engine (no aesni)); which leads me to think that on the Linux
>>side, something similiar should happen (that openssl aesni engine
>>should still be available but then there'd be a comment if the AES-NI
>>kernel module wasn't loaded).
>>
>>And now I am trying to install Solaris 11 on the host system itself
>>and it's having it's own set of issues (hardware compatibility; which
>>I might have to set up a PXE boot server so that I can patch in/update
>>drivers for the Solaris 11 install - but that's a different story for
>>some other list).
>>
>>And I also tried SuSE Linux Enterprise Server 11 SP1 (I think) - same
>>thing.
>>
>>cat /proc/cpuinfo shows that aes is available.
>>cat /proc/crypto does not.
>>
>>And people (via more googling) have said that they can load kernel
>>modules post-boot, but I don't know how to do that either.
>>
>>Any help on any recommended OS would be greatly appreciated. (It
>>doesn't matter so much to me which OS is used so long as the openssl
>>using the aesni engine works.) Thanks.
>>______________________________________________________________________
>>OpenSSL Project                                 http://www.openssl.org
>>User Support Mailing List                    [hidden email]
>>Automated List Manager                           [hidden email]
>
______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
User Support Mailing List                    [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: having a lot of troubles trying to get AES-NI working

Dr. Stephen Henson
In reply to this post by Ewen Chan
On Thu, Mar 14, 2013, Ewen Chan wrote:

> So this is a partial continuation from the discussion thread that I
> started yesterday in regards to using AES-CBC.
>
> I've got an Intel Core i7 3930K that supports AES-NI and I spent the
> greater part of last night trying to get openssl to work or at least
> recognize it, but it doesn't seem to want to do that.
>

It it probably recognising it and you don't realise it. OpenSSL 1.0.1
automatically switches to AES-NI at the EVP level without going through an
explicit AES-NI ENGINE.

You can disable AES-NI detection with the environment variable:

OPENSSL_ia32cap=~0x200000200000000

You should see a considerable speed up with "openssl speed" by comparing the
two.

Steve.
--
Dr Stephen N. Henson. OpenSSL project core developer.
Commercial tech support now available see: http://www.openssl.org
______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
User Support Mailing List                    [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: having a lot of troubles trying to get AES-NI working

Ewen Chan
Does it matter whether it's ia32 or ia64 even for an x64 processor?

Shouldn't there be some way for me to check whether AES is enabled or
being used (other than running a speed test) either in dmesg or /proc/
or with openssl itself? I'm a little confused, and surprised/shocked
that there isn't a way to probe the status of whether the AES-NI is a)
present and b) enabled/utilized.

re: OPENSSL_ia32cap=~0x200000200000000
so forgive me for asking lots of dumb questions but that would be
$ set OPENSSL_ia32cap=~0x200000200000000
$ export OPENSSL_ia32cap

correct?

And how do I re-enable it without having to reboot the system? What's
the value that I should be putting in on the right-hand-side of the
equal sign?

Your help is much appreciated.

Sincerely,
Ewen

On Thu, Mar 14, 2013 at 7:35 PM, Dr. Stephen Henson <[hidden email]> wrote:

> On Thu, Mar 14, 2013, Ewen Chan wrote:
>
>> So this is a partial continuation from the discussion thread that I
>> started yesterday in regards to using AES-CBC.
>>
>> I've got an Intel Core i7 3930K that supports AES-NI and I spent the
>> greater part of last night trying to get openssl to work or at least
>> recognize it, but it doesn't seem to want to do that.
>>
>
> It it probably recognising it and you don't realise it. OpenSSL 1.0.1
> automatically switches to AES-NI at the EVP level without going through an
> explicit AES-NI ENGINE.
>
> You can disable AES-NI detection with the environment variable:
>
> OPENSSL_ia32cap=~0x200000200000000
>
> You should see a considerable speed up with "openssl speed" by comparing the
> two.
>
> Steve.
> --
> Dr Stephen N. Henson. OpenSSL project core developer.
> Commercial tech support now available see: http://www.openssl.org
> ______________________________________________________________________
> OpenSSL Project                                 http://www.openssl.org
> User Support Mailing List                    [hidden email]
> Automated List Manager                           [hidden email]
______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
User Support Mailing List                    [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: [openssl-users] Re: having a lot of troubles trying to get AES-NI working

Erwann ABALEA
On a PC under Linux, you can do a "cat /proc/cpuinfo" and look for "aes" in the "flags".
On a PC under any OS, get the CPUID, and look for bit 25 of ECX.
That's not OpenSSL-related.


The use of OPENSSL_ia32cap environment variable allows you to alter the CPUID result (only inside OpenSSL), and alter its behaviour. It's not resistant to a reboot, it's only process dependant.

Compare the following results:
  • OPENSSL_ia32cap="~0x200000200000000" openssl speed -elapsed -evp aes-128-cbc
  • openssl speed -elapsed -evp aes-128-cbc

-- 
Erwann ABALEA

Le 15/03/2013 04:46, Ewen Chan a écrit :
Does it matter whether it's ia32 or ia64 even for an x64 processor?

Shouldn't there be some way for me to check whether AES is enabled or
being used (other than running a speed test) either in dmesg or /proc/
or with openssl itself? I'm a little confused, and surprised/shocked
that there isn't a way to probe the status of whether the AES-NI is a)
present and b) enabled/utilized.

re: OPENSSL_ia32cap=~0x200000200000000
so forgive me for asking lots of dumb questions but that would be
$ set OPENSSL_ia32cap=~0x200000200000000
$ export OPENSSL_ia32cap

correct?

And how do I re-enable it without having to reboot the system? What's
the value that I should be putting in on the right-hand-side of the
equal sign?

Your help is much appreciated.

Sincerely,
Ewen

On Thu, Mar 14, 2013 at 7:35 PM, Dr. Stephen Henson [hidden email] wrote:
On Thu, Mar 14, 2013, Ewen Chan wrote:

So this is a partial continuation from the discussion thread that I
started yesterday in regards to using AES-CBC.

I've got an Intel Core i7 3930K that supports AES-NI and I spent the
greater part of last night trying to get openssl to work or at least
recognize it, but it doesn't seem to want to do that.

It it probably recognising it and you don't realise it. OpenSSL 1.0.1
automatically switches to AES-NI at the EVP level without going through an
explicit AES-NI ENGINE.

You can disable AES-NI detection with the environment variable:

OPENSSL_ia32cap=~0x200000200000000

You should see a considerable speed up with "openssl speed" by comparing the
two.

Steve.
--
Dr Stephen N. Henson. OpenSSL project core developer.
Commercial tech support now available see: http://www.openssl.org
______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
User Support Mailing List                    [hidden email]
Automated List Manager                           [hidden email]
______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
User Support Mailing List                    [hidden email]
Automated List Manager                           [hidden email]


Reply | Threaded
Open this post in threaded view
|

Re: [openssl-users] Re: having a lot of troubles trying to get AES-NI working

Ewen Chan
ahhhh...okay. Gotcha.

Thanks!

On Fri, Mar 15, 2013 at 5:23 AM, Erwann Abalea
<[hidden email]> wrote:

> On a PC under Linux, you can do a "cat /proc/cpuinfo" and look for "aes" in
> the "flags".
> On a PC under any OS, get the CPUID, and look for bit 25 of ECX.
> That's not OpenSSL-related.
>
>
> The use of OPENSSL_ia32cap environment variable allows you to alter the
> CPUID result (only inside OpenSSL), and alter its behaviour. It's not
> resistant to a reboot, it's only process dependant.
>
> Compare the following results:
>
> OPENSSL_ia32cap="~0x200000200000000" openssl speed -elapsed -evp aes-128-cbc
> openssl speed -elapsed -evp aes-128-cbc
>
>
> --
> Erwann ABALEA
>
> Le 15/03/2013 04:46, Ewen Chan a écrit :
>
> Does it matter whether it's ia32 or ia64 even for an x64 processor?
> Shouldn't there be some way for me to check whether AES is enabled or
> being used (other than running a speed test) either in dmesg or /proc/
> or with openssl itself? I'm a little confused, and surprised/shocked
> that there isn't a way to probe the status of whether the AES-NI is a)
> present and b) enabled/utilized.
> re: OPENSSL_ia32cap=~0x200000200000000
> so forgive me for asking lots of dumb questions but that would be
> $ set OPENSSL_ia32cap=~0x200000200000000
> $ export OPENSSL_ia32cap
> correct?
> And how do I re-enable it without having to reboot the system? What's
> the value that I should be putting in on the right-hand-side of the
> equal sign?
> Your help is much appreciated.
> Sincerely,
> Ewen
> On Thu, Mar 14, 2013 at 7:35 PM, Dr. Stephen Henson <[hidden email]>
> wrote:
>
> On Thu, Mar 14, 2013, Ewen Chan wrote:
>
> So this is a partial continuation from the discussion thread that I
> started yesterday in regards to using AES-CBC.
> I've got an Intel Core i7 3930K that supports AES-NI and I spent the
> greater part of last night trying to get openssl to work or at least
> recognize it, but it doesn't seem to want to do that.
>
> It it probably recognising it and you don't realise it. OpenSSL 1.0.1
> automatically switches to AES-NI at the EVP level without going through an
> explicit AES-NI ENGINE.
> You can disable AES-NI detection with the environment variable:
> OPENSSL_ia32cap=~0x200000200000000
> You should see a considerable speed up with "openssl speed" by comparing the
> two.
> Steve.
> --
> Dr Stephen N. Henson. OpenSSL project core developer.
> Commercial tech support now available see: http://www.openssl.org
> ______________________________________________________________________
> OpenSSL Project                                 http://www.openssl.org
> User Support Mailing List                    [hidden email]
> Automated List Manager                           [hidden email]
>
> ______________________________________________________________________
> OpenSSL Project                                 http://www.openssl.org
> User Support Mailing List                    [hidden email]
> Automated List Manager                           [hidden email]
>
>
______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
User Support Mailing List                    [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: [openssl-users] Re: having a lot of troubles trying to get AES-NI working

Ewen Chan
In reply to this post by Erwann ABALEA
Do these number make sense or seem reasonable?

(I'm running a Core i7 3930K that's been OC'd to 4.5 GHz up from the
stock 3.2 GHz, running cygwin 1.7.17 on Windows 7 x64 Professional,
with 64 GB of DDR3-1600)

~$ OPENSSL_ia32cap="~0x200000200000000" openssl speed -elapsed -evp aes-256-cbc
You have chosen to measure elapsed time instead of user CPU time.
Doing aes-256-cbc for 3s on 16 size blocks: 50880991 aes-256-cbc's in 3.00s
Doing aes-256-cbc for 3s on 64 size blocks: 14919708 aes-256-cbc's in 3.00s
Doing aes-256-cbc for 3s on 256 size blocks: 3870779 aes-256-cbc's in 3.00s
Doing aes-256-cbc for 3s on 1024 size blocks: 981188 aes-256-cbc's in 3.00s
Doing aes-256-cbc for 3s on 8192 size blocks: 122920 aes-256-cbc's in 3.00s
OpenSSL 1.0.1e 11 Feb 2013
built on: Tue Feb 12 15:42:44 CET 2013
options:bn(64,32) rc4(8x,mmx) des(ptr,risc1,16,long) aes(partial) blowfish(idx)
compiler: i686-pc-cygwin-gcc -D_WINDLL -DOPENSSL_PIC -DZLIB
-DOPENSSL_THREADS  -DDSO_DLFCN -DHAVE_DLFCN_H -DTERMIOS -DL_ENDIAN
-fomit-frame-pointer -O3 -march=i486 -Wall -DOPENSSL_BN_ASM_PART_WORDS
-DOPENSSL_IA32_SSE2 -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_GF2m
-DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DMD5_ASM -DRMD160_ASM -DAES_ASM
-DVPAES_ASM -DWHIRLPOOL_ASM -DGHASH_ASM
The 'numbers' are in 1000s of bytes per second processed.
type             16 bytes     64 bytes    256 bytes   1024 bytes   8192 bytes
aes-256-cbc     271365.29k   318287.10k   330306.47k   334912.17k   335653.55k

~$ openssl speed -elapsed -evp aes-256-cbc
You have chosen to measure elapsed time instead of user CPU time.
Doing aes-256-cbc for 3s on 16 size blocks: 108926706 aes-256-cbc's in 3.00s
Doing aes-256-cbc for 3s on 64 size blocks: 29319700 aes-256-cbc's in 3.00s
Doing aes-256-cbc for 3s on 256 size blocks: 7428178 aes-256-cbc's in 3.00s
Doing aes-256-cbc for 3s on 1024 size blocks: 1863275 aes-256-cbc's in 3.00s
Doing aes-256-cbc for 3s on 8192 size blocks: 233116 aes-256-cbc's in 3.00s
OpenSSL 1.0.1e 11 Feb 2013
built on: Tue Feb 12 15:42:44 CET 2013
options:bn(64,32) rc4(8x,mmx) des(ptr,risc1,16,long) aes(partial) blowfish(idx)
compiler: i686-pc-cygwin-gcc -D_WINDLL -DOPENSSL_PIC -DZLIB
-DOPENSSL_THREADS  -DDSO_DLFCN -DHAVE_DLFCN_H -DTERMIOS -DL_ENDIAN
-fomit-frame-pointer -O3 -march=i486 -Wall -DOPENSSL_BN_ASM_PART_WORDS
-DOPENSSL_IA32_SSE2 -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_GF2m
-DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DMD5_ASM -DRMD160_ASM -DAES_ASM
-DVPAES_ASM -DWHIRLPOOL_ASM -DGHASH_ASM
The 'numbers' are in 1000s of bytes per second processed.
type             16 bytes     64 bytes    256 bytes   1024 bytes   8192 bytes
aes-256-cbc     580942.43k   625486.93k   633871.19k   635997.87k   636562.09k

Am I to interpret the results as even with the AES-NI disabled, that
I'm still abled to process at a minimum, 300 MB/s (for block sizes >
64 B) and that with AES-NI; it's only double that (~600-and-change
MB/s)?

And that for AES-128-CBC, with 8 kiB blocks, that I'm able to process
it at 465 MB/s without AES-NI, and 888 MB/s with?

That's MUCH faster than I expect it to be (even with AES-NI) and the
888 MB/s is faster than any available storage host-bus interface we've
got right now; so I want to make sure that I am not losing my marbles
here in trying to make some sense out of this data.

Thanks.

On Fri, Mar 15, 2013 at 5:23 AM, Erwann Abalea
<[hidden email]> wrote:

> On a PC under Linux, you can do a "cat /proc/cpuinfo" and look for "aes" in
> the "flags".
> On a PC under any OS, get the CPUID, and look for bit 25 of ECX.
> That's not OpenSSL-related.
>
>
> The use of OPENSSL_ia32cap environment variable allows you to alter the
> CPUID result (only inside OpenSSL), and alter its behaviour. It's not
> resistant to a reboot, it's only process dependant.
>
> Compare the following results:
>
> OPENSSL_ia32cap="~0x200000200000000" openssl speed -elapsed -evp aes-128-cbc
> openssl speed -elapsed -evp aes-128-cbc
>
>
> --
> Erwann ABALEA
>
> Le 15/03/2013 04:46, Ewen Chan a écrit :
>
> Does it matter whether it's ia32 or ia64 even for an x64 processor?
>
> Shouldn't there be some way for me to check whether AES is enabled or
> being used (other than running a speed test) either in dmesg or /proc/
> or with openssl itself? I'm a little confused, and surprised/shocked
> that there isn't a way to probe the status of whether the AES-NI is a)
> present and b) enabled/utilized.
>
> re: OPENSSL_ia32cap=~0x200000200000000
> so forgive me for asking lots of dumb questions but that would be
> $ set OPENSSL_ia32cap=~0x200000200000000
> $ export OPENSSL_ia32cap
>
> correct?
>
> And how do I re-enable it without having to reboot the system? What's
> the value that I should be putting in on the right-hand-side of the
> equal sign?
>
> Your help is much appreciated.
>
> Sincerely,
> Ewen
>
> On Thu, Mar 14, 2013 at 7:35 PM, Dr. Stephen Henson <[hidden email]>
> wrote:
>
> On Thu, Mar 14, 2013, Ewen Chan wrote:
>
> So this is a partial continuation from the discussion thread that I
> started yesterday in regards to using AES-CBC.
>
> I've got an Intel Core i7 3930K that supports AES-NI and I spent the
> greater part of last night trying to get openssl to work or at least
> recognize it, but it doesn't seem to want to do that.
>
> It it probably recognising it and you don't realise it. OpenSSL 1.0.1
> automatically switches to AES-NI at the EVP level without going through an
> explicit AES-NI ENGINE.
>
> You can disable AES-NI detection with the environment variable:
>
> OPENSSL_ia32cap=~0x200000200000000
>
> You should see a considerable speed up with "openssl speed" by comparing the
> two.
>
> Steve.
> --
> Dr Stephen N. Henson. OpenSSL project core developer.
> Commercial tech support now available see: http://www.openssl.org
> ______________________________________________________________________
> OpenSSL Project                                 http://www.openssl.org
> User Support Mailing List                    [hidden email]
> Automated List Manager                           [hidden email]
>
> ______________________________________________________________________
> OpenSSL Project                                 http://www.openssl.org
> User Support Mailing List                    [hidden email]
> Automated List Manager                           [hidden email]
>
>
______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
User Support Mailing List                    [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: [openssl-users] Re: having a lot of troubles trying to get AES-NI working

Matthew Hall
On Sat, Mar 16, 2013 at 12:06:07AM -0400, Ewen Chan wrote:
> That's MUCH faster than I expect it to be (even with AES-NI) and the
> 888 MB/s is faster than any available storage host-bus interface we've
> got right now; so I want to make sure that I am not losing my marbles
> here in trying to make some sense out of this data.
>
> Thanks.

Without AES-NI other still quite fast instructions are used from SSE, SSE2,
etc. So it is a boost but not the be-all end-all boost.

In most cases the Core i7 will clobber most any storage device bar the very
most costly SSDs. It's an extremely powerful CPU. If this isn't screaming fast
enough, use GCM instead of CBC, and enjoy the multicore awesome thereof. ;)

As fast at it is, 10 GBE will still do 1280 MiB / sec. But it should fill up
the 8 Gbps FC with no issues.

Matthew.
______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
User Support Mailing List                    [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: [openssl-users] Re: having a lot of troubles trying to get AES-NI working

Ewen Chan
Okay then, here's another one of my infamous dumb questions.

If that's the case, then why do we need the AES-NI instruction set?

If it's likely going to be storage and/or network bandwidth limited;
wouldn't the improvements made by introducing and incorporating the
AES-NI instruction set be kind of "wasted" in the sense that you can't
really use it to the fullest potential anyways?

If the storage/network I/O is going to be your bottleneck/limiting
factor, then regardless of whether you have AES-NI or not; you're
likely going to get the same answer in terms of speed.

(I actually copied the input file onto my SSD so as to try and
eliminate that as the bottleneck; and then write the output to the
same drive.)

(haha...it feels like that the more questions I ask, the more confused
I get...lol...)

Also, is that why (besides the fact that CBC can't be parallelized)
why it doesn't make sense or people really haven't spent too much time
or effort into trying to run AES encryption/decryption on GPGPUs?
Because it's already faster than anything else is capable of at the
moment?

On Sat, Mar 16, 2013 at 1:05 AM, Matthew Hall <[hidden email]> wrote:

> On Sat, Mar 16, 2013 at 12:06:07AM -0400, Ewen Chan wrote:
>> That's MUCH faster than I expect it to be (even with AES-NI) and the
>> 888 MB/s is faster than any available storage host-bus interface we've
>> got right now; so I want to make sure that I am not losing my marbles
>> here in trying to make some sense out of this data.
>>
>> Thanks.
>
> Without AES-NI other still quite fast instructions are used from SSE, SSE2,
> etc. So it is a boost but not the be-all end-all boost.
>
> In most cases the Core i7 will clobber most any storage device bar the very
> most costly SSDs. It's an extremely powerful CPU. If this isn't screaming fast
> enough, use GCM instead of CBC, and enjoy the multicore awesome thereof. ;)
>
> As fast at it is, 10 GBE will still do 1280 MiB / sec. But it should fill up
> the 8 Gbps FC with no issues.
>
> Matthew.
> ______________________________________________________________________
> OpenSSL Project                                 http://www.openssl.org
> User Support Mailing List                    [hidden email]
> Automated List Manager                           [hidden email]
______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
User Support Mailing List                    [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: [openssl-users] Re: having a lot of troubles trying to get AES-NI working

Matthew Hall
On Sat, Mar 16, 2013 at 01:16:23AM -0400, Ewen Chan wrote:
> Okay then, here's another one of my infamous dumb questions.
>
> If that's the case, then why do we need the AES-NI instruction set?

It's far from the first accelerated instruction set of dubious utility. ;)

Marketing... etc.

Actually, SSL / TLS performance is much more greatly increased by an RSA
accelerator. If I were Intel I would have made that first, before AES-NI,
because RSA signs and verifies consume a lot more resources and are a lot more
vulnerable to DoS than AES. But, of course, RSA is more complex.

The tech companies are not trying to make the best possible product, but the
best product that's economically feasible, which is a slightly different goal.

> If it's likely going to be storage and/or network bandwidth limited;
> wouldn't the improvements made by introducing and incorporating the
> AES-NI instruction set be kind of "wasted" in the sense that you can't
> really use it to the fullest potential anyways?

Amdahl's Law: the amount of overall improvement of performance by improving an
area is proportional to the amount that area is executed.

> If the storage/network I/O is going to be your bottleneck/limiting
> factor, then regardless of whether you have AES-NI or not; you're
> likely going to get the same answer in terms of speed.

Yes!

> Also, is that why (besides the fact that CBC can't be parallelized)
> why it doesn't make sense or people really haven't spent too much time
> or effort into trying to run AES encryption/decryption on GPGPUs?
> Because it's already faster than anything else is capable of at the
> moment?

Like Erwann said, the memory transfers would likely cost more time than using
AES-NI.
______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
User Support Mailing List                    [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: [openssl-users] Re: having a lot of troubles trying to get AES-NI working

Ewen Chan
Interesting...

Does this necessarily implicitly implies that the Rijndael cipher was
selected as the AES "winner" because it was also "simple" enough to be
fast, while meeting the security and protection requirements when they
initiated the open call for proposals?

I didn't realize that the AES and also modern processors were so fast
already. I always thought that AES was going to be a fairly slow and
compute-intensive process; and so that's why I was trying to make sure
that the AES-NI was working. Turns out, I might not even need it. #FML

(And I'm pretty sure that some of you guys were probably telling me
that, but I didn't realize it then. Now I see the light.)

On Sat, Mar 16, 2013 at 1:29 AM, Matthew Hall <[hidden email]> wrote:

> On Sat, Mar 16, 2013 at 01:16:23AM -0400, Ewen Chan wrote:
>> Okay then, here's another one of my infamous dumb questions.
>>
>> If that's the case, then why do we need the AES-NI instruction set?
>
> It's far from the first accelerated instruction set of dubious utility. ;)
>
> Marketing... etc.
>
> Actually, SSL / TLS performance is much more greatly increased by an RSA
> accelerator. If I were Intel I would have made that first, before AES-NI,
> because RSA signs and verifies consume a lot more resources and are a lot more
> vulnerable to DoS than AES. But, of course, RSA is more complex.
>
> The tech companies are not trying to make the best possible product, but the
> best product that's economically feasible, which is a slightly different goal.
>
>> If it's likely going to be storage and/or network bandwidth limited;
>> wouldn't the improvements made by introducing and incorporating the
>> AES-NI instruction set be kind of "wasted" in the sense that you can't
>> really use it to the fullest potential anyways?
>
> Amdahl's Law: the amount of overall improvement of performance by improving an
> area is proportional to the amount that area is executed.
>
>> If the storage/network I/O is going to be your bottleneck/limiting
>> factor, then regardless of whether you have AES-NI or not; you're
>> likely going to get the same answer in terms of speed.
>
> Yes!
>
>> Also, is that why (besides the fact that CBC can't be parallelized)
>> why it doesn't make sense or people really haven't spent too much time
>> or effort into trying to run AES encryption/decryption on GPGPUs?
>> Because it's already faster than anything else is capable of at the
>> moment?
>
> Like Erwann said, the memory transfers would likely cost more time than using
> AES-NI.
> ______________________________________________________________________
> OpenSSL Project                                 http://www.openssl.org
> User Support Mailing List                    [hidden email]
> Automated List Manager                           [hidden email]
______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
User Support Mailing List                    [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: [openssl-users] Re: having a lot of troubles trying to get AES-NI working

Ewen Chan
Is there a mailing list that I can ask Rijndael-specific questions?

On Sat, Mar 16, 2013 at 3:25 AM, Ewen Chan <[hidden email]> wrote:

> Interesting...
>
> Does this necessarily implicitly implies that the Rijndael cipher was
> selected as the AES "winner" because it was also "simple" enough to be
> fast, while meeting the security and protection requirements when they
> initiated the open call for proposals?
>
> I didn't realize that the AES and also modern processors were so fast
> already. I always thought that AES was going to be a fairly slow and
> compute-intensive process; and so that's why I was trying to make sure
> that the AES-NI was working. Turns out, I might not even need it. #FML
>
> (And I'm pretty sure that some of you guys were probably telling me
> that, but I didn't realize it then. Now I see the light.)
>
> On Sat, Mar 16, 2013 at 1:29 AM, Matthew Hall <[hidden email]> wrote:
>> On Sat, Mar 16, 2013 at 01:16:23AM -0400, Ewen Chan wrote:
>>> Okay then, here's another one of my infamous dumb questions.
>>>
>>> If that's the case, then why do we need the AES-NI instruction set?
>>
>> It's far from the first accelerated instruction set of dubious utility. ;)
>>
>> Marketing... etc.
>>
>> Actually, SSL / TLS performance is much more greatly increased by an RSA
>> accelerator. If I were Intel I would have made that first, before AES-NI,
>> because RSA signs and verifies consume a lot more resources and are a lot more
>> vulnerable to DoS than AES. But, of course, RSA is more complex.
>>
>> The tech companies are not trying to make the best possible product, but the
>> best product that's economically feasible, which is a slightly different goal.
>>
>>> If it's likely going to be storage and/or network bandwidth limited;
>>> wouldn't the improvements made by introducing and incorporating the
>>> AES-NI instruction set be kind of "wasted" in the sense that you can't
>>> really use it to the fullest potential anyways?
>>
>> Amdahl's Law: the amount of overall improvement of performance by improving an
>> area is proportional to the amount that area is executed.
>>
>>> If the storage/network I/O is going to be your bottleneck/limiting
>>> factor, then regardless of whether you have AES-NI or not; you're
>>> likely going to get the same answer in terms of speed.
>>
>> Yes!
>>
>>> Also, is that why (besides the fact that CBC can't be parallelized)
>>> why it doesn't make sense or people really haven't spent too much time
>>> or effort into trying to run AES encryption/decryption on GPGPUs?
>>> Because it's already faster than anything else is capable of at the
>>> moment?
>>
>> Like Erwann said, the memory transfers would likely cost more time than using
>> AES-NI.
>> ______________________________________________________________________
>> OpenSSL Project                                 http://www.openssl.org
>> User Support Mailing List                    [hidden email]
>> Automated List Manager                           [hidden email]
______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
User Support Mailing List                    [hidden email]
Automated List Manager                           [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: [openssl-users] Re: having a lot of troubles trying to get AES-NI working

sarav.sars
In reply to this post by Erwann ABALEA
Is it necessary to load aesni-intel module like modprobe aesni-intel ?
Loading this module makes no difference in openssl speed output.

openssl speed -elapsed -evp aes-128-cbc

before loading aesni-intel module

type             16 bytes     64 bytes    256 bytes   1024 bytes   8192 bytes
aes-128-cbc     561737.40k   598685.65k   610372.15k   610802.35k   611521.88k

after loading aesni-intel module

type             16 bytes     64 bytes    256 bytes   1024 bytes   8192 bytes
aes-128-cbc     563835.61k   598692.84k   608343.64k   610805.42k   611524.61k