[openssl-dev] [openssl.org #3897] request: add BLAKE2 hash function (let's kill md5sum!)

classic Classic list List threaded Threaded
51 messages Options
123
Reply | Threaded
Open this post in threaded view
|

[openssl-dev] [openssl.org #3897] request: add BLAKE2 hash function (let's kill md5sum!)

Rich Salz via RT
Dear OpenSSL folks:

I'm one of the authors of the BLAKE2 hash function
(https://blake2.net). I've been working with the maintainers of GNU
coreutils to make a tool named "b2sum", which I hope will eventually
replace md5sum.

md5sum is the most widely-used tool in the world for data integrity
but, as you know, MD5 is weak in ways that could endanger the users of
md5sum, depending on how they use it. I want to see md5sum phased out
entirely in our lifetimes!

BLAKE2 is a secure hash function, while being faster than MD5 (at
least on 64-bit CPUs). BLAKE2 is being used in new software projects
(https://blake2.net/#us) and there is recently an Internet Draft to
specify it (https://datatracker.ietf.org/doc/draft-saarinen-blake2/?include_text=1).

One of the coreutils maintainers suggested that we should ask OpenSSL
to add BLAKE2, because coreutils itself will probably just use a
portable C implementation, but it would use an optimized
implementation if openssl provided it. Here's that thread:
http://lists.gnu.org/archive/html/coreutils/2015-06/msg00011.html

We, the BLAKE2 maintainers, offer both reference C code and optimized
implementations: https://blake2.net/#dl . There are also other
implementations with various virtues available: https://blake2.net/#sw

Here's my blog post extolling the virtues of BLAKE2 as a
high-performance hash function:

https://leastauthority.com/blog/BLAKE2-harder-better-faster-stronger-than-MD5.html

Regards,

Zooko

_______________________________________________
openssl-bugs-mod mailing list
[hidden email]
https://mta.openssl.org/mailman/listinfo/openssl-bugs-mod

_______________________________________________
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
Reply | Threaded
Open this post in threaded view
|

Re: [openssl-dev] [openssl.org #3897] request: add BLAKE2 hash function (let's kill md5sum!)

Rich Salz via RT
On Friday 05 June 2015 16:39:36 Zooko Wilcox-OHearn via RT wrote:

> Dear OpenSSL folks:
>
> I'm one of the authors of the BLAKE2 hash function
> (https://blake2.net). I've been working with the maintainers of GNU
> coreutils to make a tool named "b2sum", which I hope will eventually
> replace md5sum.
>
> md5sum is the most widely-used tool in the world for data integrity
> but, as you know, MD5 is weak in ways that could endanger the users of
> md5sum, depending on how they use it. I want to see md5sum phased out
> entirely in our lifetimes!
>
> BLAKE2 is a secure hash function, while being faster than MD5 (at
> least on 64-bit CPUs). BLAKE2 is being used in new software projects
> (https://blake2.net/#us) and there is recently an Internet Draft to
> specify it
> (https://datatracker.ietf.org/doc/draft-saarinen-blake2/?include_text=1).
>
> One of the coreutils maintainers suggested that we should ask OpenSSL
> to add BLAKE2, because coreutils itself will probably just use a
> portable C implementation, but it would use an optimized
> implementation if openssl provided it. Here's that thread:
> http://lists.gnu.org/archive/html/coreutils/2015-06/msg00011.html
>
> We, the BLAKE2 maintainers, offer both reference C code and optimized
> implementations: https://blake2.net/#dl . There are also other
> implementations with various virtues available: https://blake2.net/#sw
>
> Here's my blog post extolling the virtues of BLAKE2 as a
> high-performance hash function:
>
> https://leastauthority.com/blog/BLAKE2-harder-better-faster-stronger-than-MD
> 5.html
>
how resistant is it against side channel attacks?

--
Regards,
Hubert Kario
Quality Engineer, QE BaseOS Security team
Web: www.cz.redhat.com
Red Hat Czech s.r.o., Purkyňova 99/71, 612 45, Brno, Czech Republic

_______________________________________________
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev

signature.asc (1K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: [openssl-dev] [openssl.org #3897] request: add BLAKE2 hash function (let's kill md5sum!)

Yoav Nir

> On Jun 8, 2015, at 1:37 PM, Hubert Kario via RT <[hidden email]> wrote:
>
> On Friday 05 June 2015 16:39:36 Zooko Wilcox-OHearn via RT wrote:
>> Dear OpenSSL folks:
>>
>> I'm one of the authors of the BLAKE2 hash function
>> (https://blake2.net). I've been working with the maintainers of GNU
>> coreutils to make a tool named "b2sum", which I hope will eventually
>> replace md5sum.
>>
>> md5sum is the most widely-used tool in the world for data integrity
>> but, as you know, MD5 is weak in ways that could endanger the users of
>> md5sum, depending on how they use it. I want to see md5sum phased out
>> entirely in our lifetimes!
>>
>> BLAKE2 is a secure hash function, while being faster than MD5 (at
>> least on 64-bit CPUs). BLAKE2 is being used in new software projects
>> (https://blake2.net/#us) and there is recently an Internet Draft to
>> specify it
>> (https://datatracker.ietf.org/doc/draft-saarinen-blake2/?include_text=1).
>>
>> One of the coreutils maintainers suggested that we should ask OpenSSL
>> to add BLAKE2, because coreutils itself will probably just use a
>> portable C implementation, but it would use an optimized
>> implementation if openssl provided it. Here's that thread:
>> http://lists.gnu.org/archive/html/coreutils/2015-06/msg00011.html
>>
>> We, the BLAKE2 maintainers, offer both reference C code and optimized
>> implementations: https://blake2.net/#dl . There are also other
>> implementations with various virtues available: https://blake2.net/#sw
>>
>> Here's my blog post extolling the virtues of BLAKE2 as a
>> high-performance hash function:
>>
>> https://leastauthority.com/blog/BLAKE2-harder-better-faster-stronger-than-MD
>> 5.html
>>
>
> how resistant is it against side channel attacks?

Since it’s based on ChaCha, it’s very resistant to timing (and power) based side channel leakage.

Yoav

_______________________________________________
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
Reply | Threaded
Open this post in threaded view
|

Re: [openssl-dev] [openssl.org #3897] request: add BLAKE2 hash function (let's kill md5sum!)

Rich Salz via RT

> On Jun 8, 2015, at 1:37 PM, Hubert Kario via RT <[hidden email]> wrote:
>
> On Friday 05 June 2015 16:39:36 Zooko Wilcox-OHearn via RT wrote:
>> Dear OpenSSL folks:
>>
>> I'm one of the authors of the BLAKE2 hash function
>> (https://blake2.net). I've been working with the maintainers of GNU
>> coreutils to make a tool named "b2sum", which I hope will eventually
>> replace md5sum.
>>
>> md5sum is the most widely-used tool in the world for data integrity
>> but, as you know, MD5 is weak in ways that could endanger the users of
>> md5sum, depending on how they use it. I want to see md5sum phased out
>> entirely in our lifetimes!
>>
>> BLAKE2 is a secure hash function, while being faster than MD5 (at
>> least on 64-bit CPUs). BLAKE2 is being used in new software projects
>> (https://blake2.net/#us) and there is recently an Internet Draft to
>> specify it
>> (https://datatracker.ietf.org/doc/draft-saarinen-blake2/?include_text=1).
>>
>> One of the coreutils maintainers suggested that we should ask OpenSSL
>> to add BLAKE2, because coreutils itself will probably just use a
>> portable C implementation, but it would use an optimized
>> implementation if openssl provided it. Here's that thread:
>> http://lists.gnu.org/archive/html/coreutils/2015-06/msg00011.html
>>
>> We, the BLAKE2 maintainers, offer both reference C code and optimized
>> implementations: https://blake2.net/#dl . There are also other
>> implementations with various virtues available: https://blake2.net/#sw
>>
>> Here's my blog post extolling the virtues of BLAKE2 as a
>> high-performance hash function:
>>
>> https://leastauthority.com/blog/BLAKE2-harder-better-faster-stronger-than-MD
>> 5.html
>>
>
> how resistant is it against side channel attacks?

Since it’s based on ChaCha, it’s very resistant to timing (and power) based side channel leakage.

Yoav


_______________________________________________
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
Reply | Threaded
Open this post in threaded view
|

Re: [openssl-dev] [openssl.org #3897] request: add BLAKE2 hash function (let's kill md5sum!)

Bill Cox
Not that my opinion here counts, but I'll second the call for BLAKE2 support.  The SIMD implementation is one of the finest works of efficient cryptographic code I've run across.  It's so efficient, it became by far the most popular hash function in the Password Hashing Competition.  BLAKE2 rocks.  It leaves SHA256 in the dust.

Bill

_______________________________________________
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
Reply | Threaded
Open this post in threaded view
|

Re: [openssl-dev] [openssl.org #3897] request: add BLAKE2 hash function (let's kill md5sum!)

Rich Salz via RT
Not that my opinion here counts, but I'll second the call for BLAKE2
support.  The SIMD implementation is one of the finest works of efficient
cryptographic code I've run across.  It's so efficient, it became by far
the most popular hash function in the Password Hashing Competition.  BLAKE2
rocks.  It leaves SHA256 in the dust.

Bill

_______________________________________________
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
Reply | Threaded
Open this post in threaded view
|

Re: [openssl-dev] [openssl.org #3897] request: add BLAKE2 hash function (let's kill md5sum!)

Kurt Roeckx
In reply to this post by Rich Salz via RT
On Fri, Jun 05, 2015 at 04:39:36PM +0000, Zooko Wilcox-OHearn via RT wrote:
>
> One of the coreutils maintainers suggested that we should ask OpenSSL
> to add BLAKE2, because coreutils itself will probably just use a
> portable C implementation, but it would use an optimized
> implementation if openssl provided it. Here's that thread:
> http://lists.gnu.org/archive/html/coreutils/2015-06/msg00011.html

So which "blake2" are we talking about?  I understand that there
are many variations.

Anyway, I think we should add it.


Kurt

_______________________________________________
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
Reply | Threaded
Open this post in threaded view
|

Re: [openssl-dev] [openssl.org #3897] request: add BLAKE2 hash function (let's kill md5sum!)

Bill Cox
On Mon, Jun 8, 2015 at 10:11 AM, Kurt Roeckx <[hidden email]> wrote:
So which "blake2" are we talking about?  I understand that there
are many variations.

Anyway, I think we should add it.


Kurt

Blake2s is 256-bit, while Blake2d is 512-bit.  These are the ones I assume that would be best for addition.  The other two, Blake2sp and Blake2bp are multi-threaded, and are optimized for multi-core CPUs.

Bill

_______________________________________________
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
Reply | Threaded
Open this post in threaded view
|

Re: [openssl-dev] [openssl.org #3897] request: add BLAKE2 hash function (let's kill md5sum!)

Salz, Rich
In reply to this post by Kurt Roeckx

> Anyway, I think we should add it.

I am in favor of doing that, too.  But there's some work that needs to be done:  hooking it up to the EVP API, and tweaking the assembler stuff to use our perl-based structure, right?
_______________________________________________
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
Reply | Threaded
Open this post in threaded view
|

Re: [openssl-dev] [openssl.org #3897] request: add BLAKE2 hash function (let's kill md5sum!)

Rich Salz via RT
In reply to this post by Bill Cox
>Blake2s is 256-bit, while Blake2d is 512-bit.  These are the ones I assume that would be best for addition.  The other two, Blake2sp and Blake2bp are multi-threaded, and are optimized for multi-core CPUs.

It is unfortunate that 's' and 'd' mean different algorithms, while 2sp and 2bp are, presumably, alternative versions of 2s and 2d, respectively.  Nobody outside the implementation should know about that second class of difference.  And note that one of the longer OpenSSL members, who is very experienced in implementations of crypto, was confused.

And is it really 2d and 2bp?  Or is one of those [db] letters a typo?  Either way, I think it makes a case for changing the names.

It is pretty common to use the size as a suffix.  I would really like to see blake2-256 and blake2-512 as the common names.  And the implementation names, as I said, need never be seen outside of, well, the implementation. :)

_______________________________________________
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
Reply | Threaded
Open this post in threaded view
|

Re: [openssl-dev] [openssl.org #3897] request: add BLAKE2 hash function (let's kill md5sum!)

Rich Salz via RT
Dear Rich Salz et al.:

"b" is for "big" — fits well with 64-bit architectures, and "s" is for
"small" — fits well with 32-bit architectures.

"p" is for "parallel" — has several parallel threads that each compute
the hash of a different subset of the input data, and then those
hashes get hashed together to result in the final output. Therefore it
isn't just an internal implementation different — blake2sp generates
different hash values than blake2s does, and blake2b and blake2bp are
all different — each of the four would produce a  different value.

In practice the parallel mode works nicely on modern systems. Hashing
a 1 GiB file on my Intel Core i5 laptop:

md5: 2.1s
sha256: 5.7s
blake2b: 1.8s
blake2sp: 1.1s

(from https://mail.google.com/mail/u/0/#label/coreutils/14d96872a0e9d1b3 )

Regards,

Zooko Wilcox-O'Hearn

Founder, CEO, and Customer Support Rep
https://LeastAuthority.com — Freedom matters.


_______________________________________________
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
Reply | Threaded
Open this post in threaded view
|

Re: [openssl-dev] [openssl.org #3897] request: add BLAKE2 hash function (let's kill md5sum!)

Rich Salz via RT
So it's really a request to add four hash functions.  Bummer.

> In practice the parallel mode works nicely on modern systems.

Well, on clients.  On servers, presumably, those cores would be busy ;)

I'd support adding 2b and 2s, in spite of the fact that the names are really really bad.  I'm less interested in seeing the parallel variants added.  FWIW.

_______________________________________________
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
Reply | Threaded
Open this post in threaded view
|

Re: [openssl-dev] [openssl.org #3897] request: add BLAKE2 hash function (let's kill md5sum!)

Zooko Wilcox-OHearn
In reply to this post by Rich Salz via RT
(re-sent because I wasn't subscribed to openssl-dev first time and it
bounced from there but went through to rt@.)


Dear Rich Salz et al.:

"b" is for "big" — fits well with 64-bit architectures, and "s" is for
"small" — fits well with 32-bit architectures.

"p" is for "parallel" — has several parallel threads that each compute
the hash of a different subset of the input data, and then those
hashes get hashed together to result in the final output. Therefore it
isn't just an internal implementation different — blake2sp generates
different hash values than blake2s does, and blake2b and blake2bp are
all different — each of the four would produce a  different value.

In practice the parallel mode works nicely on modern systems. Hashing
a 1 GiB file on my Intel Core i5 laptop:

md5: 2.1s
sha256: 5.7s
blake2b: 1.8s
blake2sp: 1.1s

(from https://mail.google.com/mail/u/0/#label/coreutils/14d96872a0e9d1b3 )

Regards,

Zooko Wilcox-O'Hearn

Founder, CEO, and Customer Support Rep
https://LeastAuthority.com — Freedom matters.
_______________________________________________
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
Reply | Threaded
Open this post in threaded view
|

Re: [openssl-dev] [openssl.org #3897] request: add BLAKE2 hash function (let's kill md5sum!)

Rich Salz via RT
(re-sent because I wasn't subscribed to openssl-dev first time and it
bounced from there but went through to rt@.)


Dear Rich Salz et al.:

"b" is for "big" — fits well with 64-bit architectures, and "s" is for
"small" — fits well with 32-bit architectures.

"p" is for "parallel" — has several parallel threads that each compute
the hash of a different subset of the input data, and then those
hashes get hashed together to result in the final output. Therefore it
isn't just an internal implementation different — blake2sp generates
different hash values than blake2s does, and blake2b and blake2bp are
all different — each of the four would produce a  different value.

In practice the parallel mode works nicely on modern systems. Hashing
a 1 GiB file on my Intel Core i5 laptop:

md5: 2.1s
sha256: 5.7s
blake2b: 1.8s
blake2sp: 1.1s

(from https://mail.google.com/mail/u/0/#label/coreutils/14d96872a0e9d1b3 )

Regards,

Zooko Wilcox-O'Hearn

Founder, CEO, and Customer Support Rep
https://LeastAuthority.com — Freedom matters.


_______________________________________________
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
Reply | Threaded
Open this post in threaded view
|

Re: [openssl-dev] [openssl.org #3897] request: add BLAKE2 hash function (let's kill md5sum!)

Zooko Wilcox-OHearn
In reply to this post by Rich Salz via RT
> I'd support adding 2b and 2s, in spite of the fact that the names are really really bad.  I'm less interested in seeing the parallel variants added.  FWIW.

Well, the reason I'm here is that the GNU coreutils maintainers rely
on openssl for high-performance crypto, and blake2sp might be the best
algorithm for the new "b2sum" tool, which I hope will replace "md5sum"
in the toolboxes of system administrators everywhere.

Regards,

Zooko Wilcox-O'Hearn

Founder, CEO, and Customer Support Rep
https://LeastAuthority.com — Freedom matters.
_______________________________________________
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
Reply | Threaded
Open this post in threaded view
|

Re: [openssl-dev] [openssl.org #3897] request: add BLAKE2 hash function (let's kill md5sum!)

Rich Salz via RT
> I'd support adding 2b and 2s, in spite of the fact that the names are really really bad.  I'm less interested in seeing the parallel variants added.  FWIW.

Well, the reason I'm here is that the GNU coreutils maintainers rely
on openssl for high-performance crypto, and blake2sp might be the best
algorithm for the new "b2sum" tool, which I hope will replace "md5sum"
in the toolboxes of system administrators everywhere.

Regards,

Zooko Wilcox-O'Hearn

Founder, CEO, and Customer Support Rep
https://LeastAuthority.com — Freedom matters.


_______________________________________________
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
Reply | Threaded
Open this post in threaded view
|

Re: [openssl-dev] [openssl.org #3897] request: add BLAKE2 hash function (let's kill md5sum!)

Salz, Rich

> Well, the reason I'm here is that the GNU coreutils maintainers rely on
> openssl for high-performance crypto, and blake2sp might be the best
> algorithm for the new "b2sum" tool, which I hope will replace "md5sum"
> in the toolboxes of system administrators everywhere.

Yes, I went and read the thread over there.  I think this is an interesting and worthwhile discussion to have, but I took "rt" off the list as I'm not sure the ticket-tracking system is the best place for that :)

If the goal is replace md5sum, then one thing to think about is which digest will have the widest reach for everyone?  Can all four versions be implemented in (mostly?) portable C code?  Is performance the only real difference?  Suppose we took just blake2s?
_______________________________________________
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
Reply | Threaded
Open this post in threaded view
|

Re: [openssl-dev] [openssl.org #3897] request: add BLAKE2 hash function (let's kill md5sum!)

Zooko Wilcox-OHearn
> If the goal is replace md5sum, then one thing to think about is which digest will have the widest reach for everyone?  Can all four versions be implemented in (mostly?) portable C code?  Is performance the only real difference?  Suppose we took just blake2s?

All four are available in mostly-portable C code, as well as in
various optimized versions: https://blake2.net/#dl and
https://blake2.net/#sw .

The differences are:

1. The b's are more efficient on 64-bit architectures, the s's are
more efficient on 32-bit architectures. Search for "blake2b" and
"blake2s" in http://bench.cr.yp.to/results-hash.html .

   For example on an Intel x86-64 Xeon E3-1275 V3
(http://bench.cr.yp.to/results-hash.html#amd64-titan0), blake2b costs
3.09 cpb and blake2s costs 5.35 cpb.

   On the other hand on an NVIDIA ARM Tegra 250
(http://bench.cr.yp.to/results-hash.html#armeabi-h2tegra), blake2b
costs 37.43 cpb and blake2s costs 13.49 cpb.

   (I looked at the worst-case quartile for 4096-byte inputs for those
measurements.)

2. The b's can emit up to 512 bits of output, the s's can emit up to
256 bits of output.

3. The 'p' versions use more cores and finish faster.

Interestingly, on my 64-bit, 4-CPU Intel Core i5 system (a Google
Chromebook Pixel 1) blake2sp is slightly faster than blake2bp. This
might be because with hyperthreading I have effectively 8 (?)
efficient threads. blake2sp is 8-way while blake2bp is 4-way. Or maybe
it is for some other reason.

Regards,

Zooko
_______________________________________________
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
Reply | Threaded
Open this post in threaded view
|

Re: [openssl-dev] [openssl.org #3897] request: add BLAKE2 hash function (let's kill md5sum!)

Salz, Rich
So if you're going to replace md5sum... which one should you use?  Which ONE HASH should replace MD5?

Or why not just use sha256 and sha512.
_______________________________________________
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
Reply | Threaded
Open this post in threaded view
|

Re: [openssl-dev] [openssl.org #3897] request: add BLAKE2 hash function (let's kill md5sum!)

Zooko Wilcox-OHearn
On Tue, Jun 9, 2015 at 12:57 AM, Salz, Rich <[hidden email]> wrote:
> So if you're going to replace md5sum... which one should you use?  Which ONE HASH should replace MD5?

I'd suggest blake2sp. It's currently the fastest on my machine, and I
guess that there will often be multiple cores in systems where hash
performance matters (i.e. hashing large files or many files).

But, if for some reason blake2sp is problematic, then any of the other
BLAKE2 variants would also work.

> Or why not just use sha256 and sha512.

It seems like we can get people to migrate off of MD5 by offering them
better performance *and* improved security, but not by offering them
worse performance and improved security.

Regards,

Zooko Wilcox-O'Hearn

Founder, CEO, and Customer Support Rep
https://LeastAuthority.com — Freedom matters.
_______________________________________________
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
123