CAVP protocol testing - what does it really consist of ?

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

CAVP protocol testing - what does it really consist of ?

jonetsu

Hello,


Sorry if this is a bit beside OpenSSL per se, the idea behind this post is to perhaps have some information form the OpenSSL experience with FIPS validation.  There was so much effort put into FIPS compliance that it would not be far-fetched to consider that there is also knowledge about what seems to be /protocol/ testing.


I would like to know what's involved in the CAVP testing of the SSH protocol.  I browsed the NIST CAVP web site, browsed some documents, although I haven't found any satisfying, technically-oriented, document on what has to be done if say, I have an editor opened with the SSH source code.  Not the fully gruesome details, but an overview of how such testing works.


Any comment appreciated !




_______________________________________________
openssl-users mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-users
Reply | Threaded
Open this post in threaded view
|

Re: CAVP protocol testing - what does it really consist of ?

Steve Marquess-4
On 10/21/2015 12:02 PM, jonetsu wrote:

>
> Hello,
>
>
> Sorry if this is a bit beside OpenSSL per se, the idea behind this
> post is to perhaps have some information form the OpenSSL experience
> with FIPS validation.  There was so much effort put into FIPS
> compliance that it would not be far-fetched to consider that there is
> also knowledge about what seems to be /protocol/ testing.
>
>
> I would like to know what's involved in the CAVP testing of the SSH
> protocol.  I browsed the NIST CAVP web site, browsed some documents,
> although I haven't found any satisfying, technically-oriented,
> document on what has to be done if say, I have an editor opened with
> the SSH source code.  Not the fully gruesome details, but an overview
> of how such testing works.

See Appendix B of the OpenSSL FIPS User Guide:

  https://openssl.org/docs/fips/UserGuide-2.0.pdf

The specific algorithm tests have changed quite a bit since then
(constant change is part of the fun), but the general concept is the
same. The algorithm testing is the easiest part of FIPS 140-2 validations.

Note the CAVP only tests specific cryptographic algorithms, not
cryptographic protocol suites like SSH (secsh). OpenSSH itself is just
application code from the perspective of FIPS 140-2 and thus out of
scope (as is OpenSSL; the OpenSSL FIPS Object Module is a separate
software component carefully crafted to satisfy the peculiar
restrictions and requirements of FIPS 140-2).

Also note that converting stock OpenSSH to exclusive use of FIPS
validated cryptographic is a non-trivial exercise.

-Steve M.

--
Steve Marquess
OpenSSL Software Foundation, Inc.
1829 Mount Ephraim Road
Adamstown, MD  21710
USA
+1 877 673 6775 s/b
+1 301 874 2571 direct
[hidden email]
[hidden email]
gpg/pgp key: http://openssl.com/docs/0x6D1892F5.asc
_______________________________________________
openssl-users mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-users
Reply | Threaded
Open this post in threaded view
|

Re: CAVP protocol testing - what does it really consist of ?

jonetsu
> From: "Steve Marquess" <[hidden email]>
> Date: 10/21/15 14:18
> See Appendix B of the OpenSSL FIPS User Guide:

>   https://openssl.org/docs/fips/UserGuide-2.0.pdf

Thanks.

> The specific algorithm tests have changed quite a bit since then
> (constant change is part of the fun), but the general concept is the
> same. The algorithm testing is the easiest part of FIPS 140-2 validations.

What would you consider being the difficult parts ?

> Note the CAVP only tests specific cryptographic algorithms, not
> cryptographic protocol suites like SSH (secsh). OpenSSH itself is just
> application code from the perspective of FIPS 140-2 and thus out of
> scope ...


It has to do with NDcPP 1.0 I think.  Key agreement schemes and key derivation functions 
for several security-related communications protocols (SNMP, TLS, SSH, etc.) 
must now be tested as part of the algorithm test process.  





_______________________________________________
openssl-users mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-users
Reply | Threaded
Open this post in threaded view
|

Re: CAVP protocol testing - what does it really consist of ?

David Johnston
I my experience of getting CAVP certification on my own (non openssl)
designs it is like this:

1) Pay few ten thousands of dollars to a certification company.
2) They send you a questionnaire about the algorithms you have in the
implementation.
3) You fill that in and return it to them with design documentation
4) Questions and answers go back and forth.
5) They send you test vectors to try out
6) At some point either you visit them or they visit you and they exercise
the system against some vectors to make sure you aren't cheating.
7) They submit paperwork to NIST
8) You await your listing on the NIST website.

The difficult bit is the vector generation software. Specs like SP800-90
have many degrees of freedom. The vector generating software doesn't have
nearly the degrees of freedom that the spec has, so there is reasonable
certainty that you can't reproduce the vectors they provide because your
implementation doesn't do things in the same order as the vector generator
software supplied by NIST.

This is when you get into deep discussions with the test house and send of
interpretations to NIST to try and get it resolved.

Bad specs, bad vectors = bad certification experience.



>> From: "Steve Marquess" <[hidden email]>
>> Date: 10/21/15 14:18
>> See Appendix B of the OpenSSL FIPS User Guide:
>
>>  Â https://openssl.org/docs/fips/UserGuide-2.0.pdf
>
> Thanks.
>
>> The specific algorithm tests have changed quite a bit since then
>> (constant change is part of the fun), but the general concept is the
>> same. The algorithm testing is the easiest part of FIPS 140-2
>> validations.
>
> What would you consider being the difficult parts ?
>
>> Note the CAVP only tests specific cryptographic algorithms, not
>> cryptographic protocol suites like SSH (secsh). OpenSSH itself is just
>> application code from the perspective of FIPS 140-2 and thus out of
>> scope ...
>
>
> It has to do with NDcPP 1.0 I think.  Key agreement schemes and key
> derivation functions 
> for several security-related communications protocols (SNMP, TLS, SSH,
> etc.) 
> must now be tested as part of the algorithm test process.  
>
>
>
>
>
> _______________________________________________
> openssl-users mailing list
> To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-users
>

_______________________________________________
openssl-users mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-users
Reply | Threaded
Open this post in threaded view
|

Re: CAVP protocol testing - what does it really consist of ?

Steve Marquess-4
In reply to this post by jonetsu
On 10/21/2015 03:22 PM, jonetsu wrote:

>> From: "Steve Marquess" <[hidden email]>
>> Date: 10/21/15 14:18
>> See Appendix B of the OpenSSL FIPS User Guide:
>
>>   https://openssl.org/docs/fips/UserGuide-2.0.pdf
>
> Thanks.
>
>> The specific algorithm tests have changed quite a bit since then
>> (constant change is part of the fun), but the general concept is the
>> same. The algorithm testing is the easiest part of FIPS 140-2 validations.
>
> What would you consider being the difficult parts ?

The CMVP part. The CAVP requirements are fairly well articulated and
consistently applied. New versions of the CAVS tool, and newly
introduced test vectors, occasionally have problems but those can
usually be worked out without too much grief. The wait time for CAVP
approvals is also a lot more predictable.

The CMVP, on the other hand, is not nearly as predictable or consistent.

-Steve M.

--
Steve Marquess
OpenSSL Software Foundation, Inc.
1829 Mount Ephraim Road
Adamstown, MD  21710
USA
+1 877 673 6775 s/b
+1 301 874 2571 direct
[hidden email]
[hidden email]
gpg/pgp key: http://openssl.com/docs/0x6D1892F5.asc
_______________________________________________
openssl-users mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-users