CAVP fips_rsastest.c not producing the correct signature?
This post has NOT been accepted by the mailing list yet.
I am trying test the rsa 186-2 openssl fips module 2.0.16 implementation using the NIST Testvectors. Using the fips_rsastest.c file the FIPS_rsa_x931_generate_key_ex(rsa, keylen, bn_e, NULL) function called to generate the modulus n as the output and taking modulus size as the input i,e [mod=1024]. Then this modulus along with the appropriate hash function and message is passed to FIPS_rsa_sign(rsa, Msg, Msglen, dgst, pad_mode, Saltlen, NULL,sigbuf, (unsigned int *)&siglen) to obtain the signature as the output.
But each time i execute the c code it generates different modulus n. Why is output changing after every execution. What am i doing wrong? I am facing the same problem with DSA parameter generation also. It will help me.