i2d_X509_REQ() -> d2i_X509_REQ() = asn1 encoding routines:c2i_ASN1_OBJECT:invalid object encoding:a_object.c:287

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
25 messages Options
12
Reply | Threaded
Open this post in threaded view
|

i2d_X509_REQ() -> d2i_X509_REQ() = asn1 encoding routines:c2i_ASN1_OBJECT:invalid object encoding:a_object.c:287

Graham Leggett
Hi all,

While porting some code across from RHEL6 (openssl-1.0.1e-42) to RHEL7 (openssl-1.0.2k-16), I am getting the failure below where previously the code worked.

The code creates an X509_REQ, populates it, and then passes this to a module. Before passing it to the module, the structure is canonicalised by passing it through i2d_X509_REQ() and then d2i_X509_REQ() on the other side.

On RHEL7, d2i_X509_REQ fails as follows:

[root@localhost ~]# openssl req -in req.bin -inform der
unable to load X509 request
139903756527504:error:0D0C40D8:asn1 encoding routines:c2i_ASN1_OBJECT:invalid object encoding:a_object.c:287:
139903756527504:error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=algorithm, Type=X509_ALGOR
139903756527504:error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=sig_alg, Type=X509_REQ

The failure occurs in the openssl code here:

424        for (i = 0, tt = it->templates; i < it->tcount; i++, tt++) {
(gdb)
427            seqtt = asn1_do_adb(pval, tt, 1);
(gdb)
428            if (seqtt == NULL)
(gdb)
427            seqtt = asn1_do_adb(pval, tt, 1);
(gdb)
428            if (seqtt == NULL)
(gdb)
430            pseqval = asn1_get_field_ptr(pval, seqtt);
(gdb)
432            if (!len)
(gdb)
430            pseqval = asn1_get_field_ptr(pval, seqtt);
(gdb)
432            if (!len)
(gdb)
435            if (asn1_check_eoc(&p, len)) {
(gdb)
434            q = p;
(gdb)
435            if (asn1_check_eoc(&p, len)) {
(gdb)
451            if (i == (it->tcount - 1))
(gdb)
459            ret = asn1_template_ex_d2i(pseqval, &p, len, seqtt, isopt, ctx,
(gdb)
461            if (!ret) {
[————error here————]
(gdb)
520    if (combine == 0)
(gdb)
521        ASN1_item_ex_free(pval, it);
(gdb)
522    if (errtt)
(gdb)
523        ERR_add_error_data(4, "Field=", errtt->field_name,
(gdb)
527    return 0;
(gdb)
528 }

The CSR can be found here: http://www.sharp.fm/req.bin

Would it be possible to confirm what is wrong with this request?

Regards,
Graham


Reply | Threaded
Open this post in threaded view
|

Re: i2d_X509_REQ() -> d2i_X509_REQ() = asn1 encoding routines:c2i_ASN1_OBJECT:invalid object encoding:a_object.c:287

Viktor Dukhovni
On Mon, Mar 18, 2019 at 01:06:19AM +0200, Graham Leggett wrote:

> [root@localhost ~]# openssl req -in req.bin -inform der
> unable to load X509 request
> 139903756527504:error:0D0C40D8:asn1 encoding routines:c2i_ASN1_OBJECT:invalid object encoding:a_object.c:287:
> 139903756527504:error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=algorithm, Type=X509_ALGOR
> 139903756527504:error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=sig_alg, Type=X509_REQ

The CSR is malformed.

> The CSR can be found here: http://www.sharp.fm/req.bin
>
> Would it be possible to confirm what is wrong with this request?

Running "openssl asn1parse -inform DER" we get:

        0:d=0  hl=4 l= 509 cons: SEQUENCE
        4:d=1  hl=4 l= 498 cons: SEQUENCE
        8:d=2  hl=2 l=   1 prim: INTEGER           :00
       11:d=2  hl=2 l=  67 cons: SEQUENCE
       13:d=3  hl=2 l=  14 cons: SET
       15:d=4  hl=2 l=  12 cons: SEQUENCE
       17:d=5  hl=2 l=   3 prim: OBJECT            :commonName
       22:d=5  hl=2 l=   5 prim: UTF8STRING        :Test1
       29:d=3  hl=2 l=  49 cons: SET
       31:d=4  hl=2 l=  47 cons: SEQUENCE
       33:d=5  hl=2 l=   3 prim: OBJECT            :serialNumber
       38:d=5  hl=2 l=  40 prim: PRINTABLESTRING   :354616bb0358f9474f1e84af5550567f8b6c4d5b
       80:d=2  hl=4 l= 290 cons: SEQUENCE
       84:d=3  hl=2 l=  13 cons: SEQUENCE
       86:d=4  hl=2 l=   9 prim: OBJECT            :rsaEncryption
       97:d=4  hl=2 l=   0 prim: NULL
       99:d=3  hl=4 l= 271 prim: BIT STRING
      374:d=2  hl=3 l= 129 cons: cont [ 0 ]
      377:d=3  hl=2 l=  42 cons: SEQUENCE
      379:d=4  hl=2 l=   9 prim: OBJECT            :challengePassword
      390:d=4  hl=2 l=  29 cons: SET
      392:d=5  hl=2 l=  27 prim: IA5STRING         :dfwrrdq4uhec96yj23io8soav0
      421:d=3  hl=2 l=  83 cons: SEQUENCE
      423:d=4  hl=2 l=   9 prim: OBJECT            :Extension Request
      434:d=4  hl=2 l=  70 cons: SET
      436:d=5  hl=2 l=  68 cons: SEQUENCE
      438:d=6  hl=2 l=  66 cons: SEQUENCE
      440:d=7  hl=2 l=   3 prim: OBJECT            :X509v3 Subject Alternative Name
      445:d=7  hl=2 l=  59 prim: OCTET STRING      [HEX DUMP]:303981373335343631366262303335386639343734663165383461663535353035363766386236633464356240756B2E736173686B65792E6F7267
      506:d=1  hl=2 l=   2 cons: SEQUENCE
      508:d=2  hl=2 l=   0 prim: OBJECT            :BAD OBJECT:[]
      510:d=1  hl=2 l=   1 prim: BIT STRING

Whereas with:

    $ openssl req -config <(
            printf "distinguished_name = dn\n[dn]\nprompt=yes\n[v3req]\n%s\n" \
                   "subjectAltName = DNS:example.com"
        ) -reqexts v3req -new -newkey rsa:1024 -keyout /dev/null \
          -nodes -subj / 2>/dev/null | openssl asn1parse

we get:

        0:d=0  hl=4 l= 360 cons: SEQUENCE
        4:d=1  hl=3 l= 210 cons: SEQUENCE
        7:d=2  hl=2 l=   1 prim: INTEGER           :00
       10:d=2  hl=2 l=   0 cons: SEQUENCE
       12:d=2  hl=3 l= 159 cons: SEQUENCE
       15:d=3  hl=2 l=  13 cons: SEQUENCE
       17:d=4  hl=2 l=   9 prim: OBJECT            :rsaEncryption
       28:d=4  hl=2 l=   0 prim: NULL
       30:d=3  hl=3 l= 141 prim: BIT STRING
      174:d=2  hl=2 l=  41 cons: cont [ 0 ]
      176:d=3  hl=2 l=  39 cons: SEQUENCE
      178:d=4  hl=2 l=   9 prim: OBJECT            :Extension Request
      189:d=4  hl=2 l=  26 cons: SET
      191:d=5  hl=2 l=  24 cons: SEQUENCE
      193:d=6  hl=2 l=  22 cons: SEQUENCE
      195:d=7  hl=2 l=   3 prim: OBJECT            :X509v3 Subject Alternative Name
      200:d=7  hl=2 l=  15 prim: OCTET STRING      [HEX DUMP]:300D820B6578616D706C652E636F6D
      217:d=1  hl=2 l=  13 cons: SEQUENCE
      219:d=2  hl=2 l=   9 prim: OBJECT            :sha256WithRSAEncryption
      230:d=2  hl=2 l=   0 prim: NULL
      232:d=1  hl=3 l= 129 prim: BIT STRING

which has a non-zero length signature algorithm OID (l = 9).  Your
example has "l=0" where one would expect the signature OID after
the extensions.

--
        Viktor.
Reply | Threaded
Open this post in threaded view
|

Re: i2d_X509_REQ() -> d2i_X509_REQ() = asn1 encoding routines:c2i_ASN1_OBJECT:invalid object encoding:a_object.c:287

Graham Leggett
On 18 Mar 2019, at 04:55, Viktor Dukhovni <[hidden email]> wrote:

> On Mon, Mar 18, 2019 at 01:06:19AM +0200, Graham Leggett wrote:
>
>> [root@localhost ~]# openssl req -in req.bin -inform der
>> unable to load X509 request
>> 139903756527504:error:0D0C40D8:asn1 encoding routines:c2i_ASN1_OBJECT:invalid object encoding:a_object.c:287:
>> 139903756527504:error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=algorithm, Type=X509_ALGOR
>> 139903756527504:error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=sig_alg, Type=X509_REQ
>
> The CSR is malformed.

The CSR is incomplete, but isn’t malformed.

The CSR is the in the process of being built. Part of that process involves sending the partially complete CSR to another module, which then completes the CSR structure. This used to work, but has regressed when moving from rhel6 to rhel7.

> which has a non-zero length signature algorithm OID (l = 9).  Your
> example has "l=0" where one would expect the signature OID after
> the extensions.

How do I fix openssl to parse this as it did before?

Regards,
Graham


Reply | Threaded
Open this post in threaded view
|

Re: i2d_X509_REQ() -> d2i_X509_REQ() = asn1 encoding routines:c2i_ASN1_OBJECT:invalid object encoding:a_object.c:287

Graham Leggett
On 18 Mar 2019, at 12:51, Graham Leggett <[hidden email]> wrote:

>> which has a non-zero length signature algorithm OID (l = 9).  Your
>> example has "l=0" where one would expect the signature OID after
>> the extensions.
>
> How do I fix openssl to parse this as it did before?

I've raised this at https://github.com/openssl/openssl/issues/8514 to give more context.

Regards,
Graham


Reply | Threaded
Open this post in threaded view
|

Re: i2d_X509_REQ() -> d2i_X509_REQ() = asn1 encoding routines:c2i_ASN1_OBJECT:invalid object encoding:a_object.c:287

Viktor Dukhovni
In reply to this post by Graham Leggett


> On Mar 18, 2019, at 6:51 AM, Graham Leggett <[hidden email]> wrote:
>
> The CSR is incomplete, but isn’t malformed.

A mandatory part of the CSR is missing.  It is malformed.

> The CSR is the in the process of being built. Part of that process involves sending the partially complete CSR to another module, which then completes the CSR structure. This used to work, but has regressed when moving from rhel6 to rhel7.
>
>> which has a non-zero length signature algorithm OID (l = 9).  Your
>> example has "l=0" where one would expect the signature OID after
>> the extensions.
>
> How do I fix openssl to parse this as it did before?

Since you don't have a CSR, the fix is to not attempt to encode the data
as a CSR.  It sounds like what you have is a CRI (that is a
CertificationRequestInfo structure) so you'll need to encode that instead.

  https://tools.ietf.org/html/rfc2986#section-4

The relevant codec functions are:  i2d_X509_REQ_INFO and d2i_X509_REQ_INFO.

A CSR is:

   CertificationRequest ::= SEQUENCE {
        certificationRequestInfo CertificationRequestInfo,
        signatureAlgorithm AlgorithmIdentifier{{ SignatureAlgorithms }},
        signature          BIT STRING
   }

which encapsulates the CRI a larger signed structure, adding precisely
the bits you're missing.

An alternative (if you must) is to create an actual CSR, with a dummy
signature OID, and signature and then ignore the signature on the other
side.

--
        Viktor.

Reply | Threaded
Open this post in threaded view
|

Re: i2d_X509_REQ() -> d2i_X509_REQ() = asn1 encoding routines:c2i_ASN1_OBJECT:invalid object encoding:a_object.c:287

Graham Leggett
On 18 Mar 2019, at 18:49, Viktor Dukhovni <[hidden email]> wrote:

> A mandatory part of the CSR is missing.  It is malformed.

Some further digging shows the CSR is indeed malformed - the old openssl code created a CSR like this:

  507:d=2  hl=2 l=   1 prim: OBJECT            :itu-t

while the new openssl code produces a CSR like this:

  508:d=2  hl=2 l=   0 prim: OBJECT            :BAD OBJECT

It looks like X509_REQ_new() in older versions of openssl created an object with an empty signature, while the new code produces a bad signature.

> Since you don't have a CSR, the fix is to not attempt to encode the data
> as a CSR.  It sounds like what you have is a CRI (that is a
> CertificationRequestInfo structure) so you'll need to encode that instead.
>
>  https://tools.ietf.org/html/rfc2986#section-4
>
> The relevant codec functions are:  i2d_X509_REQ_INFO and d2i_X509_REQ_INFO.
>
> A CSR is:
>
>   CertificationRequest ::= SEQUENCE {
>        certificationRequestInfo CertificationRequestInfo,
>        signatureAlgorithm AlgorithmIdentifier{{ SignatureAlgorithms }},
>        signature          BIT STRING
>   }
>
> which encapsulates the CRI a larger signed structure, adding precisely
> the bits you're missing.

This makes sense - however there don’t appear to be any APIs in openssl that allow you to manipulate a X509_REQ_INFO structure. I can create it, and encode/decode it, but there is no X509_REQ_INFO_get_subject_name() (or friends) to populate the structure. X509_REQ_INFO itself is opaque.

> An alternative (if you must) is to create an actual CSR, with a dummy
> signature OID, and signature and then ignore the signature on the other
> side.

This looks like a workaround for now, what API call would I use to do that?

The X509_REQ structure is opaque, so I can’t see what options I have for setting any OIDs.

Regards,
Graham


Reply | Threaded
Open this post in threaded view
|

Re: i2d_X509_REQ() -> d2i_X509_REQ() = asn1 encoding routines:c2i_ASN1_OBJECT:invalid object encoding:a_object.c:287

Dave Coombs
> This makes sense - however there don’t appear to be any APIs in openssl that allow you to manipulate a X509_REQ_INFO structure. I can create it, and encode/decode it, but there is no X509_REQ_INFO_get_subject_name() (or friends) to populate the structure. X509_REQ_INFO itself is opaque.

I believe you said you're using 1.0.2, right?  The structures aren't opaque there.  You can make your X509_REQ and populate its name etc as you already are, and then i2d_X509_REQ_INFO its req_info member.

(Even in the 1.1 API, where they are opaque, i2d_re_X509_REQ_tbs will encode a given X509_REQ's X509_REQ_INFO for you.)

Good luck,
  -Dave


smime.p7s (12K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: i2d_X509_REQ() -> d2i_X509_REQ() = asn1 encoding routines:c2i_ASN1_OBJECT:invalid object encoding:a_object.c:287

Viktor Dukhovni
> On Mar 18, 2019, at 4:02 PM, Dave Coombs <[hidden email]> wrote:
>
> (Even in the 1.1 API, where they are opaque, i2d_re_X509_REQ_tbs will encode a given X509_REQ's X509_REQ_INFO for you.)

Yes, i2d_re_X509_REQ_tbs is the key function for constructing the
"to be signed" (tbs) request:

  int i2d_re_X509_REQ_tbs(X509_REQ *req, unsigned char **pp)
  {
      req->req_info.enc.modified = 1;
      return i2d_X509_REQ_INFO(&req->req_info, pp);
  }

By setting the "modified" bit, it ensures that the DER representation
will be re-generated with any changes made to the object.

So the OP can create the "partially filled in" X509_REQ and then call
i2d_re_X509_REQ_tbs() function to generate the DER CRI blob to sign.
This removes any temptation to "cheat" by just casting the (X509_REQ *)
as an (X509_REQ_INFO *) and calling i2d_X509_REQ_INFO() on that (first
member of the X509_REQ structure).  The i2d_re_X509_REQ_tbs() function
achieves the same effect in a type safe supported manner.

--
        Viktor.

Reply | Threaded
Open this post in threaded view
|

Re: i2d_X509_REQ() -> d2i_X509_REQ() = asn1 encoding routines:c2i_ASN1_OBJECT:invalid object encoding:a_object.c:287

Graham Leggett
On 18 Mar 2019, at 22:27, Viktor Dukhovni <[hidden email]> wrote:

(Even in the 1.1 API, where they are opaque, i2d_re_X509_REQ_tbs will encode a given X509_REQ's X509_REQ_INFO for you.)

Yes, i2d_re_X509_REQ_tbs is the key function for constructing the
"to be signed" (tbs) request:

 int i2d_re_X509_REQ_tbs(X509_REQ *req, unsigned char **pp)
 {
     req->req_info.enc.modified = 1;
     return i2d_X509_REQ_INFO(&req->req_info, pp);
 }

By setting the "modified" bit, it ensures that the DER representation
will be re-generated with any changes made to the object.

So the OP can create the "partially filled in" X509_REQ and then call
i2d_re_X509_REQ_tbs() function to generate the DER CRI blob to sign.
This removes any temptation to "cheat" by just casting the (X509_REQ *)
as an (X509_REQ_INFO *) and calling i2d_X509_REQ_INFO() on that (first
member of the X509_REQ structure).  The i2d_re_X509_REQ_tbs() function
achieves the same effect in a type safe supported manner.

Can you confirm what structure is being encoded by i2d_re_X509_REQ_tbs, is this a X509_REQ_INFO?

The man page doesn’t explicitly specify the output formats of any of the related functions:


Looking at the source, we have X509_REQ_INFO being returned:

int i2d_re_X509_REQ_tbs(X509_REQ *req, unsigned char **pp)
{
    req->req_info.enc.modified = 1;
    return i2d_X509_REQ_INFO(&req->req_info, pp);
}

What would the corresponding functions need to be in the openssl v1.0.x world to achieve the same output as i2d_re_X509_REQ_tbs?

Would it just be to copy the above code in?

Regards,
Graham

Reply | Threaded
Open this post in threaded view
|

Re: i2d_X509_REQ() -> d2i_X509_REQ() = asn1 encoding routines:c2i_ASN1_OBJECT:invalid object encoding:a_object.c:287

Graham Leggett
In reply to this post by Dave Coombs
On 18 Mar 2019, at 22:02, Dave Coombs <[hidden email]> wrote:

>> This makes sense - however there don’t appear to be any APIs in openssl that allow you to manipulate a X509_REQ_INFO structure. I can create it, and encode/decode it, but there is no X509_REQ_INFO_get_subject_name() (or friends) to populate the structure. X509_REQ_INFO itself is opaque.
>
> I believe you said you're using 1.0.2, right?  The structures aren't opaque there.  You can make your X509_REQ and populate its name etc as you already are, and then i2d_X509_REQ_INFO its req_info member.
>
> (Even in the 1.1 API, where they are opaque, i2d_re_X509_REQ_tbs will encode a given X509_REQ's X509_REQ_INFO for you.)

How would I decode the X509_REQ_INFO structure on the other side, turning it back into X509_REQ?

While I can see a d2i_X509_REQ_INFO() function, I can’t find a corresponding function in openssl 1.1.0+ that assigns this to a X509_REQ, unless I am missing it?

By way of concrete example, having crossed the module boundary we need to pull out details from the X509_REQ_INFO, which can only be done if this structure has been assigned to a X509_REQ first:

https://source.redwax.eu/projects/RS/repos/mod_ca/browse/mod_ca_ldap.c#368

Regards,
Graham



smime.p7s (4K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: i2d_X509_REQ() -> d2i_X509_REQ() = asn1 encoding routines:c2i_ASN1_OBJECT:invalid object encoding:a_object.c:287

Viktor Dukhovni
> On Mar 18, 2019, at 8:22 PM, Graham Leggett <[hidden email]> wrote:
>
> How would I decode the X509_REQ_INFO structure on the other side, turning it back into X509_REQ?

The function returns the DER form of the CRI, which can then be signed.
You can use d2i_X509_REQ_INFO() to get an X509_REQ_INFO, but indeed there's
not much you can do with that, other than DER-encode it again and sign.

Why do you need to do the encode and decode?  What's wrong with the original
request object?

> While I can see a d2i_X509_REQ_INFO() function, I can’t find a corresponding function in openssl 1.1.0+ that assigns this to a X509_REQ, unless I am missing it?

It should not be needed.

> By way of concrete example, having crossed the module boundary we need to pull out details from the X509_REQ_INFO, which can only be done if this structure has been assigned to a X509_REQ first:

Can you be more specific about these "module boundaries"?

In any case, given the DER form of the CRI, it is easy to construct
the DER form of an enclosing CSR with a dummy signature:

        0:d=0  hl=4 l= 360 cons: SEQUENCE -- Outer sequence and length:  30 82 01 68

        4:d=1  hl=3 l= 210 cons: SEQUENCE -- DER encoding of CRI
        7:d=2  hl=2 l=   1 prim: INTEGER           :00
       10:d=2  hl=2 l=   0 cons: SEQUENCE
       12:d=2  hl=3 l= 159 cons: SEQUENCE
       15:d=3  hl=2 l=  13 cons: SEQUENCE
       17:d=4  hl=2 l=   9 prim: OBJECT            :rsaEncryption
       28:d=4  hl=2 l=   0 prim: NULL
       30:d=3  hl=3 l= 141 prim: BIT STRING
      174:d=2  hl=2 l=  41 cons: cont [ 0 ]
      176:d=3  hl=2 l=  39 cons: SEQUENCE
      178:d=4  hl=2 l=   9 prim: OBJECT            :Extension Request
      189:d=4  hl=2 l=  26 cons: SET
      191:d=5  hl=2 l=  24 cons: SEQUENCE
      193:d=6  hl=2 l=  22 cons: SEQUENCE
      195:d=7  hl=2 l=   3 prim: OBJECT            :X509v3 Subject Alternative Name
      200:d=7  hl=2 l=  15 prim: OCTET STRING      [HEX DUMP]:300D820B6578616D706C652E636F6D

      217:d=1  hl=2 l=  13 cons: SEQUENCE          -- Signature algorithm OID and parameters
      219:d=2  hl=2 l=   9 prim: OBJECT            :sha256WithRSAEncryption
      230:d=2  hl=2 l=   0 prim: NULL
      232:d=1  hl=3 l= 129 prim: BIT STRING        -- Signature data

In the above we see that the CRI, needs (typically) an ~4-byte prefix
of (0x30 + DER encoded length) and a suffix of the form:

        30 0d                             -- 13 byte sequence
        06 09 2a 86 48 86 f7 0d 01 01 0b  -- 9 byte OID (sha256WithRSAEncryption)
        05 00                             -- NULL parameters
        03 81 81 00                       -- 128 byte bit string with 0 unused bits
        xx xx xx xx ... xx                -- 128 bytes of random data.

your random data could be all zeros.  The trailer length is then a
fixed 147 bytes.  Add that to the length of CRI and prepend the
outer sequence (0x30 + DER encoded (length CRI + 147)), then the
CRI and then the trailer, and presto-magic you have a CSR with
a bogus signature, but one that will encode and decode, just not
pass "req -verify".

This isn't pretty, and perhaps we need some new functions to explicitly
embed a CRI in a CSR, but it is certainly something you can do in the
short term.

--
        Viktor.

Reply | Threaded
Open this post in threaded view
|

Re: i2d_X509_REQ() -> d2i_X509_REQ() = asn1 encoding routines:c2i_ASN1_OBJECT:invalid object encoding:a_object.c:287

Graham Leggett
On 19 Mar 2019, at 03:22, Viktor Dukhovni <[hidden email]> wrote:

>> On Mar 18, 2019, at 8:22 PM, Graham Leggett <[hidden email]> wrote:
>>
>> How would I decode the X509_REQ_INFO structure on the other side, turning it back into X509_REQ?
>
> The function returns the DER form of the CRI, which can then be signed.
> You can use d2i_X509_REQ_INFO() to get an X509_REQ_INFO, but indeed there's
> not much you can do with that, other than DER-encode it again and sign.
>
> Why do you need to do the encode and decode?  What's wrong with the original
> request object?
The code is a modular ca, and different modules communicate with each other generically using the standard DER encoded structures. Openssl isn’t visible through the API, it is an implementation detail.

>> While I can see a d2i_X509_REQ_INFO() function, I can’t find a corresponding function in openssl 1.1.0+ that assigns this to a X509_REQ, unless I am missing it?
>
> It should not be needed.

I don’t follow - in order to get access to the data inside the X509_REQ_INFO structure, I need to first wrap it in a X509_REQ, otherwise I have no API calls to get access to the data inside it.

>> By way of concrete example, having crossed the module boundary we need to pull out details from the X509_REQ_INFO, which can only be done if this structure has been assigned to a X509_REQ first:
>
> Can you be more specific about these "module boundaries”?

The modules are Apache httpd modules, and the boundaries between the modules are hooks that pass DER encoded structures between each module.

> In any case, given the DER form of the CRI, it is easy to construct
> the DER form of an enclosing CSR with a dummy signature:
>
> 0:d=0  hl=4 l= 360 cons: SEQUENCE -- Outer sequence and length:  30 82 01 68
>
> 4:d=1  hl=3 l= 210 cons: SEQUENCE -- DER encoding of CRI
> 7:d=2  hl=2 l=   1 prim: INTEGER           :00
>       10:d=2  hl=2 l=   0 cons: SEQUENCE
>       12:d=2  hl=3 l= 159 cons: SEQUENCE
>       15:d=3  hl=2 l=  13 cons: SEQUENCE
>       17:d=4  hl=2 l=   9 prim: OBJECT            :rsaEncryption
>       28:d=4  hl=2 l=   0 prim: NULL
>       30:d=3  hl=3 l= 141 prim: BIT STRING
>      174:d=2  hl=2 l=  41 cons: cont [ 0 ]
>      176:d=3  hl=2 l=  39 cons: SEQUENCE
>      178:d=4  hl=2 l=   9 prim: OBJECT            :Extension Request
>      189:d=4  hl=2 l=  26 cons: SET
>      191:d=5  hl=2 l=  24 cons: SEQUENCE
>      193:d=6  hl=2 l=  22 cons: SEQUENCE
>      195:d=7  hl=2 l=   3 prim: OBJECT            :X509v3 Subject Alternative Name
>      200:d=7  hl=2 l=  15 prim: OCTET STRING      [HEX DUMP]:300D820B6578616D706C652E636F6D
>
>      217:d=1  hl=2 l=  13 cons: SEQUENCE          -- Signature algorithm OID and parameters
>      219:d=2  hl=2 l=   9 prim: OBJECT            :sha256WithRSAEncryption
>      230:d=2  hl=2 l=   0 prim: NULL
>      232:d=1  hl=3 l= 129 prim: BIT STRING        -- Signature data
>
> In the above we see that the CRI, needs (typically) an ~4-byte prefix
> of (0x30 + DER encoded length) and a suffix of the form:
>
> 30 0d                             -- 13 byte sequence
>        06 09 2a 86 48 86 f7 0d 01 01 0b  -- 9 byte OID (sha256WithRSAEncryption)
>        05 00                             -- NULL parameters
>        03 81 81 00                       -- 128 byte bit string with 0 unused bits
>        xx xx xx xx ... xx                -- 128 bytes of random data.
>
> your random data could be all zeros.  The trailer length is then a
> fixed 147 bytes.  Add that to the length of CRI and prepend the
> outer sequence (0x30 + DER encoded (length CRI + 147)), then the
> CRI and then the trailer, and presto-magic you have a CSR with
> a bogus signature, but one that will encode and decode, just not
> pass "req -verify".
>
> This isn't pretty, and perhaps we need some new functions to explicitly
> embed a CRI in a CSR, but it is certainly something you can do in the
> short term.
Can we not rather fix the initialisation of the X509_REQ in X509_REQ_new() so that it works like it used to? It seems like a massive headache to do something that used to be trivial.

I see there have been changes to openssl code relating to how structures are initialised, I suspect an error has crept in where an ASN.1 object is missing instead of empty, thus the malformed CSR.

Regards,
Graham



smime.p7s (4K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: i2d_X509_REQ() -> d2i_X509_REQ() = asn1 encoding routines:c2i_ASN1_OBJECT:invalid object encoding:a_object.c:287

Viktor Dukhovni
On Tue, Mar 19, 2019 at 02:04:14PM +0200, Graham Leggett wrote:

> > Why do you need to do the encode and decode?  What's wrong with the original
> > request object?
>
> The code is a modular ca, and different modules communicate with each other
> generically using the standard DER encoded structures.

Well, the *standard* structure for passing around just the unsigned
data underlying a CSR (X509_REQ), is a CertificationRequestInfo
(X509_REQ_INFO).  So if the modules are to use *standard* structures
to communicate.  The object being passed needs to be either a CSR
(signed) or the enclosed CRI (unsigned).

You could, for example, sign the request with some suitable key
(ideally the private key corresponding to the public key in the
CSR, if available) before handing it off.  If the signing key is
not the enclosed public key, it would not pass "req -verify" (it
never did before either, for lack of a signature), but the called
module would be able to decode a CSR, and work as before.

> > > While I can see a d2i_X509_REQ_INFO() function, I can’t find a
> > > corresponding function in openssl 1.1.0+ that assigns this to a
> > > X509_REQ, unless I am missing it?
> >
> > It should not be needed.
>
> I don’t follow - in order to get access to the data inside the X509_REQ_INFO
> structure, I need to first wrap it in a X509_REQ, otherwise I have no API
> calls to get access to the data inside it.

No need to get access to the data inside an X509_REQ_INFO is expected.
That object's sole purpose is to be serializable for signing.

You have rather an edge-case, where for some reason you're delegating
signing to a CA module by passing it a non-standard structure that
*resembles* a CSR, which is however for some reason not signed with
the subject key (not signed at all), and you expect the CA to apply
policy, by decoding the CRI inside  this non-CSR.

OpenSSL 1.1.x does not have structure member accessors for a CRI,
but they would be easy to add, that's essentially what the X509_REQ
accessors do:

    long X509_REQ_get_version(const X509_REQ *req)
    {
        return ASN1_INTEGER_get(req->req_info.version);
    }

    X509_NAME *X509_REQ_get_subject_name(const X509_REQ *req)
    {
        return req->req_info.subject;
    }

    int X509_REQ_get_attr_count(const X509_REQ *req)
    {
        return X509at_get_attr_count(req->req_info.attributes);
    }

    int X509_REQ_get_attr_by_NID(const X509_REQ *req, int nid, int lastpos)
    {
        return X509at_get_attr_by_NID(req->req_info.attributes, nid, lastpos);
    }

    int X509_REQ_get_attr_by_OBJ(const X509_REQ *req, const ASN1_OBJECT *obj,
                                 int lastpos)
    {
        return X509at_get_attr_by_OBJ(req->req_info.attributes, obj, lastpos);
    }

    X509_ATTRIBUTE *X509_REQ_get_attr(const X509_REQ *req, int loc)
    {
        return X509at_get_attr(req->req_info.attributes, loc);
    }

    X509_ATTRIBUTE *X509_REQ_delete_attr(X509_REQ *req, int loc)
    {
        return X509at_delete_attr(req->req_info.attributes, loc);
    }

If one were to "void the warranty", one could cast the (X509_REQ_INFO
*) as an (X509_REQ *), and the accessors would just work, but you
must not do that, the internal details might change some day, as
they did between 1.1.x and 1.0.2 (where the X509_REQ_INFO is a
separately allocated structure pointed to by the X509_REQ).

> > Can you be more specific about these "module boundaries”?
>
> The modules are Apache httpd modules, and the boundaries between the modules
> are hooks that pass DER encoded structures between each module.

Well, so the key question is, why not pass an actual CSR.  What's
preventing the CSR from being signed?

> > This isn't pretty, and perhaps we need some new functions to explicitly
> > embed a CRI in a CSR, but it is certainly something you can do in the
> > short term.
>
> Can we not rather fix the initialisation of the X509_REQ in X509_REQ_new()
> so that it works like it used to? It seems like a massive headache to do
> something that used to be trivial.

No, because it is not broken.  The "massive headache" is peculiar to
the rather odd choice of "RPC" between these Apache modules, where
a non-CSR is masquerading as a CSR.

> I see there have been changes to openssl code relating to how structures
> are initialised, I suspect an error has crept in where an ASN.1 object is
> missing instead of empty, thus the malformed CSR.

There is no error.  The X509_REQ_INFO and X509_ALGOR (signature
algorithm) are now embedded in the CSR, and are no longer optional.

Your immediate choices are to sign a CSR, or to embed your CRI in
an outer DER wrapper that simulates a CSR.  Signing with some
key (ideally the subject key) seems simplest.

Even if we provide the "missing" CRI accessors in OpenSSL 1.1.1c,
I don't think that would practically solve your problem, because
some users would have 1.1.0, 1.1.1, 1.1.1a or 1.1.1b.

The DER wrapper is also simple (commented hex):

    $ openssl req -config <(
        printf "distinguished_name = dn\n[dn]\nprompt=yes\n[v3req]\n%s\n" \
                       "subjectAltName = DNS:example.com"
            ) \
        -reqexts v3req -new -newkey rsa:1024 -keyout /dev/null \
        -nodes -subj / 2>/dev/null |
        openssl req -outform DER |
        hexdump -ve '16/1 " %02x" "\n"'
     30 82 01 68 30 81 d2 02 01 00 30 00 30 81 9f 30
#    wrapper -->|<-- CRI DER begins here
     0d 06 09 2a 86 48 86 f7 0d 01 01 01 05 00 03 81
     8d 00 30 81 89 02 81 81 00 d0 61 3e 71 9c f6 a1
     d0 bd cf 7c f2 e6 be a2 5b 59 4a f5 8f 32 9d d6
     47 38 43 1a 57 51 18 bd e1 a2 1d 52 d0 8d c5 79
     07 c4 3c 78 19 5c ca c7 37 69 2a 22 70 05 e1 ab
     ec 35 b2 2b 1d 46 f7 8f b9 bd 30 27 08 81 1c ac
     81 a1 c3 ed 5b 9f 9e 46 22 d3 28 45 3c 85 36 30
     7e 11 46 ae 65 d0 8b 1d 19 28 3b 76 d6 f0 fd 59
     b3 62 5a e9 45 21 92 29 1d 0f af ad 87 c9 33 fc
     3d 87 5f 68 98 96 89 a6 b3 02 03 01 00 01 a0 29
     30 27 06 09 2a 86 48 86 f7 0d 01 09 0e 31 1a 30
     18 30 16 06 03 55 1d 11 04 0f 30 0d 82 0b 65 78
     61 6d 70 6c 65 2e 63 6f 6d 30 0d 06 09 2a 86 48
#    CRI DER ends here ------->|<--- begin sig OID
     86 f7 0d 01 01 0b 05 00 03 81 81 00 00 00 00 00
#    end sig OID ---->| null|<- 1024 zeroed sig bits
     00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
     00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
     00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
     00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
     00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
     00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
     00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
     00 00 00 00 00 00 00 00 00 00 00 00
#                                       |<-- end sig

Appending a fixed 147 byte suffix, and prepending an outer sequence
tag and length is not difficult.  But, since the signature is fake
anyway, it could be much shorter, for example truncating it to an
8-bit "0" signature (19 byte suffix with the OID and null parameters),
I get:

    $ perl -pe 's/\s//g; s/(..)/chr(hex($1))/eg;' <<-EOF | openssl req -inform DER -text
         30 81 e8 30 81 d2 02 01 00 30 00 30 81 9f 30
         0d 06 09 2a 86 48 86 f7 0d 01 01 01 05 00 03 81
         8d 00 30 81 89 02 81 81 00 d0 61 3e 71 9c f6 a1
         d0 bd cf 7c f2 e6 be a2 5b 59 4a f5 8f 32 9d d6
         47 38 43 1a 57 51 18 bd e1 a2 1d 52 d0 8d c5 79
         07 c4 3c 78 19 5c ca c7 37 69 2a 22 70 05 e1 ab
         ec 35 b2 2b 1d 46 f7 8f b9 bd 30 27 08 81 1c ac
         81 a1 c3 ed 5b 9f 9e 46 22 d3 28 45 3c 85 36 30
         7e 11 46 ae 65 d0 8b 1d 19 28 3b 76 d6 f0 fd 59
         b3 62 5a e9 45 21 92 29 1d 0f af ad 87 c9 33 fc
         3d 87 5f 68 98 96 89 a6 b3 02 03 01 00 01 a0 29
         30 27 06 09 2a 86 48 86 f7 0d 01 09 0e 31 1a 30
         18 30 16 06 03 55 1d 11 04 0f 30 0d 82 0b 65 78
         61 6d 70 6c 65 2e 63 6f 6d 30 0d 06 09 2a 86 48
         86 f7 0d 01 01 0b 05 00 03 02 00 00
        EOF
Certificate Request:
    Data:
        Version: 0 (0x0)
        Subject:
        Subject Public Key Info:
            Public Key Algorithm: rsaEncryption
                Public-Key: (1024 bit)
                Modulus:
                    00:d0:61:3e:71:9c:f6:a1:d0:bd:cf:7c:f2:e6:be:
                    a2:5b:59:4a:f5:8f:32:9d:d6:47:38:43:1a:57:51:
                    18:bd:e1:a2:1d:52:d0:8d:c5:79:07:c4:3c:78:19:
                    5c:ca:c7:37:69:2a:22:70:05:e1:ab:ec:35:b2:2b:
                    1d:46:f7:8f:b9:bd:30:27:08:81:1c:ac:81:a1:c3:
                    ed:5b:9f:9e:46:22:d3:28:45:3c:85:36:30:7e:11:
                    46:ae:65:d0:8b:1d:19:28:3b:76:d6:f0:fd:59:b3:
                    62:5a:e9:45:21:92:29:1d:0f:af:ad:87:c9:33:fc:
                    3d:87:5f:68:98:96:89:a6:b3
                Exponent: 65537 (0x10001)
        Attributes:
        Requested Extensions:
            X509v3 Subject Alternative Name:
                DNS:example.com
    Signature Algorithm: sha256WithRSAEncryption
         00
-----BEGIN CERTIFICATE REQUEST-----
MIHoMIHSAgEAMAAwgZ8wDQYJKoZIhvcNAQEBBQADgY0AMIGJAoGBANBhPnGc9qHQ
vc988ua+oltZSvWPMp3WRzhDGldRGL3hoh1S0I3FeQfEPHgZXMrHN2kqInAF4avs
NbIrHUb3j7m9MCcIgRysgaHD7VufnkYi0yhFPIU2MH4RRq5l0IsdGSg7dtbw/Vmz
YlrpRSGSKR0Pr62HyTP8PYdfaJiWiaazAgMBAAGgKTAnBgkqhkiG9w0BCQ4xGjAY
MBYGA1UdEQQPMA2CC2V4YW1wbGUuY29tMA0GCSqGSIb3DQEBCwUAAwIAAA==
-----END CERTIFICATE REQUEST-----

The prefix would often be 4 bytes (i.e. 16-bit rather than an 8-bit
length), rather than 3.  As, for example always with 2048-bit RSA
subject key.  [ The above CSR is atypically short. ]

--
        Viktor.
Reply | Threaded
Open this post in threaded view
|

Re: i2d_X509_REQ() -> d2i_X509_REQ() = asn1 encoding routines:c2i_ASN1_OBJECT:invalid object encoding:a_object.c:287

Graham Leggett
On 19 Mar 2019, at 18:00, Viktor Dukhovni <[hidden email]> wrote
:
> Well, the *standard* structure for passing around just the unsigned
> data underlying a CSR (X509_REQ), is a CertificationRequestInfo
> (X509_REQ_INFO).  So if the modules are to use *standard* structures
> to communicate.  The object being passed needs to be either a CSR
> (signed) or the enclosed CRI (unsigned).

I agree - it is the ideal structure to use, however translating this into real world implementation there aren’t any APIs in openssl that allow us to do this today, and it is very likely that the same limitation exists in other APIs we would like to support in future (NSS, native APIs, etc).

This software however is 7 years old, we’re not in a position to drop everything and rewrite it.

> You could, for example, sign the request with some suitable key
> (ideally the private key corresponding to the public key in the
> CSR, if available) before handing it off.  If the signing key is
> not the enclosed public key, it would not pass "req -verify" (it
> never did before either, for lack of a signature), but the called
> module would be able to decode a CSR, and work as before.

In our world we’re translating from various protocols (scep, spkac, etc etc) where proof of possession isn’t a signed X509_REQ, but is rather a challenge passphrase, previous certificate, etc etc, to a standard object (CSR) that can then be signed by a range of modules (simple local signing, signing on a smartcard, etc etc).

As a result while in the ideal world we would be dealing with signed CSRs, in our world we have to support a CSR with proof of possession supplied alongside in a range of possible formats.

We might have to go with the sign-with-a-dummy-signature route, but this would be unfortunate.

>> I don’t follow - in order to get access to the data inside the X509_REQ_INFO
>> structure, I need to first wrap it in a X509_REQ, otherwise I have no API
>> calls to get access to the data inside it.
>
> No need to get access to the data inside an X509_REQ_INFO is expected.
> That object's sole purpose is to be serializable for signing.
>
> You have rather an edge-case, where for some reason you're delegating
> signing to a CA module by passing it a non-standard structure that
> *resembles* a CSR, which is however for some reason not signed with
> the subject key (not signed at all), and you expect the CA to apply
> policy, by decoding the CRI inside  this non-CSR.
>
> OpenSSL 1.1.x does not have structure member accessors for a CRI,
> but they would be easy to add, that's essentially what the X509_REQ
> accessors do:
>
>    long X509_REQ_get_version(const X509_REQ *req)
>    {
> return ASN1_INTEGER_get(req->req_info.version);
>    }
>
>    X509_NAME *X509_REQ_get_subject_name(const X509_REQ *req)
>    {
> return req->req_info.subject;
>    }
>
>    int X509_REQ_get_attr_count(const X509_REQ *req)
>    {
> return X509at_get_attr_count(req->req_info.attributes);
>    }
>
>    int X509_REQ_get_attr_by_NID(const X509_REQ *req, int nid, int lastpos)
>    {
> return X509at_get_attr_by_NID(req->req_info.attributes, nid, lastpos);
>    }
>
>    int X509_REQ_get_attr_by_OBJ(const X509_REQ *req, const ASN1_OBJECT *obj,
> int lastpos)
>    {
> return X509at_get_attr_by_OBJ(req->req_info.attributes, obj, lastpos);
>    }
>
>    X509_ATTRIBUTE *X509_REQ_get_attr(const X509_REQ *req, int loc)
>    {
> return X509at_get_attr(req->req_info.attributes, loc);
>    }
>
>    X509_ATTRIBUTE *X509_REQ_delete_attr(X509_REQ *req, int loc)
>    {
> return X509at_delete_attr(req->req_info.attributes, loc);
>    }
>
> If one were to "void the warranty", one could cast the (X509_REQ_INFO
> *) as an (X509_REQ *), and the accessors would just work, but you
> must not do that, the internal details might change some day, as
> they did between 1.1.x and 1.0.2 (where the X509_REQ_INFO is a
> separately allocated structure pointed to by the X509_REQ).
We’re a modular CA, we don’t dictate to our modules what they can and can’t do, so to say “that object’s sole purpose” is a contradiction.

It turns out that the module that checks the proof of possession against an LDAP server needs to pull out the subject of the CSR in order to make the LDAP query, which in turn means it needs to be able to read the contents of the X509_REQ (or X509_REQ_INFO).

X509_REQ_INFO prevents us from seeing the information, and so this won’t work for us.

>> The modules are Apache httpd modules, and the boundaries between the modules
>> are hooks that pass DER encoded structures between each module.
>
> Well, so the key question is, why not pass an actual CSR.  What's
> preventing the CSR from being signed?

Protocols like scep and spkac don’t supply a CSR, that’s not how they work.

>>> This isn't pretty, and perhaps we need some new functions to explicitly
>>> embed a CRI in a CSR, but it is certainly something you can do in the
>>> short term.
>>
>> Can we not rather fix the initialisation of the X509_REQ in X509_REQ_new()
>> so that it works like it used to? It seems like a massive headache to do
>> something that used to be trivial.
>
> No, because it is not broken.  The "massive headache" is peculiar to
> the rather odd choice of "RPC" between these Apache modules, where
> a non-CSR is masquerading as a CSR.
DER encoding is well standardised, and modularity is well understood. None of this is an odd choice in a modular CA.

>> I see there have been changes to openssl code relating to how structures
>> are initialised, I suspect an error has crept in where an ASN.1 object is
>> missing instead of empty, thus the malformed CSR.
>
> There is no error.  The X509_REQ_INFO and X509_ALGOR (signature
> algorithm) are now embedded in the CSR, and are no longer optional.

And yet here we are.

The problem seems that at some point during the 1.0.2 releases, the X509_REQ was previously serialised with what looks like an empty X509_ALGOR structure:

 507:d=2  hl=2 l=   1 prim: OBJECT            :itu-t

and at some point openssl changed to serialise with a bad object instead:

 508:d=2  hl=2 l=   0 prim: OBJECT            :BAD OBJECT

A change from something from optional to mandatory should have happened in the 1.1.0 release, not within the 1.0.2 point releases.

Regards,
Graham



smime.p7s (4K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: i2d_X509_REQ() -> d2i_X509_REQ() = asn1 encoding routines:c2i_ASN1_OBJECT:invalid object encoding:a_object.c:287

Viktor Dukhovni
> On Mar 20, 2019, at 8:36 PM, Graham Leggett <[hidden email]> wrote:
>
> The problem seems that at some point during the 1.0.2 releases, the X509_REQ was previously serialised with what looks like an empty X509_ALGOR structure:
>
> 507:d=2  hl=2 l=   1 prim: OBJECT            :itu-t
>
> and at some point openssl changed to serialise with a bad object instead:
>
> 508:d=2  hl=2 l=   0 prim: OBJECT            :BAD OBJECT
>
> A change from something from optional to mandatory should have happened in the 1.1.0 release, not within the 1.0.2 point releases.

Sorry, are you saying that the serialized form changed in a 1.0.2
patch release?  I may have missed that detail.

Your github issue notes a change from 1.0.1 to 1.0.2, not a change
within the 1.0.2 patch release series...

Changes of behaviour between 1.0.1 and 1.0.2 were not off limits.

When testing a short program to reproduce the reported symptoms
I see the same output encoding (BAD OBJECT) with both 1.0.1 and
1.0.2.  For me, it is only 0.9.8 and 1.0.0 that generate the "0" OID.

    0:d=0  hl=3 l= 163 cons: SEQUENCE
    3:d=1  hl=3 l= 152 cons: SEQUENCE
    6:d=2  hl=2 l=   1 prim: INTEGER           :00
    9:d=2  hl=2 l=  51 cons: SEQUENCE
   11:d=3  hl=2 l=  11 cons: SET
   13:d=4  hl=2 l=   9 cons: SEQUENCE
   15:d=5  hl=2 l=   3 prim: OBJECT            :countryName
   20:d=5  hl=2 l=   2 prim: PRINTABLESTRING   :US
   24:d=3  hl=2 l=  13 cons: SET
   26:d=4  hl=2 l=  11 cons: SEQUENCE
   28:d=5  hl=2 l=   3 prim: OBJECT            :organizationName
   33:d=5  hl=2 l=   4 prim: UTF8STRING        :None
   39:d=3  hl=2 l=  21 cons: SET
   41:d=4  hl=2 l=  19 cons: SEQUENCE
   43:d=5  hl=2 l=   3 prim: OBJECT            :commonName
   48:d=5  hl=2 l=  12 prim: UTF8STRING        :test.example
   62:d=2  hl=2 l=  92 cons: SEQUENCE
   64:d=3  hl=2 l=  13 cons: SEQUENCE
   66:d=4  hl=2 l=   9 prim: OBJECT            :rsaEncryption
   77:d=4  hl=2 l=   0 prim: NULL
   79:d=3  hl=2 l=  75 prim: BIT STRING
  156:d=2  hl=2 l=   0 cons: cont [ 0 ]
  158:d=1  hl=2 l=   3 cons: SEQUENCE
  160:d=2  hl=2 l=   1 prim: OBJECT            :0.0
  163:d=1  hl=2 l=   1 prim: BIT STRING

With 1.0.1 I get:

    0:d=0  hl=3 l= 162 cons: SEQUENCE
    3:d=1  hl=3 l= 152 cons: SEQUENCE
    6:d=2  hl=2 l=   1 prim: INTEGER           :00
    9:d=2  hl=2 l=  51 cons: SEQUENCE
   11:d=3  hl=2 l=  11 cons: SET
   13:d=4  hl=2 l=   9 cons: SEQUENCE
   15:d=5  hl=2 l=   3 prim: OBJECT            :countryName
   20:d=5  hl=2 l=   2 prim: PRINTABLESTRING   :US
   24:d=3  hl=2 l=  13 cons: SET
   26:d=4  hl=2 l=  11 cons: SEQUENCE
   28:d=5  hl=2 l=   3 prim: OBJECT            :organizationName
   33:d=5  hl=2 l=   4 prim: UTF8STRING        :None
   39:d=3  hl=2 l=  21 cons: SET
   41:d=4  hl=2 l=  19 cons: SEQUENCE
   43:d=5  hl=2 l=   3 prim: OBJECT            :commonName
   48:d=5  hl=2 l=  12 prim: UTF8STRING        :test.example
   62:d=2  hl=2 l=  92 cons: SEQUENCE
   64:d=3  hl=2 l=  13 cons: SEQUENCE
   66:d=4  hl=2 l=   9 prim: OBJECT            :rsaEncryption
   77:d=4  hl=2 l=   0 prim: NULL
   79:d=3  hl=2 l=  75 prim: BIT STRING
  156:d=2  hl=2 l=   0 cons: cont [ 0 ]
  158:d=1  hl=2 l=   2 cons: SEQUENCE
  160:d=2  hl=2 l=   0 prim: OBJECT            :BAD OBJECT
  162:d=1  hl=2 l=   1 prim: BIT STRING

--
        Viktor.

Reply | Threaded
Open this post in threaded view
|

Re: i2d_X509_REQ() -> d2i_X509_REQ() = asn1 encoding routines:c2i_ASN1_OBJECT:invalid object encoding:a_object.c:287

Viktor Dukhovni
> On Mar 20, 2019, at 9:35 PM, Viktor Dukhovni <[hidden email]> wrote:
>
> Your github issue notes a change from 1.0.1 to 1.0.2, not a change
> within the 1.0.2 patch release series...
>
> Changes of behaviour between 1.0.1 and 1.0.2 were not off limits.
>
> When testing a short program to reproduce the reported symptoms
> I see the same output encoding (BAD OBJECT) with both 1.0.1 and
> 1.0.2.  For me, it is only 0.9.8 and 1.0.0 that generate the "0" OID.

The change dates back to:

  https://github.com/openssl/openssl/commit/38a503fb8adaad331b8a67aaa9eb2fa915113ed0

which first appeared in OpenSSL 1.0.1i.  It was cherry-picked onto the
1.0.2 branch:

   https://github.com/openssl/openssl/commit/ff4cfc4c588c41d5e8d2d530231bc36cbc525add

The effect of this is that "NID_undef" generates an empty ASN.1 value, rather
than an OID of "0.0".  Single component OIDs are not representable in ASN.1
BER/DER since the OID encoding combines the first elements as a single value.

Encoding "NID_undef" (uninitialized OID) as 0.0 isn't right, because there is no
OID, and we're pretending to have an OID of 0.0.  Of course not encoding anything
isn't great either, we're then skipping an element of the enclosing structure and
generating incorrect ASN.1 for the containing structure.

Really what could/should probably happen here, is that i2d should fail, because
the structure specifies an object that has no ASN.1 encoding.

--
        Viktor.

Reply | Threaded
Open this post in threaded view
|

Re: i2d_X509_REQ() -> d2i_X509_REQ() = asn1 encoding routines:c2i_ASN1_OBJECT:invalid object encoding:a_object.c:287

OpenSSL - User mailing list
In reply to this post by Graham Leggett
>    This software however is 7 years old, we’re not in a position to drop everything and rewrite it.
 

Then don't upgrade?  If it's for a CA you don't need TLS 1.3 for example.

Or take the existing OpenSSL code that works and jam it into the current release.

Reply | Threaded
Open this post in threaded view
|

Re: i2d_X509_REQ() -> d2i_X509_REQ() = asn1 encoding routines:c2i_ASN1_OBJECT:invalid object encoding:a_object.c:287

Viktor Dukhovni
On Thu, Mar 21, 2019 at 01:00:14PM +0000, Salz, Rich via openssl-users wrote:

> >    This software however is 7 years old, we’re not in a position to drop everything and rewrite it.
>
> Then don't upgrade?  If it's for a CA you don't need TLS 1.3 for example.
>
> Or take the existing OpenSSL code that works and jam it into the current release.

Perhaps the OP is not an end-user, but rather maintains these modules
for a user community, in which case "don't upgrade" is not generally
an option.  In any case, it seems a bit premature to close the
conversation.

While the original decision to use incomplete (and thus invalid)
CSRs, is unfortunate, and not OpenSSL's fault, we can still continue
to discuss meaningful options.  On the DER padding front, the minimal
working suffix is 7 bytes:

        30 03    -- Length 3 sequence
        06 01 00 -- OBJECT ID: 0.0
        03 01 00 -- empty BIT STRING

One then also prepends a prefix (typically 4 bytes):

        30 82 xx xx -- a sequence of 256 to 65535 bytes
        30 81 xx    -- a sequence of 128 to 255 bytes
        30 xx       -- a sequence of up to 127 bytes

The "xx" length is the DER length of the CRI + 7 bytes for the
suffix.

On the OpenSSL side, having found that we emit dubious encodings
of structures with an (unspecified) null OID element, I am considering
whether it would make sense to encode them as a zero-length (invalid,
but faithful) ASN.1 OBJECT:

    06 00

*and* decode these back to a zero length NID_undef object.  While
these are likely to not interoperate with many other ASN.1 decoder
implementations, just interoperating with the same version of OpenSSL
would make it possible to encode/decode partially initialized
structures in which some "objects" are as yet unspecified.

The reason I'm considering changes is that it now appears that the
original commmit that stopped encoding single element OIDs, is not
fully baked.  Just skipping required components of structures is
not a good outcome.

--
        Viktor.
Reply | Threaded
Open this post in threaded view
|

Re: i2d_X509_REQ() -> d2i_X509_REQ() = asn1 encoding routines:c2i_ASN1_OBJECT:invalid object encoding:a_object.c:287

Blumenthal, Uri - 0553 - MITLL
+1 to Viktor's points.

Regards,
Uri

Sent from my iPhone

> On Mar 21, 2019, at 12:52, Viktor Dukhovni <[hidden email]> wrote:
>
> On Thu, Mar 21, 2019 at 01:00:14PM +0000, Salz, Rich via openssl-users wrote:
>
>>>   This software however is 7 years old, we’re not in a position to drop everything and rewrite it.
>>
>> Then don't upgrade?  If it's for a CA you don't need TLS 1.3 for example.
>>
>> Or take the existing OpenSSL code that works and jam it into the current release.
>
> Perhaps the OP is not an end-user, but rather maintains these modules
> for a user community, in which case "don't upgrade" is not generally
> an option.  In any case, it seems a bit premature to close the
> conversation.
>
> While the original decision to use incomplete (and thus invalid)
> CSRs, is unfortunate, and not OpenSSL's fault, we can still continue
> to discuss meaningful options.  On the DER padding front, the minimal
> working suffix is 7 bytes:
>
>    30 03    -- Length 3 sequence
>    06 01 00 -- OBJECT ID: 0.0
>    03 01 00 -- empty BIT STRING
>
> One then also prepends a prefix (typically 4 bytes):
>
>    30 82 xx xx -- a sequence of 256 to 65535 bytes
>    30 81 xx    -- a sequence of 128 to 255 bytes
>    30 xx       -- a sequence of up to 127 bytes
>
> The "xx" length is the DER length of the CRI + 7 bytes for the
> suffix.
>
> On the OpenSSL side, having found that we emit dubious encodings
> of structures with an (unspecified) null OID element, I am considering
> whether it would make sense to encode them as a zero-length (invalid,
> but faithful) ASN.1 OBJECT:
>
>    06 00
>
> *and* decode these back to a zero length NID_undef object.  While
> these are likely to not interoperate with many other ASN.1 decoder
> implementations, just interoperating with the same version of OpenSSL
> would make it possible to encode/decode partially initialized
> structures in which some "objects" are as yet unspecified.
>
> The reason I'm considering changes is that it now appears that the
> original commmit that stopped encoding single element OIDs, is not
> fully baked.  Just skipping required components of structures is
> not a good outcome.
>
> --
>    Viktor.

smime.p7s (7K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: i2d_X509_REQ() -> d2i_X509_REQ() = asn1 encoding routines:c2i_ASN1_OBJECT:invalid object encoding:a_object.c:287

Viktor Dukhovni
On Thu, Mar 21, 2019 at 05:22:24PM +0000, Blumenthal, Uri - 0553 - MITLL wrote:

> > On the DER padding front, the minimal
> > working suffix is 7 bytes:

Apparently I can't count today, clearly the suffix is 8 bytes.

> >
> >    30 03    -- Length 3 sequence
> >    06 01 00 -- OBJECT ID: 0.0
> >    03 01 00 -- empty BIT STRING
> >
> > On the OpenSSL side, having found that we emit dubious encodings
> > of structures with an (unspecified) null OID element, I am considering
> > whether it would make sense to encode them as a zero-length (invalid,
> > but faithful) ASN.1 OBJECT:
> >
> >    06 00
> >
> > *and* decode these back to a zero length NID_undef object.

After discussing this idea with a friend, I am less enthusiastic
about this option.  His point is that if OpenSSL starts emitting
invalid empty OIDs as a way to support encoding incompletely
initialized structures, this could contaminate the ecosystem with
subsequent new downstream work-arounds in other implementations.

His order of "preference" is:

    1.  Return failure from i2d_ASN_OBJECT(), which then percolates
        up to failure to encode the containing structure.

    2.  Emit a "harmless" default OID (such as 0.0), returning to
        the behaviour prior to 1.0.1i

    3.  Emit the invalid empty OID (06 00) in the expectation that
        this would not be something that other decoders would have
        to support.  That is, it would only be used, as in this case,
        to serialize and deserialize objects *within* an application,
        and there would be no pressure on other implementations to
        follow suit.

I am curious what other OpenSSL developers and users would like to
see happen.  Any of the above?  Or something else?  The present
behaviour seems wrong to me, because we're silently generating
invalid structures with missing required fields (when encoding
incompletely initialized structures).

Failing in i2d_ASN1_OBJECT() is unlikely to do harm, because the
current invalid output is not better, and we've not seen any
complaints until now in ~5 years of OpenSSL 1.0.2 deployment.
So use of i2d on partially created objects looks rather rare,
and perhaps explicit failure is better than any ad-hoc output?

--
        Viktor.
12