CMS/PKCS7 signatures do not use strict DER, causing a deprecation warning from pyca/cryptography

[DemiMarie]

Describe the Bug

CMS/PKCS7 signatures do not use DER. This causes pyca/cryptography to emit a deprecation warning because they want to remove support for arbitrary BER.

I am not the person who found this problem. I am merely relaying the report by @uedvt359 in pyca/cryptography#12936 (comment). Therefore, some of the information is missing.

To Reproduce

I do not have a reproducer myself, but see pyca/cryptography#12936 (comment).

Expected Behavior

CMS signatures use DER.

Screenshots and Logs

I don't have access to the deployment in question

Product Deployment

Please complete the following information:

• Deployment format: [e.g. software, container] Not sure.
• Version [e.g. 8.0.0] 9.3.7

Desktop

N/A as the request is being made from Python using the API. The relevant code is here.

Additional Context

pyca/cryptography has two parsers for CMS and PKCS12: a strict DER parser written in Rust and a BER parser using OpenSSL. The OpenSSL-based parser is deprecated, and pyca/cryptography raises a warning when it must be used. See pyca/cryptography#12936 for details.

[uedvt359]

jftr: i believe the ejbca behaviour is spec-compliant, and the python crypto people are in the wrong to try and convince the whole ecosystem to stop using BER-encoding in that way.

[DemiMarie]

It’s worth noting that RPKI explicitly requires DER.

[primetomas]

I believe this is about using indefinite length encoding.

For PKCS#12 files we've already moved to strict definite length encoding, because certain modes of old openssl versions required that.

Reading the thread it looks clear that for CMS messages it's actually the current best practice. I have to say it's bold of pyca to decide to not be interoperable with the windows ecosystem :-). Having said that, we don't mind being "better" than MS, but it's clear that it's not a bug right. If you can re-label this as a feature request I will at least start asking the Bouncy Castle team on their opinion.

[primetomas]

It’s worth noting that RPKI explicitly requires DER.

Can you be more specific here and point to a source, DER for what?

[DemiMarie]

It’s worth noting that RPKI explicitly requires DER.

Can you be more specific here and point to a source, DER for what?

RFC6488 requires that RPKI objects be DER-encoded. They use a subset of CMS.

[DemiMarie]

I believe this is about using indefinite length encoding.

Not just indefinite lengths, but full strict DER, including (but not limited to) minimum-length encodings and sorting of SET tags. Though at least definite-length encoding would be enough to make parsing (much) easier and interoperate with mbedTLS.

For PKCS#12 files we've already moved to strict definite length encoding, because windows and openssl both started requiring that.

That’s good.

Reading the thread it looks clear that for CMS messages it's actually the current best practice. I have to say it's bold of pyca to decide to not be interoperable with the windows ecosystem :-).

EDK2 actually checks that the outmost SEQUENCE of an Authenticode signature uses 2-byte definite-length encoding, so I thought that at least Authenticode use definite-length encoding everywhere.

Having said that, we don't mind being "better" than MS, but it's clear that it's not a bug right. If you can re-label this as a feature request I will at least start asking the Bouncy Castle team on their opinion.

I changed the title, but I don’t have permission to change the tags.

[primetomas]

I was wrong on the p12 topic. You can find that as a closed issue here. It was only a problem with some specific usages of old versions of OpenSSL, but we decided it was worthwhile changing.

[primetomas]

Regarding RPKI and Authenticode, we’re not arguing here about other standards usage of CMS, or by weather one type of encoding is considered better or worse. Anything not following an RFC is of course a bug. But here we talk about certs-only CMS messages used to carry CA and end user certificates, where the RFC does allow this, and it’s used widely.
EJBCA doesn’t create neither RPKI nor Authenticode asignatures, so those are not relevant here. Right, or is there some else that I’m missing?

[DemiMarie]

Regarding RPKI and Authenticode, we’re not arguing here about other standards usage of CMS, or by weather one type of encoding is considered better or worse. Anything not following an RFC is of course a bug. But here we talk about certs-only CMS messages used to carry CA and end user certificates, where the RFC does allow this, and it’s used widely.

I was not aware that these messages even existed! I know next to nothing about CMS outside of signatures.

EJBCA doesn’t create neither RPKI nor Authenticode asignatures, so those are not relevant here. Right, or is there some else that I’m missing?

To me, RPKI requiring DER implies that DER is considered better than BER for CMS. That is all.

[primetomas]

I was not aware that these messages even existed! I know next to nothing about CMS outside of signatures.

CMS can be used in a myriad of different places. It would be good to know what kind of messages the pyca thread refers to. But with EJBCA the typical places are "download CA certificates chain", "issued certificate chain in some protocols". Protocol RFC should be checked so it doesn't specify anything else, i.e. rfc5272 talks about certs-only CMS, but doesn't specify details, except for somewhere found "It is a degenerate SignedData structure where the signer information and content are omitted, containing only the certificates field."

What some consider better others consider worse, standards is always about compliance never about good or bad. The only ones that get away by violating standards are the big ones, i.e. everyone have to be interoperable with them anyhow. Which is why it's hard to enforce definite length encoded CMS messages if ADCS indefinite length encoded ones.

[DemiMarie]

Ah, I was not aware of ADCS using indefinite length encoding.