I guess I could have raised it myself -- see https://www.rfc-editor.org/errata/eid6909 :-)

I will add a GlueCOSE test vector for this.

It is weird to check IVs as we never use them but we should do it since it is go-cose.

I have a general question here.

In RFC 8152 3.1, it states

The IV can be placed in the unprotected header as
modifying the IV will cause the decryption to yield plaintext that
is readily detectable as garbled.

and also

The 'Initialization Vector' and 'Partial
Initialization Vector' parameters MUST NOT both be present in the
same security layer.

What does the "security layer" refer to?

To me, it refers to the "protected" layer and "unprotected" layer, which means we also need to check the unprotected header.

"security layer" is not defined in the spec, but what I understood is that every layer is a security layer, but the spec is just using a fancy adjective.

And yes, we also have to check the unprotected header, as it is part of the same layer the protected header is in.

To me, it refers to the "protected" layer and "unprotected" layer, which means we also need to check the unprotected header.

In principle one can nest COSE messages. See https://www.rfc-editor.org/authors/rfc9052.html#appendix-B for an example.

Sibling header buckets (protected and unprotected) -- i.e., found in the same COSE message -- are considered to be at the same layer.

In principle one can nest COSE messages. See https://www.rfc-editor.org/authors/rfc9052.html#appendix-B for an example.

Thanks @thomas-fossati for pointing me to the right place. From the above appendix, a security layer means one COSE structure. In our case, that's COSE_Sign / COSE_Sign1 / COSE_Signature.

Therefore, the IV and Partial IV cannot exists in the same header set. For example, it's not allowed if there is an IV in the protected header and Partial IV in the unprotected header in a COSE_Sign1 message since they are in the same security layer. However, it is allowed to have IV in the header of COSE_Sign and Partial IV in the header of one of its COSE_Signature since they are different security layers.

Therefore, the IV and Partial IV cannot exists in the same header set. For example, it's not allowed if there is an IV in the protected header and Partial IV in the unprotected header in a COSE_Sign1 message since they are in the same security layer. However, it is allowed to have IV in the header of COSE_Sign and Partial IV in the header of one of its COSE_Signature since they are different security layers.

@shizhMSFT I've implemented this check, please take another look.