tedge-p11-server client signing failed due to selected different keys to the given uri #3601
Description
Describe the bug
A user which is exploring stores stored in a Nitrokey HSM 2 module which has multiple private keys (at least 3 different ones) failed to connect to Cumulocity using the private key which is accessed via the tedge-p11-server service.
When doing a tedge connect c8y, the following error appears on the tedge-p11-server logs and the client fails to connect to the cloud.
May 08 23:27:09 100666-2-nc-hmi tedge-p11-server[1907642]: 1: Function::Sign: PKCS11 error: The plaintext input data to a cryptographic operation has a bad length. Depending on the operation’s mechanism, this could mean that the plaintext data is too short, too long, or is not a multiple of some particular block size. This return value has higher priority than CKR_DATA_INVALID.
Upon closer inspection it seems that the tedge-p11-server key selection (which occurs multiple times in the handshake) does not always find the correct key using the given key's url (either specified on the tedge-p11-server --uri flag, or using the client side setting device.key_uri). The algorithm selection seems to be ok, but when it tries to sign the data, it selects a different key which results in the error.
For instance, in this case, the private key is an EC based key, however for signing, the tedge-p11-server picked another key which was RSA based.
It seems that the tedge-p11-server key selection does not always honour the given key's uri. When hard coding the key's label, tedge was then able to connect successfully, below shows the hard-coded key template criteria.
let mut key_template = vec![
Attribute::Token(true),
Attribute::Private(true),
Attribute::Sign(true),
];
// TEST: Hard code a unique label just to prove there was nothing wrong with the key itself
key_template.push(Attribute::Label("device-cert".as_bytes().to_vec()));To Reproduce
Setup
- tedge-p11-server running on the host
- tedge running inside a container and using the tedge-p11-server socket mounted inside the container (with correct permissions)
- Nitrokey HSM 2 with at least 3 different keys on the same slot
Expected behavior
The TLS scheme selection and signing should use the same key as specified by the PKCS11 URI which is either specified via the tedge-p11-server --uri <uri> flag, or via the client.
Screenshots
Environment (please complete the following information):
| Property | Value |
|---|---|
| OS [incl. version] | Debian GNU/Linux 12 (bookworm) |
| Hardware [incl. revision] | unknown |
| System-Architecture | Linux 100666-2-nc-hmi 6.1.0-32-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.1.129-1 (2025-03-06) x86_64 GNU/Linux |
| thin-edge.io version | 1.5.0 |
Additional context
Below shows the full logs from the tedge-p11-server showing the requests run during a tedge connect c8y.
May 08 23:27:09 100666-2-nc-hmi tedge-p11-server[1907642]: 2025-05-08T21:27:09.072298Z TRACE choose_scheme: tedge_p11_server::service: request=ChooseSchemeRequest { offered: [SignatureScheme(ECDSA_NISTP256_SHA256), SignatureScheme(ECDSA_NISTP384_SHA384), SignatureScheme(ECDSA_NISTP521_SHA512), SignatureScheme(RSA_PSS_SHA256), SignatureScheme(RSA_PSS_SHA384), SignatureScheme(RSA_PSS_SHA512)], uri: Some("pkcs11:model=PKCS%2315%20emulated;manufacturer=www.CardContact.de;serial=DENK12345;token=SmartCard-HSM%20%28UserPIN%29;id=%01;type=private") }
May 08 23:27:09 100666-2-nc-hmi tedge-p11-server[1907642]: 2025-05-08T21:27:09.146150Z DEBUG choose_scheme: tedge_p11_server::pkcs11: Selected slot slot_info=SlotInfo { slot_description: "Nitrokey Nitrokey HSM (DENK123450000 ) 00 00", manufacturer_id: "Nitrokey", flags: TOKEN_PRESENT | REMOVABLE_DEVICE | HW_SLOT, hardware_version: Version { major: 0, minor: 0 }, firmware_version: Version { major: 0, minor: 0 } } token_info=TokenInfo { label: "SmartCard-HSM (UserPIN)", manufacturer_id: "www.CardContact.de", model: "PKCS#15 emulated", serial_number: "DENK12345", flags: RNG | LOGIN_REQUIRED | USER_PIN_INITIALIZED | TOKEN_INITIALIZED, max_session_count: Infinite, session_count: Some(0), max_rw_session_count: Infinite, rw_session_count: Some(0), max_pin_len: 15, min_pin_len: 6, total_public_memory: None, free_public_memory: None, total_private_memory: None, free_private_memory: None, hardware_version: Version { major: 24, minor: 13 }, firmware_version: Version { major: 4, minor: 1 }, utc_time: None }
May 08 23:27:09 100666-2-nc-hmi tedge-p11-server[1907642]: 2025-05-08T21:27:09.223111Z DEBUG choose_scheme: tedge_p11_server::pkcs11: Opened a readonly session session_info=SessionInfo { slot_id: Slot { slot_id: 0 }, state: RoUser, flags: SERIAL_SESSION, device_error: 0 }
May 08 23:27:09 100666-2-nc-hmi tedge-p11-server[1907642]: 2025-05-08T21:27:09.223187Z TRACE choose_scheme: tedge_p11_server::pkcs11: Finding a key key_template=[Token(true), Private(true), Sign(true), Id([1])]
May 08 23:27:09 100666-2-nc-hmi tedge-p11-server[1907642]: 2025-05-08T21:27:09.223291Z TRACE choose_scheme: tedge_p11_server::pkcs11: Offered signature schemes. offered=[ECDSA_NISTP256_SHA256, ECDSA_NISTP384_SHA384, ECDSA_NISTP521_SHA512, RSA_PSS_SHA256, RSA_PSS_SHA384, RSA_PSS_SHA512]
May 08 23:27:09 100666-2-nc-hmi tedge-p11-server[1907642]: 2025-05-08T21:27:09.223324Z TRACE choose_scheme: tedge_p11_server::pkcs11: Matching scheme: Some("ECDSA_NISTP256_SHA256")
May 08 23:27:09 100666-2-nc-hmi tedge-p11-server[1907642]: 2025-05-08T21:27:09.223357Z TRACE choose_scheme: tedge_p11_server::pkcs11: Using Signature scheme: Some("ECDSA_NISTP256_SHA256")
May 08 23:27:09 100666-2-nc-hmi tedge-p11-server[1907642]: 2025-05-08T21:27:09.266130Z INFO tedge_p11_server::server: Incoming request successful
May 08 23:27:09 100666-2-nc-hmi tedge-p11-server[1907642]: 2025-05-08T21:27:09.267689Z TRACE sign: tedge_p11_server::service: request=SignRequest { to_sign: [32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 32, 84, 76, 83, 32, 49, 46, 51, 44, 32, 99, 108, 105, 101, 110, 116, 32, 67, 101, 114, 116, 105, 102, 105, 99, 97, 116, 101, 86, 101, 114, 105, 102, 121, 0, 148, 119, 40, 114, 204, 50, 175, 211, 225, 217, 20, 218, 253, 167, 114, 71, 160, 245, 195, 34, 60, 87, 245, 196, 15, 253, 26, 103, 37, 2, 203, 136, 143, 65, 55, 188, 192, 241, 58, 18, 186, 189, 193, 129, 59, 230, 82, 155], uri: Some("pkcs11:model=PKCS%2315%20emulated;manufacturer=www.CardContact.de;serial=DENK12345;token=SmartCard-HSM%20%28UserPIN%29;id=%01;type=private") }
May 08 23:27:09 100666-2-nc-hmi tedge-p11-server[1907642]: 2025-05-08T21:27:09.349118Z DEBUG sign: tedge_p11_server::pkcs11: Selected slot slot_info=SlotInfo { slot_description: "Nitrokey Nitrokey HSM (DENK123450000 ) 00 00", manufacturer_id: "Nitrokey", flags: TOKEN_PRESENT | REMOVABLE_DEVICE | HW_SLOT, hardware_version: Version { major: 0, minor: 0 }, firmware_version: Version { major: 0, minor: 0 } } token_info=TokenInfo { label: "SmartCard-HSM (UserPIN)", manufacturer_id: "www.CardContact.de", model: "PKCS#15 emulated", serial_number: "DENK12345", flags: RNG | LOGIN_REQUIRED | USER_PIN_INITIALIZED | TOKEN_INITIALIZED, max_session_count: Infinite, session_count: Some(0), max_rw_session_count: Infinite, rw_session_count: Some(0), max_pin_len: 15, min_pin_len: 6, total_public_memory: None, free_public_memory: None, total_private_memory: None, free_private_memory: None, hardware_version: Version { major: 24, minor: 13 }, firmware_version: Version { major: 4, minor: 1 }, utc_time: None }
May 08 23:27:09 100666-2-nc-hmi tedge-p11-server[1907642]: 2025-05-08T21:27:09.425153Z DEBUG sign: tedge_p11_server::pkcs11: Opened a readonly session session_info=SessionInfo { slot_id: Slot { slot_id: 0 }, state: RoUser, flags: SERIAL_SESSION, device_error: 0 }
May 08 23:27:09 100666-2-nc-hmi tedge-p11-server[1907642]: 2025-05-08T21:27:09.425219Z TRACE sign: tedge_p11_server::pkcs11: Finding a key key_template=[Token(true), Private(true), Sign(true), Id([1])]
May 08 23:27:09 100666-2-nc-hmi tedge-p11-server[1907642]: 2025-05-08T21:27:09.425345Z TRACE sign: tedge_p11_server::pkcs11: input_message= TLS 1.3, client CertificateVerify
May 08 23:27:09 100666-2-nc-hmi tedge-p11-server[1907642]: [261B blob data]
May 08 23:27:09 100666-2-nc-hmi tedge-p11-server[1907642]: 2025-05-08T21:27:09.425395Z TRACE sign: tedge_p11_server::pkcs11: Session::sign mechanism=Sha256RsaPkcsPss(PkcsPssParams { hash_alg: MechanismType { val: 592 }, mgf: PkcsMgfType { val: 2 }, s_len: Ulong { val: 32 } })
May 08 23:27:09 100666-2-nc-hmi tedge-p11-server[1907642]: 2025-05-08T21:27:09.564860Z ERROR tedge_p11_server::server: Incoming request failed: Failed to sign using PKCS #11
May 08 23:27:09 100666-2-nc-hmi tedge-p11-server[1907642]: Caused by:
May 08 23:27:09 100666-2-nc-hmi tedge-p11-server[1907642]: 0: pkcs11: Failed to sign message
May 08 23:27:09 100666-2-nc-hmi tedge-p11-server[1907642]: 1: Function::Sign: PKCS11 error: The plaintext input data to a cryptographic operation has a bad length. Depending on the operation’s mechanism, this could mean that the plaintext data is too short, too long, or is not a multiple of some particular block size. This return value has higher priority than CKR_DATA_INVALID.