OpenSSL server ECDSA signature curve mismatch rejection #2661
Description
Description
When connecting to an OpenSSL server (1.0.2k) using ECDSA P-256 certificate, rustls client fails with a signature algorithm mismatch error, while OpenSSL client (3.0.13) connects successfully.
The server is configured with -serverpref option which affects the signature algorithm selection. The server chooses ecdsa_secp521r1_sha512 for the ServerKeyExchange signature, despite using a P-256-H-256 certificate.
Repro Steps
Server:
$ openssl version
OpenSSL 1.0.2k-fips 26 Jan 2017$ openssl ecparam -name prime256v1 -genkey -noout -out key.pem && openssl req -x509 -nodes -key key.pem -out cert.pem$ openssl x509 -in cert.pem -text | grep -i algorithm
Signature Algorithm: ecdsa-with-SHA256
Public Key Algorithm: id-ecPublicKey
Signature Algorithm: ecdsa-with-SHA256$ openssl s_server -accept 4433 -cert cert.pem -key key.pem -serverprefClient
use anyhow::Result;
use std::{convert::TryInto, sync::Arc};
use tokio::net::TcpStream;
#[tokio::main]
async fn main() -> Result<()> {
let verifier = Verifier::default();
let tls_config = rustls::client::ClientConfig::builder()
.dangerous()
.with_custom_certificate_verifier(Arc::new(verifier))
.with_no_client_auth();
let tls_connector = tokio_rustls::TlsConnector::from(Arc::new(tls_config));
let tcp_stream = TcpStream::connect("myserver:4433").await?;
tls_connector
.connect("myserver".try_into()?, tcp_stream)
.await?;
Ok(())
}
#[derive(Debug)]
struct Verifier(rustls_platform_verifier::Verifier);
impl Default for Verifier {
fn default() -> Self {
let crypto_provider = rustls::crypto::aws_lc_rs::default_provider();
Self(rustls_platform_verifier::Verifier::new(Arc::new(crypto_provider)).unwrap())
}
}
impl rustls::client::danger::ServerCertVerifier for Verifier {
fn verify_server_cert(
&self,
_end_entity: &rustls::pki_types::CertificateDer<'_>,
_intermediates: &[rustls::pki_types::CertificateDer<'_>],
_server_name: &rustls::pki_types::ServerName<'_>,
_ocsp_response: &[u8],
_now: rustls::pki_types::UnixTime,
) -> std::result::Result<rustls::client::danger::ServerCertVerified, rustls::Error> {
Ok(rustls::client::danger::ServerCertVerified::assertion())
}
fn verify_tls12_signature(
&self,
message: &[u8],
cert: &rustls::pki_types::CertificateDer<'_>,
dss: &rustls::DigitallySignedStruct,
) -> std::result::Result<rustls::client::danger::HandshakeSignatureValid, rustls::Error> {
rustls::client::danger::ServerCertVerifier::verify_tls12_signature(
&self.0, message, cert, dss,
)
}
fn verify_tls13_signature(
&self,
message: &[u8],
cert: &rustls::pki_types::CertificateDer<'_>,
dss: &rustls::DigitallySignedStruct,
) -> std::result::Result<rustls::client::danger::HandshakeSignatureValid, rustls::Error> {
rustls::client::danger::ServerCertVerifier::verify_tls13_signature(
&self.0, message, cert, dss,
)
}
fn supported_verify_schemes(&self) -> Vec<rustls::SignatureScheme> {
rustls::client::danger::ServerCertVerifier::supported_verify_schemes(&self.0)
}
}$ cargo run
Error: invalid peer certificate: UnsupportedSignatureAlgorithmForPublicKeyContext { signature_algorithm_id: [6, 8, 42, 134, 72, 206, 61, 4, 3, 4], public_key_algorithm_id: [6, 7, 42, 134, 72, 206, 61, 2, 1, 6, 8, 42, 134, 72, 206, 61, 3, 1, 7] }(Additional Note) OpenSSL client (same server)
$ openssl version
OpenSSL 3.0.13 30 Jan 2024 (Library: OpenSSL 3.0.13 30 Jan 2024)
$ openssl s_client -connect myserver:4433
... // it's ok
Analysis
The server is choosing ecdsa_secp521r1_sha512 for the ServerKeyExchange signature. This is visible in the TLS handshake:
TLSv1.2 Record Layer: Handshake Protocol: Server Key Exchange
Content Type: Handshake (22)
Version: TLS 1.2 (0x0303)
Length: 149
Handshake Protocol: Server Key Exchange
Handshake Type: Server Key Exchange (12)
Length: 145
EC Diffie-Hellman Server Params
Curve Type: named_curve (0x03)
Named Curve: secp256r1 (0x0017)
Pubkey Length: 65
Pubkey: ...
Signature Algorithm: ecdsa_secp521r1_sha512 (0x0603)
Signature Hash Algorithm Hash: SHA512 (6)
Signature Hash Algorithm Signature: ECDSA (3)
Signature Length: 72
Signature: ...
While I believe rustls's behavior is correct in rejecting this mismatch, I'm opening this issue to track this behavior since OpenSSL clients accept it.
Feel free to close this issue and/or point me to related issue if any.
Questions
- Is this rejection the intended behavior for rustls?
- Is this a known interoperability issue with older OpenSSL servers?