NSS >= 3.27 cannot be used

When using libnss >= 3.27 loading the PKCS11 token fails with CKR_ATTRIBUTE_READ_ONLY errors. There is an open BUG JDK-8180837 that affects all Java versions and the RCA is not yet complete, nor is there an indication of when this will be resolved.

TLS1.2 cannot be used with PKCS11-NSS

See unresolved jdk bug

Loading system keystore/truststore from a PKCS11 token

PKCS#11 tokens can be used as keystores and truststores. To use a PKCS#11 token as the JDK Default keystore and truststore ( a.k.a. javax.net.ssl.keyStore and javax.net.ssk.TrustStore ), one needs to set jaxax.net.ssl.keyStore and javax.net.ssl.trustStore to NONE (case sensitive) according to the PKCS#11 Reference Guide.

We do not handle the NONE parameter correctly, and this gets interpreted as the name of the keystore and truststore, relevant to $ES_CONF_PATH.

Issue addressed in a PR shortly (See : #33460)

Multiple PKCS#11 keystores

We can't have multiple PKCS#11 keystores. PKCS#11 is supposed to be a wrapper to access HW/SW tokens and the token is configured in the provider's configuration that is referenced in java.security. As such, each JVM can only point to one token, i.e. one Keystore/Trustore. PKCS#11 KeyStores/TrustStores cannot be stored on disk and when loading the KeyStore via KeyStore.load() we need to pass null as the InputStream in order to access the token configured in the security properties.

In practice this means that keystores cannot be used with the NSS FIPS provider and only the System Truststore(PKCS#11 token) can be used, adding trusted certificates to it, as needed. This is NOT an option for trust configuration for TLS on the transport layer as it means that all certificates signed by the JVM trusted CAs will be able to connect to an Elasticsearch node.
It can't either be used for storing key material, unless we introduce the option to define the key alias to be selected from a keystore when a keystore is configured, instead of assuming there will be only one key as we do now. (This might be a nice todo irrespective to the FIPS effort)

This is not an issue per se, just a configuration limitation.

PKCS#11 tokens as password protected keystores

PKCS#11 tokens need to be password protected (NOTE: NSS tooling by default doesn't add a password to the db), otherwise loading the store fails with

javax.security.auth.login.LoginException: no password provided, and no callback handler available for retrieving password

PKCS#11 keystores cannot be loaded from disk and in effect one can only use a single store so this only applies to the default PKCS#11 token (accessed as the system default keystore) that needs to be password protected. The password for it is passed using javax.net.ssl.keyStorePassword and javax.net.ssl.TrustStorePassword that cannot be empty.
Again, a limitation rather than an issue.

Another way this affects us is that if no trust configuration is defined, we resolve to using the default JDK Trust Configuration and we do that by initializing an TrustManager with a null Keystore and no password. We need to detect that we're using a PKCS11-NSS Provider and that the default truststore is acually a PKCS#11 token and pass the javax.net.ssl.TrustStorePassword when loading the null Keystore.

Issue addressed in a PR shortly

In memory keystores

When using PEM files for key and trust material, we create in memory KeyStores (of applicable type - in this case PKCS#11) and load these keys and certificates so that we can then initialize TrustManager and KeyManager objects using the keystores. We pass null as the InputStream parameter to indicate that it should be initialized as an empty keystore, but in the case of PKCS#11 this will mean that the default PKCS#11 token will be loaded in the KeyStore. This, in turn, adds a number of complications:

• Adding a certificate using KeyStore#setCertificateEntry() doesn't work as the CKA_ID attribute which is required for the PKCS#11 format is not set, and this fails with an NPE. We could code around this if needed.
• The TrustManager that will be created, will be trusting all certificates that are in the default token, plus the PEM one we add, instead of just the one we add, which would be the intended behavior.
• The KeyManager will be created containing all certificate entries plus the private key entry we add. This is not problematic, unless we need to load many private key entries (one for transport TLS, one for http TLS, etc. ), when the problem becomes how to select the correct entry, which can be solved with specifying aliases in the configuration as mentioned above.

PEM files cannot be used as is.

PKCS#12 and FIPS 140 compliance

Looking for possible solutions for the above, I revisited why PKCS#12 keystores are not FIPS 140-2 compliant and came to the conclusion that a PKCS#12 keystore can be FIPS 140-2 compliant.

To give some context, the original idea is that a PKCS#12 is not FIPS 140-2 compliant
(for example BCFIPS disallows the use of PKCS12 stores in fips-approved mode) because the algorithms required for PBE key generation (for the key that is generated from the store's password and is used to encrypt the actual keys that are stored in the keystore) are not FIPS 140-2 compliant. However, reading through the RFC, PKCS#12 defines that PKCS#5 should be used and in particular

Specifically, PBES2 should be used as encryption scheme, with PBKDF2 as the key derivation function.

Now, PBES2 combines a password-based key derivation function (PBKDF2 is called out explicitly) with an underlying encryption scheme. Appendix B2 of RFC2898 defines DES-EDE3-CBC-Pad ( three-key triple-DES in CBC mode with the RFC 1423 padding operation) as one of the possible schemes of encryption. Triple-DES is a NIST Approved algorithm and as such FIPS 140-2 compliant. Additionally, common tooling (i.e. openssl) allows for creating PKCS#12 keystores with arbitrary (even not specified in PKCS#12 RFC) (FIPS 140-2 compliant) algorithms and encryption schemes.
For example

openssl pkcs12 -inkey key.pem -in certificate.pem -macalg SHA256 -keypbe AES-192-CBC -certpbe AES-256-CFB -export -out test.p12

will produce a PKCS#12 store with the following information

openssl pkcs12 -info -in test.p12 -noout

MAC:sha256 Iteration 2048
PKCS7 Encrypted data: PBES2, PBKDF2, AES-256-CFB, Iteration 2048, PRF hmacWithSHA256
Certificate bag
PKCS7 Data
Shrouded Keybag: PBES2, PBKDF2, AES-192-CBC, Iteration 2048, PRF hmacWithSHA256

which for all effects and purposes is FIPS 140-2 compliant since it only uses approved algorithms (AES for encryption and PBKDF2 as the key derivation function.

To remain compliant to PKCS#12 and PKCS#5 2.0, one could create a keystore with

openssl pkcs12 -inkey key.pem -in certificate.pem -macalg SHA256 -keypbe DES-EDE3-CBC -certpbe DES-EDE3-CBC -export -out test.p12

Granted, existing PKCS12 stores are not FIPS 140 compliant since the default algorithms schemes are pbeWithSHA1And3-KeyTripleDES-CBC for the inner (private key encryption) and pbeWithSHA1And40BitRC2-CBC for the outer (PKCS#7) encryption.

The problem with the above is that the KeyManagerImpl specifically requires that the keystore used to initialize a KeyManager, must be created by the same Security Provider that is actually being used

https://github.com/frohoff/jdk8u-jdk/blob/da0da73ab82ed714dc5be94acd2f0d00fbdfe2e9/src/share/classes/sun/security/ssl/KeyManagerFactoryImpl.java#L65

Unfortunately PKCS#12 stores created by openssl do not have a provider set and if they did, it wouldn't be the SunPKCS-NSS that will be in use in runtime, and using the SunPCKS11-NSS provider with keytool , i.e.

/usr/lib/jvm/java-8-openjdk-amd64-fips-nss/bin/keytool -genkeypair -keystore instance.p12 -storetype pkcs12 -storepass password -alias cert0 -keyalg RSA -keysize 2048 -validity 99999 -dname "CN=My SSL Certificate" -providerName SunPKCS11-NSS

yields an error as this provider can't handle PKCS12 keystores.

Summary

• NONE keyword needs to be handled accordingly ( Correctly handle PKCS#11 tokens for system keystore #33460)
• PKCS#12 keystores , JKS keystores and PEM files cannot be used for key configuration.
• PKCS#12 keystores can be used for trust configuration and as shown above can be FIPS 140 compliant.
• The HW/SW PKCS#11 token that is used as the system keystore and truststore must be password protected.

Actions

• Focus on supporting the use of PEM files for key configuration. This would require specific handling for adding keys and certificates to the PKCS#11 token and to allow to specify in configuration the alias that should be used to read a key from a PKCS#11 token that might contain multiple private key entries (edited see below for justification.)
• Allow for selecting a Security Provider to use ( and in PKCS#11 case, the associated keystore/token ) when configuring a keystore or key/certificate pair in any related configuration option.