Per-key encoding formats for ML-KEM and ML-DSA by vdukhovni · Pull Request #29206 · openssl/openssl

vdukhovni · 2025-11-24T15:45:49Z

We support selection of ML-KEM and ML-DSA key formats on input and output at the provider level, these are essentially global defaults, in effect for the lifetime of the process.

Unfortunately, the JAVA interface in openssl-jostle needs to be able to output a specific key in seed-only form. To that end, this PR
introduces a new "output-formats" PKEY encoding parameter, that can be used with OSSL_ENCODER_CTX_set_params(3) when encoding a key to PKCS#8, after using OSSL_ENCODER_CTX_new_for_key(3), rather than i2d_PrivateKey(3), i2d_PKCS8PrivateKey(3) or PEM equivalents.

Problem originally reported in: openssl-projects/openssl-jostle#11 (comment)
Once this is merged, corresponding code changes will be needed in Jostle to ensure that seed-only key construction sets the new parameter when needed.

Checklist

documentation is added or updated
tests are added or updated

paulidale · 2025-11-25T00:14:31Z

Just wondering if getting this via the key manager is more prudent than adding a custom encode format?

vdukhovni · 2025-11-25T00:58:04Z

Just wondering if getting this via the key manager is more prudent than adding a custom encode format?

Thanks for taking a look, I don't know what you mean by "getting this via the key manager".
If you mean having a new key type that is structurally the same but encodes differently, I think that a parameter is a more lightweight and natural way to do that. In any case, when loading a PKCS#8 object the OID will still map to the current ML-DSA key type that serialises as both the seed and the expanded key, so a new key type would not help. It could be an option for keygen or import, but it is unclear why that's better (the key is an ML-DSA key, and introducing a variant name breaks "isa" tests, ...). But perhaps you have something else in mind...

By way of background, the JAVA mechanism in Bouncy Castle for getting a seed-only form of a key is to create a second key from just the seed of the first, and then (in the native non-jostle implementation) the key remembers that it was imported from a seed and is output in that format automatically.

This PR makes it possible for openssl-jostle to arrange for the same behaviour when importing into OpenSSL.
And in any case, per-key control of the output format seems useful anyway. It could make sense to add a corresponding feature to the encoders (OSSL_ENCODER_CTX_new_for_pkey()), which is not what is needed in openssl-jostle, but perhaps some users might want to choose the format "just in time", at the cost needing to use a lower-layer API than i2d_PrivateKey(3) or i2d_PKCS8PrivateKey(3).

paulidale · 2025-11-25T01:25:53Z

My thought won't work because the required function is not public. The idea was to get the key manager from the PKEY and then call evp_keymgmt_export to get the seed.

Making enough public would be an alternative to this. The import/export provider side APIs are part of the public API contract.

vdukhovni · 2025-11-25T05:16:04Z

My thought won't work because the required function is not public. The idea was to get the key manager from the PKEY and then call evp_keymgmt_export to get the seed.

Making enough public would be an alternative to this. The import/export provider side APIs are part of the public API contract.

The problem (for jostle, and other potential applications) isn't lack of a mechanism to just get the seed, that's available via EVP_PKEY_todata(3), rather it is that making an encoded PKCS#8 with just the seed currently is only possible via global configuration (provider layer parameters). Applications that want at some point in time to configure a selected set of keys for seed-only export need to be able to propagate that choice to the encoder layer.

This PR providers two ways to do that, and I'm about to add a commit that completes the story by adding a third.

Early configuration as part of decoding, provided OSSL_DECODER_CTX_new_for_pkey(3) is something the application delveoper is willing to use instead of d2i_PrivateKey(3) or equivalent.
- This is tested to work for both "PEM" and "DER" input forms.
Eariy configuration as part of key generation or import, provided the OSSL_PKEY_PARAM_SEED_ONLY parameter is set on the EVP_PKEY_CTX used for that purpose.
Just-in-time configuration as part of encoding, provided OSSL_ENCODER_CTX_new_for_pkey(3) is something the application develop is willing to use instead of i2d_PrivateKey(3) or equivalent.
- This is tested to work for both the DER and PEM output forms.

t-j-h

Thinking on the whole concept here, what if someone wants expanded-only - will we add that here too as a separate flag?

Should this be expanded to cover the general concepts more than just one specific choice here - although I think that choice is the pressing one and others could be delayed until some expresses a need for it.

I'm happy with the code and the names - I would just tweak the accept undefined values behaviour and make things fail if we cannot actually produce seed-only output.

t-j-h · 2025-11-25T05:48:43Z


+=item "seed-only" (B<OSSL_PKEY_PARAM_SEED_ONLY>) <unsigned int>
+
+When this parameter is set, its value is ignored, and the generated key is


As a general style approach, I would not have an ignored value here. I would require the user to provide a fixed value (e.g. 1) simply to be consistent and allow for any future possible choices where we may want to expand the values for this parameter.

If users are free to put whatever they like then we can never use the value.
Fixing it to 1 allows for possible future changes - which we may never make.

Yes, I would treat it as boolean though. Ignored value is ugly and we do not use this concept elsewhere (AFAIK).

A boolean setting makes little sense though, because just not setting the parameter is the correct/simplest way to not force seed-only. Setting it to zero meaning "do nothing" makes no sense.

If we want to support more values, they should have some non-default effect. For example:

Do not output the seed, giving an expanded key-only output.

As in this PR, output only the seed.

Output both seed and expanded key

All three cases override the provparam output-format, if different.

Basically, any setting here needs to mean something concrete, rather than just be a NOP.

I don't see a problem with allowing the default value to be an input.
It comes down to a preference about how user code is written and we in general do not enforce this restriction anywhere else.

There is not much difference between

*p++ = OSSL_PARAM_construct_int("foo", &val); AND if (val) *p++ = OSSL_PARAM_construct_int("foo", &val);

t-j-h · 2025-11-25T05:50:03Z

 When the seed was not initially known, or was not retained, B<PKCS#8> private
 key files will contain only the private key in FIPS 204 C<sk> format.

+=item "seed-only" (B<OSSL_PKEY_PARAM_SEED_ONLY>) <unsigned int>


Seeing this as "seed-only" is fine - but if we have just an expanded-only input then I would expect things to fail of someone sets this parameter - given the name.

The code is "seed-only-if-you-have-it" isn't it?

I would make things fail if there is a request for seed-only and it cannot be provided.

Or, we can change the name, to "preserve-seed-only" or "seed-only-preferred", ...
The code is more "preserve what got on input" if the input is seed only. Of course that's the only possible outcome with expanded-key-only input, and with "both" we get both by default (unless provparams are set to prefer another output form).

So there are some possible design choices here.

Implement more faithful "seed-only" semantics keeping the name, but ignore presence or absence of expanded key and fail if the seed is missing. Java would have to use the setting with care.

Implement more faithful "as-is" semantics, changing the name, and preserving also the "both" input form on import and decode, but then likely keygen ends up always producing "both", and one needs to export and reimport in seed-only form to get seed-only output from a freshly generated key.

Implement a new option to store an ordered list of output formats along with the key (mirroring the global provparams)

And of course anyone willing to store the preferred output form along with the key can always use the OSSL_ENCODER_CTX_new_from_pkey(3) API to get seed-only even when both the seed and expanded key came in.

Likely some variations on the theme. Anyway, for now I'll adopt the forced value, but if anyone has a clear vision for what users would find most useful/least surprising here, please share. All the relevant code is in place, and only small tweaks are needed to adjust naming and/or semantics.

I wouldn't argue if you'd want to support both semantics, i.e. both "seed-only" (implying a hard fail if seed isn't available) and "prefer-seed-only" (which falls back to whatever is available if the seed isn't).

I would not be opposed to levitte's suggestion.
It could be an single param with multiple settings of 0 = default, 1 = seed_only, 2 = prefer_seed_only.

Sashan

all my comments are nits. I could not spot nothing wrong with your PR..
I'm just pointing them out so you can decide if it is worth to address them or not.

thanks

t-j-h

Still approved

Sashan

looks good to me. thanks.

vdukhovni · 2025-12-03T05:11:17Z

Tests and doc nit updated, based on review from @bob-beck (thanks), code remains the same.

We support selection of ML-KEM and ML-DSA key formats on input and output at the provider level, these are essentially global defaults, in effect for the lifetime of the process. Unfortunately, the JAVA interface in openssl-jostle needs to be able to output a specific key in seed-only form. To that end, this PR introduces a new "output-formats" PKEY encoding parameter, that can be used with OSSL_ENCODER_CTX_set_params(3) when encoding a key to PKCS#8, after using OSSL_ENCODER_CTX_new_for_key(3), rather than i2d_PrivateKey(3), i2d_PKCS8PrivateKey(3) or PEM equivalents.

In was premature to make OSSL_(EN|DE)CODER_CTX_[sg]et_finalized() be public interfaces. Forunately, these have not yet appeared outside the "master" branch, so we can still retract them. Also, in the case of decoders, the implementation failed to take into account that the context was duplicated before it was returned to the user, and the duplicated copy failed to copy the "finalized" field. This commit also renames "finalized" to "frozen", because finalisation is a misleading term in this context, it suggests resource reclamation during garbage collection or deallocation, not marking a structure partly immutable.

bob-beck

Hooray now with test coverage.

LGTM.

esyr · 2025-12-02T14:28:40Z

@@ -65,12 +65,6 @@ OpenSSL 4.0



Would be nice if the commit message stated that it partially reverts commit f480a73.

esyr · 2025-12-04T11:22:43Z

 #define OSSL_KEM_PARAM_OPERATION_DHKEM      "DHKEM"

-/* Provider configuration variables */
-#define OSSL_PKEY_RETAIN_SEED   "pkey_retain_seed"


Backport note: this macro definition has been added in commit openssl-3.5.0-alpha1~523 and the change shouldn't be backported to 3.x branches (provided such backport will be happening).

This won't be backported unless given an exception which is pretty unlikely.

openssl-machine · 2025-12-04T23:00:36Z

This pull request is ready to merge

paulidale · 2025-12-04T23:08:54Z

Merged, thanks for the contribution.

vdukhovni requested review from levitte, slontis, t-j-h and t8m November 24, 2025 15:45

vdukhovni force-pushed the seed-only-jostle branch from c1cbd8b to 68c5a4d Compare November 24, 2025 15:47

github-actions Bot added the severity: fips change The pull request changes FIPS provider sources label Nov 24, 2025

loci-dev mentioned this pull request Nov 24, 2025

UPSTREAM PR #29206: Per-key seed-only option for ML-KEM and ML-DSA auroralabs-loci/openssl#121

Open

2 tasks

vdukhovni force-pushed the seed-only-jostle branch 3 times, most recently from 6fd48b0 to b9c748c Compare November 25, 2025 00:10

vdukhovni force-pushed the seed-only-jostle branch 2 times, most recently from 06a08d5 to 0090538 Compare November 25, 2025 05:25

t-j-h reviewed Nov 25, 2025

View reviewed changes

t8m added branch: master Applies to master branch approval: review pending This pull request needs review by a committer triaged: feature The issue/pr requests/adds a feature tests: present The PR has suitable tests present labels Nov 25, 2025

Sashan reviewed Nov 25, 2025

View reviewed changes

Comment thread crypto/ml_kem/ml_kem.c Outdated

Comment thread providers/implementations/encode_decode/ml_dsa_codecs.c Outdated

Comment thread providers/implementations/encode_decode/ml_kem_codecs.c Outdated

Comment thread providers/implementations/keymgmt/ml_dsa_kmgmt.c Outdated

vdukhovni force-pushed the seed-only-jostle branch 3 times, most recently from a859539 to 3179d81 Compare November 25, 2025 17:45

t-j-h previously approved these changes Nov 25, 2025

View reviewed changes

vdukhovni requested a review from Sashan November 26, 2025 07:47

Sashan previously approved these changes Nov 26, 2025

View reviewed changes

Sashan added approval: done This pull request has the required number of approvals and removed approval: review pending This pull request needs review by a committer labels Nov 26, 2025

paulidale added the approval: review pending This pull request needs review by a committer label Dec 2, 2025

vdukhovni requested review from bob-beck and t-j-h December 2, 2025 08:23

vavroch2010 assigned vdukhovni Dec 2, 2025

vavroch2010 added this to Development Board Dec 2, 2025

vavroch2010 moved this to Waiting Review in Development Board Dec 2, 2025

bob-beck suggested changes Dec 2, 2025

View reviewed changes

Comment thread doc/man7/EVP_PKEY-ML-KEM.pod Outdated

Comment thread test/evp_extra_test.c Outdated

vdukhovni force-pushed the seed-only-jostle branch from 8363fad to 61b34b7 Compare December 3, 2025 05:06

vavroch2010 requested a review from jogme December 3, 2025 08:59

t8m requested changes Dec 3, 2025

View reviewed changes

Comment thread doc/man3/OSSL_DECODER_CTX.pod Outdated

Comment thread doc/man3/OSSL_ENCODER_CTX.pod Outdated

Comment thread doc/man7/EVP_PKEY-ML-DSA.pod Outdated

vdukhovni force-pushed the seed-only-jostle branch 3 times, most recently from c448cf3 to 0dd8f39 Compare December 3, 2025 11:28

Viktor Dukhovni added 2 commits December 3, 2025 22:33

vdukhovni force-pushed the seed-only-jostle branch from 0dd8f39 to a1d8c63 Compare December 3, 2025 11:33

t8m approved these changes Dec 3, 2025

View reviewed changes

bob-beck approved these changes Dec 3, 2025

View reviewed changes

nhorman approved these changes Dec 3, 2025

View reviewed changes

github-project-automation Bot moved this from Waiting Review to Waiting Merge in Development Board Dec 3, 2025

nhorman added approval: done This pull request has the required number of approvals and removed approval: review pending This pull request needs review by a committer labels Dec 3, 2025

esyr approved these changes Dec 4, 2025

View reviewed changes

vdukhovni mentioned this pull request Dec 5, 2025

Trying to get Ml-DSA PKCS#8 Seed encoding after Keypair generation openssl-projects/openssl-jostle#11

Closed

davidben mentioned this pull request Mar 11, 2026

Supported PKCS8 import formats for ML-KEM and ML-DSA WICG/webcrypto-modern-algos#29

Closed

panva mentioned this pull request Mar 12, 2026

crypto: default ML-KEM and ML-DSA pkcs8 export to seed-only format nodejs/node#62178

Merged

esyr mentioned this pull request Apr 14, 2026

[4.0] CHANGES.md: updates for 4.0.0 final release #30817

Closed


		=item "seed-only" (B<OSSL_PKEY_PARAM_SEED_ONLY>) <unsigned int>

		When this parameter is set, its value is ignored, and the generated key is

Uh oh!

Conversation

vdukhovni commented Nov 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Checklist

Uh oh!

paulidale commented Nov 25, 2025

Uh oh!

vdukhovni commented Nov 25, 2025

Uh oh!

paulidale commented Nov 25, 2025

Uh oh!

vdukhovni commented Nov 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

t-j-h left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Sashan left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

t-j-h left a comment

Choose a reason for hiding this comment

Uh oh!

Sashan left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

vdukhovni commented Dec 3, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

bob-beck left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

openssl-machine commented Dec 4, 2025

Uh oh!

paulidale commented Dec 4, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

12 participants

vdukhovni commented Nov 24, 2025 •

edited

Loading

vdukhovni commented Nov 25, 2025 •

edited

Loading