This PR seems pretty mature. What still needs to be done to merge it?

This PR seems pretty mature. What still needs to be done to merge it?

Well, for one, all CI should pass. Then review should be requested. Finally, there's the question whether, how and where this should be added to some application layer to prove actual utility. Any input on that @SWilson4 ?

In terms of naming, I think "seed" is better than "coins" since that's the term used in FIPS 203. As for whether to call the function _internal or _derand or _deterministic: I don't think _internal is very descriptive, even though that's the FIPS 203 term; so I'd go with _derand or _deterministic, but don't have a strong preference between them.

Marking as "Ready for Review" now that CI passes.

This PR exposes the ML-KEM "internal" keygen function via a new OQS KEM API for derandomized key generation.

My main intent with this PR is to enable experimentation with post-quantum crypto in protocols that require the HPKE interface, which includes a "DeriveKeyPair" function that is essentially derandomized key generation (see RFC 9180 and draft-connolly-cfrg-hpke-mlkem). It turns out that this also allows users of liboqs to work with ML-KEM seeds (derandomized keygen converts a seed into an expanded keypair).

I've called the API "keypair_derand" since that was the name in the ML-KEM and Kyber codebases, but I'm open to changing this ("keypair_deterministic"? "keypair_derive"? "keypair_from_seed"?)

Currently, calls to the new API will fail for algorithms other than ML-KEM.

Well, for one, all CI should pass. Then review should be requested. Finally, there's the question whether, how and where this should be added to some application layer to prove actual utility. Any input on that @SWilson4 ?

@bifurcation Do you have any work on post-quantum MLS to point to here?

Currently, calls to the new API will fail for algorithms other than ML-KEM.

This statement I think is worth while documenting with higher visibility (README? Web site?): OQS by now clearly has pretty different levels of support for PQC algorithms also at the API level that I think users should be more prominently made aware of

• APIs only implemented for ML-DSA and ML-KEM
• A generally implemented API supported for all algorithms -- with some caveats:
  • other standardized algorithms are only included at a non-standard implementation level (Sphincs+/SLH-DSA, Falcon)
  • other standardized algorithms with some APIs disabled by default (XMSS, LMS)
  • "adjunct standard" algorithms (better wording welcome) like Frodo, Bike (?)
  • legacy algorithms like ntruprime, Kyber, Dilithium, HQC with limited update support
  • new standards-track proposals at different implementation levels like Cross, Mayo, hopefully UOV

The more I think about it, this seems worthy of a separate issue: This PR only contributes to an incoherence/deviation from the original OQS promise to support all PQC algorithms at equal measure that started earlier and goes deeper than just for this new API.

The more I think about it, this seems worthy of a separate issue: This PR only contributes to an incoherence/deviation from the original OQS promise to support all PQC algorithms at equal measure that started earlier and goes deeper than just for this new API.

I don't agree that OQS has such a promise. There are many examples in liboqs where different algorithms have had different support or treatment: e.g., how far they are through the standards process (selections/alternates), constant-time support, platform support, large-stack usage. We recently introduced the sign-with-context API because ML-DSA needed it. We shouldn't do this without thought, but I think it is quite reasonable that the more important algorithms (ML-KEM and ML-DSA) merit special treatment. I think it may be more true that consumers of liboqs (OQS Provider, the language wrappers) have taken more of the approach you mentioned.

We have an action item from the last TSC to document algorithm support, similar to PLATFORMS.md.

Well, for one, all CI should pass. Then review should be requested. Finally, there's the question whether, how and where this should be added to some application layer to prove actual utility. Any input on that @SWilson4 ?

@bifurcation Do you have any work on post-quantum MLS to point to here?

About a year ago, I did a prototype of XWing + MLS based on just copy/pasting the BoringSSL Kyber implementation into MLSpp. I'm eager to do a more proper version of that using libOQS (ideally using OpenSSL, but I can go direct as an intermediate step). This issue has been the major blocker on that.

I don't agree that OQS has such a promise.

I do recall very vividly hearing very regularly that "OQS does not pick winners" whenever I suggested dropping some support (e.g., testing, platform-specific optimizations, profiling support) for some algorithms: Did I dream this? This may, well, actually must, have changed in light of the apparent drop in community contributions towards OQS. But I don't think this (whether change or not) is goodness and a more impartial support for all possible PQC algorithms would be goodness for OQS to enable true apple-to-apples comparisons between PQC algs.

We have an action item from the last TSC to document algorithm support, similar to PLATFORMS.md.

True. Would be good to resolve. Maybe my list above could serve to deliver a top-line summary/starting point for that?

a prototype of XWing + MLS based on just copy/pasting the BoringSSL Kyber implementation into MLSpp.

That confuses me: According to its documentation, MLSpp is an

Implementation of the proposed Messaging Layer Security protocol in C++. Depends on C++17, STL for data structures, and OpenSSL or BoringSSL for crypto.

As both OpenSSL and BoringSSL support ML-KEM "natively" by now, what additional work is needed? What does liboqs add?

I don't agree that OQS has such a promise.

I do recall very vividly hearing very regularly that "OQS does not pick winners" whenever I suggested dropping some support (e.g., testing, platform-specific optimizations, profiling support) for some algorithms: Did I dream this? This may, well, actually must, have changed in light of the apparent drop in community contributions towards OQS. But I don't think this (whether change or not) is goodness and a more impartial support for all possible PQC algorithms would be goodness for OQS to enable true apple-to-apples comparisons between PQC algs.

Indeed we don't pick winners; that language is still in our README. But other people have picked winners, and I am in favour of giving the needs of those algorithms special consideration.

Thank you @SWilson4 for adding this useful API. The integration looks good to me.

I do have one question: Is there a specific reason why this API has been added only for KEMs and not for Signatures? I'm asking because there is also the use case of generating the key from a seed in the IETF draft for ML-DSA signatures.

The original PR was only to support HPKE / MLS, and was made before the ML-KEM final standard was released (and all the discussion about seeds started in earnest). I imagine it would be straightforward to extend it to signatures (albeit only to ML-DSA for now) as well, if people would use the functionality. Does the use case from the IETF draft necessitate going through oqs-provider?

Thank you @SWilson4 for adding this useful API. The integration looks good to me.
I do have one question: Is there a specific reason why this API has been added only for KEMs and not for Signatures? I'm asking because there is also the use case of generating the key from a seed in the IETF draft for ML-DSA signatures.

The original PR was only to support HPKE / MLS, and was made before the ML-KEM final standard was released (and all the discussion about seeds started in earnest). I imagine it would be straightforward to extend it to signatures (albeit only to ML-DSA for now) as well, if people would use the functionality. Does the use case from the IETF draft necessitate going through oqs-provider?

@SWilson4 thank you for adding this API. Regarding signatures I want to mention the use case from Post-Quantum PGP RFC where both ML-KEM and ML-DSA share same serialization format

The original PR was only to support HPKE / MLS, and was made before the ML-KEM final standard was released (and all the discussion about seeds started in earnest). I imagine it would be straightforward to extend it to signatures (albeit only to ML-DSA for now) as well, if people would use the functionality. Does the use case from the IETF draft necessitate going through oqs-provider?

I believe the key encoding/decoding functionality would be implemented in the oqs-provider, while liboqs would still need to provide an API for (re-)generating the keys from the seed. However, I think it's fine to proceed with merging this PR as is focusing only on the KEM API.

I've done a rebase to clean up the git history a little and run the extended tests. Once those pass, I will hit the "merge" button.

Regarding naming of the new API: in the absence of a strong consensus, I prefer "derand" as that has been in use by ML-KEM / Kyber implementations for some time and matches the naming conventions of our upstream sources.