Skip to content

Replumbing: make it possible for providers to specify multiple names#8985

Closed
levitte wants to merge 12 commits intoopenssl:masterfrom
levitte:provider-multiple-names
Closed

Replumbing: make it possible for providers to specify multiple names#8985
levitte wants to merge 12 commits intoopenssl:masterfrom
levitte:provider-multiple-names

Conversation

@levitte
Copy link
Member

@levitte levitte commented May 23, 2019

This modifies OSSL_ALGORITHM to support a NULL-terminated list of
names instead of just one name.

Related to #8967


NOTE: This includes #9969, please review that one first

@levitte levitte added the branch: master Applies to master branch label May 23, 2019
@levitte levitte force-pushed the provider-multiple-names branch 2 times, most recently from 5ae34c4 to 9f515df Compare May 24, 2019 13:10
@levitte levitte force-pushed the provider-multiple-names branch from 9f515df to 1ca315f Compare June 4, 2019 09:09
@levitte levitte force-pushed the provider-multiple-names branch from 1ca315f to 76fa75d Compare June 4, 2019 10:43
@levitte levitte mentioned this pull request Aug 1, 2019
2 tasks
@levitte levitte mentioned this pull request Aug 27, 2019
2 tasks
@levitte levitte force-pushed the provider-multiple-names branch from f72c63f to 34d212d Compare September 12, 2019 12:15
@levitte
Copy link
Member Author

levitte commented Sep 12, 2019

Rebased, but needs more work, since provider code has moved around a lot

@levitte levitte force-pushed the provider-multiple-names branch from 34d212d to 7c8ff00 Compare September 12, 2019 15:21
@paulidale
Copy link
Contributor

@slontis and I discussed this recently. I'm not sure this is a great path to follow -- making all names into arrays of names seems like it is needlessly complex. Using a single name should be the norm.

We were thinking along the lines of using the object database so that the core can populate aliases automatically (i.e. if the name matches either SN_XXX or LN_XXX, add both). The core needs the object database regardless, so there is no waste there and it allows providers to not have to worry with aliasing.

Well almost, there also needs to be a way for a provider to add aliases -- I'm thinking an explicit function that adds an alias (or aliases) for an already defined name.

@levitte
Copy link
Member Author

levitte commented Sep 13, 2019

Contrary to you, I claim that the new core has zero need for the old method object database. The namemap replaces it.

Something I also realised is also that we shouldn't copy the name string when creating new method instances, but rather the namemap id. That means that we can easily support multiple names there without having to duplicate a whole darn array.

@levitte
Copy link
Member Author

levitte commented Sep 13, 2019

Regarding pre-populating the namemap, see #8984

@paulidale
Copy link
Contributor

OIDs still depend on the object database....

@levitte
Copy link
Member Author

levitte commented Sep 13, 2019

Yeah, but the new core doesn't use OIDs, at all. It just uses names.
We will still need to do OID to name translation and therefore have a OID<->name db, but that's a separate thing. (and sure, we can add OID support, and probably should, but we don't have that right now)

@paulidale
Copy link
Contributor

What about restricting each algorithm to a single name and having the core know about some aliases for backwards compatibility? It isn't like we need to have multiple names for things, they are handles to access algorithms as far as the core is concerned.

This would let us clean up some of the mess (e.g. id-XXX algorithms).

If a provider really does want the same thing registered under two names, it can register it twice. It will get different IDs but they're not publicly visible or terribly meaningful (unlike NIDs).

@slontis
Copy link
Member

slontis commented Sep 14, 2019

What about restricting each algorithm to a single name and having the core know about some aliases for backwards compatibility? It isn't like we need to have multiple names for things, they are handles to access algorithms as far as the core is concerned.

Just about everything is registered with 2 names currently in the old system (SN and LN)

@paulidale
Copy link
Contributor

The two names are often not distinct when looked up case insensitively.

@levitte
Copy link
Member Author

levitte commented Sep 14, 2019

What about restricting each algorithm to a single name and having the core know about some aliases for backwards compatibility? It isn't like we need to have multiple names for things, they are handles to access algorithms as far as the core is concerned.

That's essentially what #8984 does (why not have a look?)

@paulidale
Copy link
Contributor

I was more thinking of having the core's EVP layer handle the mapping and not duplicating names all over the place in different name maps. The effect is similar though.

I think we can rely on providers to use the canonical name for each algorithm but not application code. The former is new and we can define the rules, the latter is legacy and the rules escaped long ago.

@levitte
Copy link
Member Author

levitte commented Sep 14, 2019

I was more thinking of having the core's EVP layer handle the mapping and not duplicating names all over the place in different name maps. The effect is similar though.

Ah, so actually exactly the same, just different spot...

I think we can rely on providers to use the canonical name for each algorithm but not application code.

... I think we need a zoomed out (i.e. not just technical) discussion on this.
What's the "canonical" name? Who's the authority that decides this?

Just to take an example that we have in our code, scrypt. According to the ASN.1 module in RFC 7914, it goes by "id-scrypt":

   id-scrypt OBJECT IDENTIFIER ::= {1 3 6 1 4 1 11591 4 11}

We would like to call it just "SCRYPT", because that conforms better with the rest of the names we use, right?
So what would be the "canonical" name, "id-scrypt" or "SCRYPT"? On whose authority? Yours? Mine? RFC 7914's?

So, as a way to avoid that kind of dispute, why can't we simply think in terms of a set of names, all equal?

Back to the more technical side of things:

  • the new namemap already supports a set of names without making any of them "canonical". They simply get the same identity number.

A more personal note:

  • I loathe the notion of having an implementation in one corner of the code and the names it goes by in another corner. Having them together in code, like this PR does, avoids having to find the implementation in one spot and then having to go look elsewhere to find the name(s) it goes by. (Yes, I know that's what we currently do... :-/)

@levitte
Copy link
Member Author

levitte commented Sep 14, 2019

I was more thinking of having the core's EVP layer handle the mapping and not duplicating names all over the place in different name maps. The effect is similar though.

Ah, so actually exactly the same, just different spot...

Oh, wait, I forget... #8984 does exactly that!

@levitte
Copy link
Member Author

levitte commented Sep 20, 2019

Rebased and cleaned up to match the changes that have been made re name vs name_id.

This almost tests right... only test_sslapi acts up, still.

@levitte
Copy link
Member Author

levitte commented Sep 20, 2019

Speaking of this, I realised there's really no reason why this PR and #8984 should be alternatives for each other... they should actually combine perfectly well.

@levitte
Copy link
Member Author

levitte commented Sep 20, 2019

Ah, figured the last failure out... I had mixed up certain names with each other.
(some ciphers with different key sizes are in 256, 192, 128 order while others are in 128, 192, 256 order... when cutting and pasting a bit and getting in a flow, it's no wonder I didn't get it right in some places...)

@levitte levitte changed the title [WIP] Replumbing: make it possible for providers to specify multiple names Replumbing: make it possible for providers to specify multiple names Sep 21, 2019
@levitte
Copy link
Member Author

levitte commented Sep 21, 2019

This is, btw, no longer WIP

@levitte levitte force-pushed the provider-multiple-names branch from 3b47576 to debe358 Compare September 30, 2019 15:28
@levitte
Copy link
Member Author

levitte commented Sep 30, 2019

Rebased the lot too

@paulidale
Copy link
Contributor

paulidale commented Oct 1, 2019

@mattcaswell, compatible naming is main reason I wanted the core to add the aliases it knows about. #8984 will achieve this. I wanted to write consistent naming, but it isn't currently.

As for providers listing several names for an algorithm, I see this as unnecessary if the core knows them. One should be enough. This has been a sticking point between me and @levitte 😀

We have to trust that the providers will do the right thing (whatever that is). A typo in a provider's name list for an algorithm could cause problems like the one suggested -- there is no nice mitigation. The best I can think of is to: insist that the names must either be all unknown or all known and referring to the same algorithm. i.e. no additional new names are permitted for algorithms the core knows about.

[edit: bolded the suggestion, I woudn't mind seeing this implemented]

@levitte
Copy link
Member Author

levitte commented Oct 1, 2019

I think you're making a problem where there isn't one, or rather, you're making it our problem when it's really somebody else's. And most of all, it's a social problem, not a technical one.
The basic message here is "know your providers" (for anyone who wants to use them), but also "know the expectations" (for the provider writers).

To put it quite plainly, this has two sides:

  1. if someone writes an application, or configures for a provider that doesn't provide algorithms with the names they expect, they should investigate what the configured providers does use. This is a use case for the openssl provider tool and why that tool is important.
  2. if someone writes a provider and wants to implement some well known algorithms, it would be wise of them to use at least one or two well known names, or face a reality that their provider will probably not grow very popular.

Something that I don't want to see is us becoming some kind of registry of all names possible, including for whatever upcoming algorithms. I believe that could become an unnecessary hurdle. I know that's not quite what you're proposing, @paulidale, but that's a possible consequence that we have already seen (we've become the central registry of NIDs and associated names).

@levitte
Copy link
Member Author

levitte commented Oct 1, 2019

The configuration problem that's been mentioned is just same as configuring for using the FIPS module only, and having an application that tries to fetch, say, the "sm3" hash algo. That will fail because it's not available. The FAQ should be "Did you check that this providers provides that name?"

@paulidale
Copy link
Contributor

I'm not worried about malicious providers -- they lie beyond our control. I'm worried about a provider that, for example, registers CAMELIA-256-CBC:CAMELLIA256 and the bad name then being saved in a configuration file.

I guess the core of the message is know your providers as @levitte wrote.

My suggestion was probably a bit extreme after all.

@levitte
Copy link
Member Author

levitte commented Oct 2, 2019

BTW, if we're inventing a naming standard for our names, maybe we should document it in some sane place other than a comment in the code. Please suggest.

@paulidale
Copy link
Contributor

How about: doc/internal/man7/the_naming_of_cats.pod :)

I agree it should be documented, either as a README or a POD file. I lean towards the latter.

{ "SHA2-256:SHA-256:SHA256", "default=yes", sha256_functions },
{ "SHA2-384:SHA-384:SHA384", "default=yes", sha384_functions },
{ "SHA2-512:SHA-512:SHA512", "default=yes", sha512_functions },
{ "SHA2-512/224:SHA-512/224:SHA512-224", "default=yes",
Copy link
Member

@slontis slontis Oct 3, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is everyone happy with the / here?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am OK with it. What would be the alternative?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Its probably better than '_'
and '-' would probably be confusing.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NIST uses / to separate the 512 and 224. I wonder about the final name SHA512-224, I assume this is historic (which would be my fault I suspect).

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's not very hard to check what names we have used...

: ; openssl version
OpenSSL 1.1.1d  10 Sep 2019
: ; openssl list -digest-algorithms
RSA-MD4 => MD4
RSA-MD5 => MD5
RSA-RIPEMD160 => RIPEMD160
RSA-SHA1 => SHA1
RSA-SHA1-2 => RSA-SHA1
RSA-SHA224 => SHA224
RSA-SHA256 => SHA256
RSA-SHA3-224 => SHA3-224
RSA-SHA3-256 => SHA3-256
RSA-SHA3-384 => SHA3-384
RSA-SHA3-512 => SHA3-512
RSA-SHA384 => SHA384
RSA-SHA512 => SHA512
RSA-SHA512/224 => SHA512-224
RSA-SHA512/256 => SHA512-256
RSA-SM3 => SM3
BLAKE2b512
BLAKE2s256
id-rsassa-pkcs1-v1_5-with-sha3-224 => SHA3-224
id-rsassa-pkcs1-v1_5-with-sha3-256 => SHA3-256
id-rsassa-pkcs1-v1_5-with-sha3-384 => SHA3-384
id-rsassa-pkcs1-v1_5-with-sha3-512 => SHA3-512
MD4
md4WithRSAEncryption => MD4
MD5
MD5-SHA1
md5WithRSAEncryption => MD5
ripemd => RIPEMD160
RIPEMD160
ripemd160WithRSA => RIPEMD160
rmd160 => RIPEMD160
SHA1
sha1WithRSAEncryption => SHA1
SHA224
sha224WithRSAEncryption => SHA224
SHA256
sha256WithRSAEncryption => SHA256
SHA3-224
SHA3-256
SHA3-384
SHA3-512
SHA384
sha384WithRSAEncryption => SHA384
SHA512
SHA512-224
sha512-224WithRSAEncryption => SHA512-224
SHA512-256
sha512-256WithRSAEncryption => SHA512-256
sha512WithRSAEncryption => SHA512
SHAKE128
SHAKE256
SM3
sm3WithRSAEncryption => SM3
ssl3-md5 => MD5
ssl3-sha1 => SHA1
whirlpool

This all comes from crypto/objects/objects.txt, so you can look there too.

(it's interesting to see that the RSA aliases use those slashes, so we do have a precedent)

aes192wrap_functions },
{ "AES-128-WRAP:id-aes128-wrap:AES128-WRAP", "fips=yes",
aes128wrap_functions },
{ "AES-256-WRAP-PADid-aes256-wrap-pad::AES256-WRAP-PAD", "fips=yes",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oops.. missing seperator ':' which is why some travis tests are failing in fips.
Still not sure that I like these strings being duplicated across providers.
Would defines be any better?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

following 2 strings are also missing the :

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#define OSSL_CIPHER_NAME_AES_256_WRAP_PAD ""AES-256-WRAP-PAD:id-aes256-wrap-pad"

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I kind of think that the FIPS provider could get away with just defining the names NIST uses.

With #10070 being a reasonable prospect, we'll be reconsidering how this works...

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My original proposal was to have symbols that carried the names, alongside the implementations, and having fipsprov.c, defltprov.c et al simply use those symbols.

#10070 certainly changes things, but not so much the central naming mechanism, so it shouldn't affect this PR.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what should ChaCha20-Poly1305 look like?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Its a combination of two algorithms. According to the naming scheme:

ALGNAME[VERSION?][-SUBNAME[VERSION?]?][-SIZE?][-MODE?]

The first algname is CHACHA20, and there is a subname POLY1305. Therefore:

CHACAH20-POLY1305

aes192wrap_functions },
{ "AES-128-WRAP:id-aes128-wrap:AES128-WRAP", "fips=yes",
aes128wrap_functions },
{ "AES-256-WRAP-PADid-aes256-wrap-pad::AES256-WRAP-PAD", "fips=yes",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure what the double seperator before AES256-WRAP-PAD would do :)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They are run-away sheep! I've just nudged them back in their fold...

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@slontis prefers goats to sheep 🤣

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@richsalz
Copy link
Contributor

richsalz commented Oct 3, 2019

I kind of think that the FIPS provider could get away with just defining the names NIST uses.

YES!!!

@levitte
Copy link
Member Author

levitte commented Oct 3, 2019

So, er, what is the NIST name for AES-265 in CBC mode? SP 800-38A doesn't help much, unless you wanna see names like CBC-AES256...

Copy link
Member

@t8m t8m left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good now

@levitte
Copy link
Member Author

levitte commented Oct 3, 2019

I'll take that approval. Should someone want to make name changes, do so in another PR.

levitte added a commit that referenced this pull request Oct 3, 2019
This modifies the treatment of algorithm name strings to allow
multiple names separated with colons.

Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from #8985)
levitte added a commit that referenced this pull request Oct 3, 2019
Reviewed-by: Tomas Mraz <tmraz@fedoraproject.org>
(Merged from #8985)
@levitte
Copy link
Member Author

levitte commented Oct 3, 2019

Merged.

695d195 Replumbing: make it possible for providers to specify multiple names
df553b7 Adapt existing providers to posibly have name lists

@levitte levitte closed this Oct 3, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

branch: master Applies to master branch

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants