-
Notifications
You must be signed in to change notification settings - Fork 4.9k
AesGcm, AesCcm #31389
AesGcm, AesCcm #31389
Conversation
|
It would be nice if the macOS platform used the same implementation from Unix and optionally "light up" macOS of they happen to have a suitable version of OpenSSL installed. The motivation for this is so that there is something that is workable in macOS. I realize OpenSSL is not exactly in-the-box on macOS (aside from an ancient version and an fork of it as libressl). However, for aiding in local development, it would be useful to be able to use it if I happen to have it installed with homebrew or however I get it installed (similar to how things worked in the 1.x days). |
| CheckBoolReturn(Interop.Crypto.EvpCipherFinalEx(_ctx, outputCurrent, out bytesWritten)); | ||
| outputBytes += bytesWritten; | ||
| } | ||
| Span<byte> outputSpan = new Span<byte>(output).Slice(outputBytes); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
output.AsSpan(outputBytes)
| inputCurrent, | ||
| count); | ||
| } | ||
| ReadOnlySpan<byte> inputSpan = new ReadOnlySpan<byte>(input).Slice(inputOffset); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
input.AsSpan(inputOffset)
| count); | ||
| } | ||
| ReadOnlySpan<byte> inputSpan = new ReadOnlySpan<byte>(input).Slice(inputOffset); | ||
| Span<byte> outputSpan = new Span<byte>(output).Slice(outputOffset); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
output.AsSpan(outputOffset)
| } | ||
| Buffer.BlockCopy(key, 0, blob, sizeof(BCRYPT_KEY_DATA_BLOB_HEADER), keySize); | ||
|
|
||
| key.CopyTo(blob.AsSpan().Slice(sizeof(BCRYPT_KEY_DATA_BLOB_HEADER))); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
AsSpan(sizeof(BCRYPT_KEY_DATA_BLOB_HEADER))
Please revisit other uses of new Span/Slice/AsSpan as necessary.
| public static new System.Security.Cryptography.Aes Create() { throw null; } | ||
| public static new System.Security.Cryptography.Aes Create(string algorithmName) { throw null; } | ||
| } | ||
| public class AesGcm : IDisposable |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
i think these should be sealed
|
Eh, seems like CCM might not work on Ubuntu 14.04 - per openssl release notes it seems to have been added between OpenSSL 1.0.2h and OpenSSL 1.1.0 and 14.04 seems to be on 1.0.1. Seems like we already do that in some places:
|
|
@vcsjones - I personally think this would be unnecessarily complicated but do not have strong opinion about it. As for openssl on OSX - considering I'll likely need to add version checks I think it might be possible to make it check for presence of OpenSSL and make it work on OSX if OpenSSL with appropriate version is installed. Not sure if this will happen in this PR or later though. |
That's fine, you could also just remove the macos exclusion here. https://github.com/dotnet/corefx/pull/31389/files#diff-f5613604087291d4cee8422733c29366R383 Since ubuntu Ubuntu 14.04 might be forcing you to implement a openssl version check, you may get that for macOS for "free" as well too.
See and it's usages corefx/src/Common/src/Interop/Unix/System.Security.Cryptography.Native/Interop.OpenSslVersion.cs Line 17 in 711c428
|
| } | ||
| Buffer.BlockCopy(key, 0, blob, sizeof(BCRYPT_KEY_DATA_BLOB_HEADER), keySize); | ||
|
|
||
| key.CopyTo(blob.AsSpan().Slice(sizeof(BCRYPT_KEY_DATA_BLOB_HEADER))); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Instead of AsSpan().Slice( you should be able to just do AsSpan(
| /// <summary> | ||
| /// Copies bytes and holds them as native blob | ||
| /// </summary> | ||
| internal unsafe sealed class SafeKeyHandle : SafeHandle |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't know that this really belongs in Common. And nothing about it really makes it "a key", other than usage. It's really SafeExternalizedDataHandle.
Assuming (from hallway discussions) that this is just for Linux CCM, just store it as byte[], don't bother with moving it to native memory (unless there's a persistence requirement for this pointer in OpenSSL, but I'm pretty sure they copy the data, not hold the pointer)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The only concern with byte[] is that it can be moved around in memory
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yep. But there's an outstanding feature request to get GC compaction to clean up after itself. And we already do it with the Aes class.
|
|
||
| public override string ToString() | ||
| { | ||
| return $"{Source}; Id={CaseId}; KeyLen={BitLength(Key)}; NonceLen={BitLength(Nonce)}; PTLen={BitLength(Plaintext)}; AADLen={BitLength(AssociatedData)}; TagLen={BitLength(Tag)}"; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This should just be Source and ID. The purpose of this class is to make the test log easier to read. The places that are varying on nonce length should rename their source to take that into account.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The motivation here was that some NIST test vectors have duplicate Count = X in the same file - not sure how to rename the source to account for that (except for doing it as above except with hard-coded string)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ok nvm, explained later
| // cases 17, 18 have not supported nonce size | ||
| }; | ||
|
|
||
| // [ CaseId; Key.Length; Nonce.Length; Plaintext.Length; AssociatedData.Length; Tag.Length ] is unique |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'd like to see the Source take into account the file and a short form of the section, just to keep the test log easier to read.
Something like Source = $"{NistGcmTestVectors} - gcmEncryptExtV128 - 128/96/0/0/128",, then the Count= item would be the CaseId here.
| } | ||
|
|
||
| // CaseId is unique per test case | ||
| private static AEADTest[] s_nistGcmSpecTestCases = new AEADTest[] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The field can be declared readonly. This applies to the other case-sets, too.
| { | ||
| private static readonly SafeAlgorithmHandle s_aesGcm = AesBCryptModes.OpenAesAlgorithm(Cng.BCRYPT_CHAIN_MODE_GCM); | ||
| private SafeKeyHandle _keyHandle; | ||
| public static KeySizes TagByteSizes { get; } = AesAEAD.GetTagLengths(s_aesGcm); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why bother asking the OS for the answer we already know? I'd just make this be defined as a literal in the AnyOS version.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think this is implementation-specific
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
IIRC, Windows only allows a single size here, and we locked the OpenSSL/Unix one to the single Windows value.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@bartonjs I was wondering if this should be hard coded or "always ask OS" - will change to hard-coded as it probably won't change in the future
| const int datalength = 32; | ||
| byte[] originalData = Enumerable.Range(1, datalength).Select((x) => (byte)x).ToArray(); | ||
| byte[] nonce = Convert.FromBase64String("tBMp3WSvLDA2ZhtG"); | ||
| byte[] key = Convert.FromBase64String("1aGU7ZDP4Iq+zUaRmXzrLA=="); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Most of the rest of crypto uses hex.
|
|
||
| if (!AesAEAD.MatchesKeySizes(tagSizeInBytes, TagByteSizes)) | ||
| { | ||
| throw new CryptographicException(SR.Cryptography_InvalidTagLength); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
These feel like ArgumentException
| AesAEAD.CheckKeySize(keyLength); | ||
|
|
||
| _ctxHandle = Interop.Crypto.EvpCipherCreatePartial( | ||
| GetCipher(keyLength), |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This seems overly indented
| public static void NullKey() | ||
| { | ||
| Assert.Throws<ArgumentNullException>(() => new AesCcm((byte[])null)); | ||
| Assert.Throws<ArgumentNullException>(() => new AesCcm((ReadOnlySpan<byte>)null)); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The span ctor should not throw an ArgumentNullException.
|
@vcsjones I think we can make OSX work but I currently don't have clean OSX to test scenarios where openssl >= 1.0.0 is not present (CIs have it preinstalled). I think I'll add version detection to non-OSX for now and will leave the issue open to fix it later (or maybe someone could help testing this). We need to make sure we get PNSE and not any other exception in such cases |
Happy to help if you want to go that direction. I have a 2012 MBP that has a bare 10.12 installation. I will make sure not to install openssl on it :-). |
|
@vcsjones thank you for offering your help! |
|
In the interest of moving the PR along, I'm fine with going in with macOS throwing PNSE in this PR. I can open a separate issue once this is merged to have macOS use OpenSSL if present with the version check and do that myself since I'm the one that wants it, assuming there are no other objections to it. 😊 |
|
On Linux we have a requirement that OpenSSL be found, so getting a TypeInitializationException due to the... TargetInvocationException(?) is acceptable. On macOS we should really do a PNSE, since it's an optional component. If we let Ubuntu 14.04 and macOS both do PNSE: internal static class AeadAvailability
{
internal static readonly bool IsSupported = CheckSupport();
private static void CheckSupport()
{
try
{
if (Interop.Crypto.HoweverWeCheckOpenSslVersions > Whatever)
{
return true;
}
}
catch (TypeInitializationException)
{
}
return false;
}
}I think. The intermediate type is needed to keep the TypeInitializationException from Interop.Crypto from cascading. |
|
Though instead of checking versions it might make more sense to just add a shim method to answer the question. Assuming that Aes128Gcm cipher (and friends) weren't defined in the version that Ubuntu 14.04 has, they'd need to be moved to non-required portable bind, and you can make a detector method See the handling for EC_GF2m_simple_method for how this will need to work. |
| } | ||
| else | ||
| { | ||
| byte nonEmptyArray = 123; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm not sure I understand what this block is for... is it to handle a 0 length ciphertext that still has associated data? If so, I don't understand why we're passing in this value
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yes, will add comment
| { | ||
| if (!Interop.Crypto.EvpCipherUpdate(ctx, ref MemoryMarshal.GetReference(plaintext), out plaintextBytesWritten, ref MemoryMarshal.GetReference(ciphertext), ciphertext.Length)) | ||
| { | ||
| throw new CryptographicException(SR.Cryptography_AuthTagMismatch); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
EvpCipherUpdate doesn't check the tag (it can't since you might add more data). This should have a different message.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actually for CCM data is fixed and known upfront and EvpCipherUpdate does check the tag (what you said is true for GCM though - CCM always fails when calling *FinalEx)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do any of the tests feed ciphertext that is too long or too short into decrypt? If not, please add those test cases.
| byte[] nonce = new byte[AesCcm.NonceByteSizes.MinSize]; | ||
| byte[] tag = new byte[AesCcm.TagByteSizes.MinSize]; | ||
| rng.GetBytes(key); | ||
| rng.GetBytes(nonce); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
RandomNumberGenerator.Fill(key) lets you get rid of the using.
| [InlineData(41)] | ||
| [InlineData(48)] | ||
| [InlineData(50)] | ||
| public static void EncryptDecryptRoundtripNoAAD(int dataLength) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What is this adding over the NIST cases? (If it's adding something I'm happy to have it, but I'm not seeing what it is)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
for gcm probably not much - will remove - for ccm - nist test cases are not great (most of them have nonce size we don't support) and haven't seen any cases without AAD (will remove those which don't add value)
| } | ||
| else | ||
| { | ||
| byte nonEmptyArray = 123; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do we even need to support 0-length plaintext? What's the case for just producing authenticated data with AES?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think for something like TLS that's "I'm sending an empty keepalive frame"
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@morganbr mixed feeling about supporting vs non-supporting this scenario - on one hand there is probably no good scenarios where you should do that over i.e. calculating hash on the other hand it might make sense if for whatever reason you got such data over the wire (or the other end expected it) and would like to validate it. Since this is a low-level API I do not think we should not support it
| byte[] key = new byte[keyLength]; | ||
| rng.GetBytes(key); | ||
|
|
||
| Assert.Throws<CryptographicException>(() => new AesCcm(key)); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Since this is "your input is bad" it should be something based on ArgumentException
|
|
||
| if (!Interop.Crypto.EvpCipherFinalEx(_ctxHandle, ref MemoryMarshal.GetReference(plaintext.Slice(plaintextBytesWritten)), out int bytesWritten)) | ||
| { | ||
| throw Interop.Crypto.CreateOpenSslCryptographicException(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Shouldn't this have the invalid tag string?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yes, thanks for spotting
| { | ||
| aesGcm.Encrypt(nonce, plaintext, ciphertext, tag); | ||
|
|
||
| int tamperedIdx = r.Next() % tag.Length; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Since this is random, it should log the tag on failure
| ref byte nullRef = ref MemoryMarshal.GetReference(Span<byte>.Empty); | ||
|
|
||
| int keyLength = _keyHandle.Key.Length * 8; | ||
| using (SafeEvpCipherCtxHandle ctx = Interop.Crypto.EvpCipherCreatePartial( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
A comment is probably warranted for why CCM opens a new key handle every time but GCM doesn't
Update: I guess you had one up at the field. Nevermind.
|
had to rebase after merge conflict |
|
@krwq we do not use api-ready-for-review on PRs. Please create a new issue (or reuse existing one) with the API proposal. |
|
API got approved today, squashing and merging... |
* AesGcm, AesCcm * add osx stubs * disable tests on osx and desktop (APIs not there/not supported) * TagByteSizes on OSX * fix TagByteSizes compilation error * apply review feedback * fix typo when setting tag/nonce length in ccm * add missing SetCcmTagLength * attempt to detect if ccm is available on Ubuntu 14.04 * disable tests on ubuntu 14.04, attempt for osx support * attempt to fix osx * fix osx * review feedback * disable new ccm testcases on ubuntu 14.04 * attempt to make Ubuntu 14.04 work * add Interop.Initialization to OSX * fix KeySizeHelpers after merge conflict Signed-off-by: dotnet-bot <dotnet-bot@microsoft.com>
* AesGcm, AesCcm * add osx stubs * disable tests on osx and desktop (APIs not there/not supported) * TagByteSizes on OSX * fix TagByteSizes compilation error * apply review feedback * fix typo when setting tag/nonce length in ccm * add missing SetCcmTagLength * attempt to detect if ccm is available on Ubuntu 14.04 * disable tests on ubuntu 14.04, attempt for osx support * attempt to fix osx * fix osx * review feedback * disable new ccm testcases on ubuntu 14.04 * attempt to make Ubuntu 14.04 work * add Interop.Initialization to OSX * fix KeySizeHelpers after merge conflict Signed-off-by: dotnet-bot <dotnet-bot@microsoft.com>
|
Tears in my eyes. Thank you, @krwq! ❤️ |
* AesGcm, AesCcm * add osx stubs * disable tests on osx and desktop (APIs not there/not supported) * TagByteSizes on OSX * fix TagByteSizes compilation error * apply review feedback * fix typo when setting tag/nonce length in ccm * add missing SetCcmTagLength * attempt to detect if ccm is available on Ubuntu 14.04 * disable tests on ubuntu 14.04, attempt for osx support * attempt to fix osx * fix osx * review feedback * disable new ccm testcases on ubuntu 14.04 * attempt to make Ubuntu 14.04 work * add Interop.Initialization to OSX * fix KeySizeHelpers after merge conflict Signed-off-by: dotnet-bot <dotnet-bot@microsoft.com>
* AesGcm, AesCcm * add osx stubs * disable tests on osx and desktop (APIs not there/not supported) * TagByteSizes on OSX * fix TagByteSizes compilation error * apply review feedback * fix typo when setting tag/nonce length in ccm * add missing SetCcmTagLength * attempt to detect if ccm is available on Ubuntu 14.04 * disable tests on ubuntu 14.04, attempt for osx support * attempt to fix osx * fix osx * review feedback * disable new ccm testcases on ubuntu 14.04 * attempt to make Ubuntu 14.04 work * add Interop.Initialization to OSX * fix KeySizeHelpers after merge conflict Signed-off-by: dotnet-bot <dotnet-bot@microsoft.com>
|
Why 3.0 and not 2.2 if it's already merged? Reference: https://github.com/dotnet/core/blob/master/roadmap.md. |
|
@pgolebiowski master flows automatically only into 3.0. .NET Core 2.2 is more servicing release for lower layers (CoreCLR/CoreFx). There is more churn (incl. automatic flow from master branch) in higher levels (e.g. ASP.NET Core). |
|
Thank you for the clarification, sir. |
|
Anyway this can be used from the old .NET Framework? |
|
No, changes in Core don't flow into .NET Framework. Too much risk of breaking something. |
* AesGcm, AesCcm * add osx stubs * disable tests on osx and desktop (APIs not there/not supported) * TagByteSizes on OSX * fix TagByteSizes compilation error * apply review feedback * fix typo when setting tag/nonce length in ccm * add missing SetCcmTagLength * attempt to detect if ccm is available on Ubuntu 14.04 * disable tests on ubuntu 14.04, attempt for osx support * attempt to fix osx * fix osx * review feedback * disable new ccm testcases on ubuntu 14.04 * attempt to make Ubuntu 14.04 work * add Interop.Initialization to OSX * fix KeySizeHelpers after merge conflict Commit migrated from dotnet/corefx@831dc11
Fixes: https://github.com/dotnet/corefx/issues/23629
Low level functions for AES-GCM, AES-CCM
API shape (equivalent for AesCcm)
Design notes: