Skip to content

Add darwin64-arm64-cc build target for OpenSSL 1.1.1#13476

Closed
felixbuenemann wants to merge 2 commits intoopenssl:OpenSSL_1_1_1-stablefrom
felixbuenemann:openssl-1.1.1-darwin-arm64-support
Closed

Add darwin64-arm64-cc build target for OpenSSL 1.1.1#13476
felixbuenemann wants to merge 2 commits intoopenssl:OpenSSL_1_1_1-stablefrom
felixbuenemann:openssl-1.1.1-darwin-arm64-support

Conversation

@felixbuenemann
Copy link
Copy Markdown

@felixbuenemann felixbuenemann commented Nov 22, 2020 •
edited
Loading

This adds support for the Apple Silicon M1 hardware to OpenSSL 1.1.1 on macOS Big Sur and enables aarch64 assembly.

The changes have been adapted from master and were verified using an Apple MacBook Air (2020, M1) running macOS 11.1 Beta (20C5048l) and have been compiled with Xcode 12.3 beta (12C5020f).

The test suite passes and benchmarks show the expected speed-ups.

Related #12591 Fixes #12254 Duplicate of #12369

@felixbuenemann
Copy link
Copy Markdown
Author

felixbuenemann commented Nov 22, 2020

Benchmark Results

Benchmarks were done on an Apple MacBook Air (2020, M1) with 16GB RAM on macOS 11.1 Beta (20C5048l).

Before Patch

$ openssl speed -evp aes-128-ctr
OpenSSL 1.1.1h  22 Sep 2020
built on: Sun Nov 22 20:29:09 2020 UTC
options:bn(64,64) rc4(int) des(int) aes(partial) idea(int) blowfish(ptr)
compiler: clang -fPIC -arch arm64 -O3 -Wall -DL_ENDIAN -DOPENSSL_PIC -D_REENTRANT -DNDEBUG
The 'numbers' are in 1000s of bytes per second processed.
type             16 bytes     64 bytes    256 bytes   1024 bytes   8192 bytes  16384 bytes
aes-128-ctr     296743.83k   303441.83k   303945.05k   301039.96k   301049.15k   300764.21k
$ openssl speed -evp aes-128-gcm
OpenSSL 1.1.1h  22 Sep 2020
built on: Sun Nov 22 20:29:09 2020 UTC
options:bn(64,64) rc4(int) des(int) aes(partial) idea(int) blowfish(ptr)
compiler: clang -fPIC -arch arm64 -O3 -Wall -DL_ENDIAN -DOPENSSL_PIC -D_REENTRANT -DNDEBUG
The 'numbers' are in 1000s of bytes per second processed.
type             16 bytes     64 bytes    256 bytes   1024 bytes   8192 bytes  16384 bytes
aes-128-gcm     170056.28k   172320.83k   167888.40k   167156.53k   167220.57k   167308.59k
$ openssl speed -evp chacha20-poly1305
OpenSSL 1.1.1h  22 Sep 2020
built on: Sun Nov 22 20:29:09 2020 UTC
options:bn(64,64) rc4(int) des(int) aes(partial) idea(int) blowfish(ptr)
compiler: clang -fPIC -arch arm64 -O3 -Wall -DL_ENDIAN -DOPENSSL_PIC -D_REENTRANT -DNDEBUG
The 'numbers' are in 1000s of bytes per second processed.
type             16 bytes     64 bytes    256 bytes   1024 bytes   8192 bytes  16384 bytes
chacha20-poly1305   272939.17k   367433.11k   359265.71k   356910.42k   355388.07k   355068.59k

After Patch

$ openssl speed -evp aes-128-ctr
OpenSSL 1.1.1h  22 Sep 2020
built on: Sun Nov 22 21:07:38 2020 UTC
options:bn(64,64) rc4(int) des(int) aes(partial) idea(int) blowfish(ptr)
compiler: clang -fPIC -arch arm64 -O3 -Wall -DL_ENDIAN -DOPENSSL_PIC -DOPENSSL_CPUID_OBJ -DOPENSSL_BN_ASM_MONT -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DKECCAK1600_ASM -DVPAES_ASM -DECP_NISTZ256_ASM -DPOLY1305_ASM -D_REENTRANT -DNDEBUG
The 'numbers' are in 1000s of bytes per second processed.
type             16 bytes     64 bytes    256 bytes   1024 bytes   8192 bytes  16384 bytes
aes-128-ctr    1132263.40k  3529939.22k  7579524.52k  9961685.33k 10911386.28k 11001217.02k
$ openssl speed -evp aes-128-gcm
OpenSSL 1.1.1h  22 Sep 2020
built on: Sun Nov 22 21:07:38 2020 UTC
options:bn(64,64) rc4(int) des(int) aes(partial) idea(int) blowfish(ptr)
compiler: clang -fPIC -arch arm64 -O3 -Wall -DL_ENDIAN -DOPENSSL_PIC -DOPENSSL_CPUID_OBJ -DOPENSSL_BN_ASM_MONT -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DKECCAK1600_ASM -DVPAES_ASM -DECP_NISTZ256_ASM -DPOLY1305_ASM -D_REENTRANT -DNDEBUG
The 'numbers' are in 1000s of bytes per second processed.
type             16 bytes     64 bytes    256 bytes   1024 bytes   8192 bytes  16384 bytes
aes-128-gcm    1115565.20k  3058913.96k  4781916.30k  4637353.98k  4537237.50k  4602647.89k
$ openssl speed -evp chacha20-poly1305
OpenSSL 1.1.1h  22 Sep 2020
built on: Sun Nov 22 21:07:38 2020 UTC
options:bn(64,64) rc4(int) des(int) aes(partial) idea(int) blowfish(ptr)
compiler: clang -fPIC -arch arm64 -O3 -Wall -DL_ENDIAN -DOPENSSL_PIC -DOPENSSL_CPUID_OBJ -DOPENSSL_BN_ASM_MONT -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DKECCAK1600_ASM -DVPAES_ASM -DECP_NISTZ256_ASM -DPOLY1305_ASM -D_REENTRANT -DNDEBUG
The 'numbers' are in 1000s of bytes per second processed.
type             16 bytes     64 bytes    256 bytes   1024 bytes   8192 bytes  16384 bytes
chacha20-poly1305   384908.34k   582412.41k  1081279.31k  1727471.90k  1744085.02k  1740851.71k

@felixbuenemann felixbuenemann force-pushed the openssl-1.1.1-darwin-arm64-support branch from 98cec34 to 53bdb4e Compare November 22, 2020 22:07
@felixbuenemann
Copy link
Copy Markdown
Author

felixbuenemann commented Nov 22, 2020

Updated to split the CHANGES to a separate commit. This makes it easier to apply a patch to older 1.1.1 versions.

@felixbuenemann felixbuenemann mentioned this pull request Nov 22, 2020
Closed
This was referenced Nov 22, 2020
Merged
Closed
blissdev
blissdev reviewed Nov 23, 2020
View reviewed changes
@felixbuenemann
Add darwin64-arm64-cc build target
658696e 
This adds support for the Apple Silicon M1 hardware on macOS Big Sur.

CLA: trivial
@felixbuenemann
Add CHANGES for darwin64-arm64-cc target
105be6e 
CLA: trivial
@felixbuenemann felixbuenemann force-pushed the openssl-1.1.1-darwin-arm64-support branch from 53bdb4e to 105be6e Compare November 23, 2020 00:14
@richiksc
Copy link
Copy Markdown

richiksc commented Nov 23, 2020 •
edited
Loading

@felixbuenemann Is this PR not almost identical to #12369? The only difference is that this one updates the changelog. That PR has already been discussed, and is "approved: ready to merge", but held from merging because OpenSSL 1.1 is in "bugfix only" mode and the maintainers believe that it is a stretch to call darwin-arm64 support a "bugfix". The OMC is currently discussing whether to create a new LTS+ release in parallel to the LTS release that can add support for new platforms.

@mattcaswell
Copy link
Copy Markdown
Member

mattcaswell commented Nov 23, 2020

Added the OMC hold label as per the comment above. Probably though this should be closed in favour of the pre-existing #12369.

@felixbuenemann
Copy link
Copy Markdown
Author

felixbuenemann commented Nov 23, 2020

Yes, #12369 is fine, this can be closed as a dupe.

Isn't fixing ASM support on Darwin arm64 a bugfix? This doesn't add any new code to support it.

@felixbuenemann
Copy link
Copy Markdown
Author

felixbuenemann commented Nov 23, 2020

So to elaborate on the current situation:

  • When adding patches to Homebrew Formulae, the patches have to be pushed upstream, which is what I did here (sorry for the dupe).
  • This change is configuration only, the chance to break existing targets is zero.
  • The existing aarch64 assembly code is tested on iOS via ios64 assembly target.
  • The M1 chip is an evolution of those ARMv8 chips (it is ARMv8.4 compatible) and can even run existing iOS apps without emulation, showing it is indeed fully compatible.

@felixbuenemann
Copy link
Copy Markdown
Author

felixbuenemann commented Nov 23, 2020

Duplicate of #12369

@felixbuenemann felixbuenemann mentioned this pull request Nov 23, 2020
Closed
@richiksc
Copy link
Copy Markdown

richiksc commented Nov 23, 2020

@felixbuenemann Do you want to close this PR and post your elaboration comment in the main PR?

@felixbuenemann
Copy link
Copy Markdown
Author

felixbuenemann commented Nov 23, 2020 •
edited
Loading

@richiksc Yeah, let's not split the discussion.

Closed in favour of #12369.

@johnalanwoods
Copy link
Copy Markdown

johnalanwoods commented Nov 23, 2020

@felixbuenemann, I wonder why Apple's OpenSSL doesn't see these speed-ups even though it's compiled natively?

i.e LibreSSL 2.8.3,

/usr/bin/openssl: Mach-O universal binary with 2 architectures: [x86_64:Mach-O 64-bit executable x86_64] [arm64e:Mach-O 64-bit executable arm64e]
/usr/bin/openssl (for architecture x86_64):	Mach-O 64-bit executable x86_64
/usr/bin/openssl (for architecture arm64e):	Mach-O 64-bit executable arm64e

@felixbuenemann
Copy link
Copy Markdown
Author

felixbuenemann commented Nov 23, 2020 •
edited
Loading

@felixbuenemann, I wonder why Apple's OpenSSL doesn't see these speed-ups even though it's compiled natively?

i.e LibreSSL 2.8.3,

@johnalanwoods Not much to wonder, since libressl 2.8.3 has zero armv8 assembly code:
https://github.com/libressl-portable/openbsd/tree/libressl-v2.8.3/src/lib/libcrypto/aes/asm

Note that /usr/bin/openssl on macOS is mostly there for compatibility with scripts and apps calling it for example to generate certificates. You cannot link to it or anything like that.

You will get better speeds with arch -x86_64 /usr/bin/openssl speed … since Rosetta 2 translates the x86_64 SIMD instructions to equivalent arm64 code. I won't post complete benchmarks here to avoid confusion between LibreSSL and OpenSSL, but the translated x86_64 code reaches about 65% of the arm64 ASM results above for aes-128ctr.

@GJKrupa GJKrupa mentioned this pull request Nov 29, 2020
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

1 more reviewer

@blissdev blissdev blissdev left review comments

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@felixbuenemann @richiksc @mattcaswell @johnalanwoods @blissdev