-
Notifications
You must be signed in to change notification settings - Fork 38.7k
release: support cross-compiling for arm64-apple-darwin #21851
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
The following sections might be updated with supplementary metadata relevant to reviewers and maintainers. ConflictsNo conflicts as of last run. |
|
Tried your binaries but they are faulty bash-3.2$ ./bitcoind --help
Killed: 9and bash-3.2$ otool -L ./bitcoind
./bitcoind:
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1292.0.0)
/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 904.4.0)But I don't have these libraries. Searching around gives https://developer.apple.com/forums/thread/655588?answerId=623718022#623718022 and https://developer.apple.com/documentation/macos-release-notes/macos-big-sur-11_0_1-release-notes:
|
|
Did the same So I guess it's a different issue. Any hint? |
7e2a989 depends: latest config.sub (2021-04-30) (fanquake) f16d4cd depends: latest config.guess (2021-05-24) (fanquake) Pull request description: This is split out of #21851. Updating these files should be mechanical, and shouldn't have to wait for that PR. Also, having support in depends for the new `arm-apple-darwin` target (added in [2593751ef276497e312d7c4ce7fd049614c7bf80](https://git.savannah.gnu.org/cgit/config.git/commit/?id=2593751ef276497e312d7c4ce7fd049614c7bf80)) is useful when debugging. i.e #22070. If you try and compile depends for a `arm-apple-darwin` target using master, on a x86_64 darwin machine, currently you'll get: ```bash gmake -C depends -j9 HOST=arm64-apple-darwin Invalid configuration `arm64-apple-darwin': machine `arm64-apple' not recognized shasum: hosts/.mk: No such file or directory <omitted> Makefile:111: hosts/.mk: No such file or directory gmake: *** No rule to make target 'hosts/.mk'. Stop. ``` ACKs for top commit: laanwj: ACK 7e2a989 Tree-SHA512: 8ed99b5d486c6cbca8929a752460338b6ee17f6bf93013c76589605678853c3a01ebd631b4d3f5d6aaeb6e5c21b7bbe39afc4454d3a697fafb27678f6d2c021e
93d0761 to
b1984ac
Compare
7e2a989 depends: latest config.sub (2021-04-30) (fanquake) f16d4cd depends: latest config.guess (2021-05-24) (fanquake) Pull request description: This is split out of bitcoin#21851. Updating these files should be mechanical, and shouldn't have to wait for that PR. Also, having support in depends for the new `arm-apple-darwin` target (added in [2593751ef276497e312d7c4ce7fd049614c7bf80](https://git.savannah.gnu.org/cgit/config.git/commit/?id=2593751ef276497e312d7c4ce7fd049614c7bf80)) is useful when debugging. i.e bitcoin#22070. If you try and compile depends for a `arm-apple-darwin` target using master, on a x86_64 darwin machine, currently you'll get: ```bash gmake -C depends -j9 HOST=arm64-apple-darwin Invalid configuration `arm64-apple-darwin': machine `arm64-apple' not recognized shasum: hosts/.mk: No such file or directory <omitted> Makefile:111: hosts/.mk: No such file or directory gmake: *** No rule to make target 'hosts/.mk'. Stop. ``` ACKs for top commit: laanwj: ACK 7e2a989 Tree-SHA512: 8ed99b5d486c6cbca8929a752460338b6ee17f6bf93013c76589605678853c3a01ebd631b4d3f5d6aaeb6e5c21b7bbe39afc4454d3a697fafb27678f6d2c021e
|
@promag sorry for such a slow follow up. I've only recently gotten an M1 box, and am now getting back to this. As far as I'm aware, the issue you're seeing is related to code-signing requirements. As suggested in #22403 if you run: codesign --sign - --deep --force bitcoindon my binaries they should then work. I have tested this after downloading them onto an M1 box, and running the ad-hoc signing. That's great, as it means our current toolchain seems to be setup to produce working M1 binaries (with the changes here). I will figure out how best to incorporate that signing into the build process for non-release builds. |
508ba09 to
5323956
Compare
|
This is now based on #22436. I've added a commit to add the Running Guix builds I've found that we'll need to update our security-checks tests, because ld: warning: -no_pie ignored for arm64
F
======================================================================
FAIL: test_MACHO (__main__.TestSecurityChecks)
----------------------------------------------------------------------
Traceback (most recent call last):
File "./contrib/devtools/test-security-check.py", line 83, in test_MACHO
self.assertEqual(call_security_check(cc, source, executable, ['-Wl,-no_pie','-Wl,-flat_namespace','-Wl,-allow_stack_execute','-fno-stack-protector']),
AssertionError: Tuples differ: (1, 'test1: failed NOUNDEFS NX LAZY_BINDINGS Canary CONTROL_FLOW') != (1, 'test1: failed PIE NOUNDEFS NX LAZY_BINDINGS Canary CONTROL_FLOW')
First differing element 1:
'test1: failed NOUNDEFS NX LAZY_BINDINGS Canary CONTROL_FLOW'
'test1: failed PIE NOUNDEFS NX LAZY_BINDINGS Canary CONTROL_FLOW'
- (1, 'test1: failed NOUNDEFS NX LAZY_BINDINGS Canary CONTROL_FLOW')
+ (1, 'test1: failed PIE NOUNDEFS NX LAZY_BINDINGS Canary CONTROL_FLOW')
? ++++We also need to make some update in regards to how we are doing code signing. Outside of that, this PR currently produces working M1 binaries. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Tested 53239569e2d2d041211ac37f6507ecbfb572fe00 on Linux Mint 20.2 (x86_64):
$ make -C depends qt_configured HOST=arm64-apple-darwin20
...
Configure summary:
Building on: linux-g++ (x86_64, CPU features: mmx sse sse2)
Building for: macx-clang-linux (x86_64, CPU features: cx16 mmx sse sse2 sse3 ssse3 sse4.1)
Target compiler: clang (Apple) 10.0.1
Configuration: cross_compile sse2 aesni sse3 ssse3 sse4_1 sse4_2 avx avx2 avx512f avx512bw avx512cd avx512dq avx512er avx512ifma avx512pf avx512vbmi avx512vl f16c largefile precompile_header rdrnd shani silent x86SimdAlways release c++11 c++14 c++1z reduce_exports static stl
I think in Building for: should be arm64 rather x86_64.
UPDATE: solved in #21851 (review)
5323956 to
d1a1af4
Compare
d200313 to
f26d41b
Compare
|
This is now ready for review. I've updated the security checks to account for arm64 Darwin. Switching to using autoconf-2.71 has solved the problems with outdated config.guess & config.sub. There will still be a few followup changes in regards to code-signing etc, but I think as-is, this change could go in, meaning we could be releasing 23.0 with arm64 Darwin binaries. I've added Guix hashes to the PR description. Edit:
I thought we had already removed running the test security checks from our CI. Will address. |
Guix build: |
This should be sufficient to support building for Apple ARM when cross-compiling.
This allows for building with newer targets, like arm64-apple-darwin, due to having a newer bundled config.guess and config.sub.
f26d41b to
edd70b5
Compare
|
Fixed the CI, and added hashes for all builds to the PR description.
Can you try running the binary directly, and/or providing a more informative error message. |
Guix builds: |
For now I got Trying to get more info. |
#22546 ? |
helps. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ACK edd70b5, a Guix bitcoin-edd70b52fcd7-osx-unsigned.dmg tested on Mac mini (M1, 2020) + macOS Monterey 12.1 (21C52).
…-darwin edd70b5 doc: add arm macOS depends platform triplet (jarolrod) 0628815 guix: add arm64-apple-darwin triplet (fanquake) ca47f2e guix: use autoconf 2.71 (fanquake) 6fe5516 contrib: support arm64 darwin in security checks (fanquake) 446e73c build: use macOS 11 SDK (Xcode 12.2) (fanquake) Pull request description: With a few minor changes, our macOS toolchain, and Guix, supports building for `arm64-apple-darwin` (Apple M1). Newer [`config.guess`](https://git.savannah.gnu.org/cgit/config.git/plain/config.guess) and [`config.sub`](https://git.savannah.gnu.org/cgit/config.git/plain/config.sub) are required (via autoconf 2.71), as support for recognising `arm64-apple-darwin` [was only recently added](https://git.savannah.gnu.org/cgit/config.git/commit/?id=2593751ef276497e312d7c4ce7fd049614c7bf80). A newer version of the macOS SDK (11.0+) is also required; I've taken this from Xcode 12.2. Guix Build: ```bash bash-5.1# find guix-build-$(git rev-parse --short=12 HEAD)/output/ -type f -print0 | env LC_ALL=C sort -z | xargs -r0 sha256sum c9c64c6252927c82d4a96eb2b1a13c557ac230afa3640a7f2feec87fabe2b7e5 guix-build-edd70b52fcd7/output/aarch64-linux-gnu/SHA256SUMS.part b57dd461ec6a0b22ce9614ab824bcbff4c3e6e935c22367aff9a718f46c3fd39 guix-build-edd70b52fcd7/output/aarch64-linux-gnu/bitcoin-edd70b52fcd7-aarch64-linux-gnu-debug.tar.gz 66da9f57a6789108db3460d2b54a8bed68b7e147627404e51f66254d91500402 guix-build-edd70b52fcd7/output/aarch64-linux-gnu/bitcoin-edd70b52fcd7-aarch64-linux-gnu.tar.gz 13213523984837fc66fe27cc4dac269f6ef554ec56316af92337f4c4845f6acf guix-build-edd70b52fcd7/output/arm-linux-gnueabihf/SHA256SUMS.part e1a17b41c92879ccb6608a6a6f82aac22d999f10d798f40f96297d91ebd90d20 guix-build-edd70b52fcd7/output/arm-linux-gnueabihf/bitcoin-edd70b52fcd7-arm-linux-gnueabihf-debug.tar.gz 6a6d54081c2dd8f9a44293275aea875587b5fa7cc31a30925fb5156bb1e55bf3 guix-build-edd70b52fcd7/output/arm-linux-gnueabihf/bitcoin-edd70b52fcd7-arm-linux-gnueabihf.tar.gz 79f617a2f0a11e6a8fcf8058e05204a374ab5f689ccb15a2ff449ee2dd5e8216 guix-build-edd70b52fcd7/output/arm64-apple-darwin/SHA256SUMS.part 6fb04a18b17812174ec35a6788dce5745d6c2dec558056dcfdd356b57d089b99 guix-build-edd70b52fcd7/output/arm64-apple-darwin/bitcoin-edd70b52fcd7-arm64-apple-darwin.tar.gz a67e415631bea940adab86adec9899593fa35e1773d4ac1149efd249adf89996 guix-build-edd70b52fcd7/output/arm64-apple-darwin/bitcoin-edd70b52fcd7-osx-unsigned.dmg 2a41c273afb8bb621226de80302a7190eee057c2447d113999f40a9485430d44 guix-build-edd70b52fcd7/output/arm64-apple-darwin/bitcoin-edd70b52fcd7-osx-unsigned.tar.gz c8d9358471e1d72d56122ae6f53dc2a42ae0925628bf347acd1a3ae7110d8759 guix-build-edd70b52fcd7/output/dist-archive/bitcoin-edd70b52fcd7.tar.gz 1985a4869307418ec2ee85f5f0ccfea0944eba0792318a8676ae213a9589ed97 guix-build-edd70b52fcd7/output/powerpc64-linux-gnu/SHA256SUMS.part 6a1d3525011da84e24997c9ab39563b3597ae881b4d1af5a09a6e673cfc59808 guix-build-edd70b52fcd7/output/powerpc64-linux-gnu/bitcoin-edd70b52fcd7-powerpc64-linux-gnu-debug.tar.gz cda50c28b2920caef74de4b3e78fd4d81592d1aadc9758d1882f1cad16dc83ca guix-build-edd70b52fcd7/output/powerpc64-linux-gnu/bitcoin-edd70b52fcd7-powerpc64-linux-gnu.tar.gz ca92936cf796543d13250f94175af772a9fc985e834017d6af2cfb36eea289e6 guix-build-edd70b52fcd7/output/powerpc64le-linux-gnu/SHA256SUMS.part 64c46248065149a384c5710adeb4fd8e795ffee7d6441e4edb71468c6cf1478e guix-build-edd70b52fcd7/output/powerpc64le-linux-gnu/bitcoin-edd70b52fcd7-powerpc64le-linux-gnu-debug.tar.gz bb9c3400a1dd88fa5fd26e8e22ba0f3a43263bdbb762233ed1670394d3d69ed1 guix-build-edd70b52fcd7/output/powerpc64le-linux-gnu/bitcoin-edd70b52fcd7-powerpc64le-linux-gnu.tar.gz 926ef7a23ecdf02a6619f9dba0db814373c1a5105347e4c13b80a116910d1c8d guix-build-edd70b52fcd7/output/riscv64-linux-gnu/SHA256SUMS.part cdf7ce78f206378a44122865150784c20bf1bec6943b48e35a41633c0f1f05ed guix-build-edd70b52fcd7/output/riscv64-linux-gnu/bitcoin-edd70b52fcd7-riscv64-linux-gnu-debug.tar.gz 026bbb6662f4074054457ce1678fc1d697bfbf87c80f1c3d5d560bcc18e3bea6 guix-build-edd70b52fcd7/output/riscv64-linux-gnu/bitcoin-edd70b52fcd7-riscv64-linux-gnu.tar.gz 0190b93fc5d5202c6b252e51c34799fb153a284cc53b0e6f62d2521dc61866bd guix-build-edd70b52fcd7/output/x86_64-apple-darwin/SHA256SUMS.part 69aac1a962aeeec01c30015f79d93c57d68a484c3905b8a526f90e7e7382bc1b guix-build-edd70b52fcd7/output/x86_64-apple-darwin/bitcoin-edd70b52fcd7-osx-unsigned.dmg ea37691f1df0389afedbb9eece45660dcb2ee867ffda1d26b244a471af2a6230 guix-build-edd70b52fcd7/output/x86_64-apple-darwin/bitcoin-edd70b52fcd7-osx-unsigned.tar.gz f56ff4035e3bc70c2cbb04f8a7f5ef23096088189e84fa172cec97f59336cf2b guix-build-edd70b52fcd7/output/x86_64-apple-darwin/bitcoin-edd70b52fcd7-osx64.tar.gz a1c1bf47083c6f19bcc68761ea56bec666cfe128f4ba878a5655ae12874fe038 guix-build-edd70b52fcd7/output/x86_64-linux-gnu/SHA256SUMS.part cad26c66c764361b30c8204e7743dbb6c0e6dd1eb3af9418d77bd566e05e1470 guix-build-edd70b52fcd7/output/x86_64-linux-gnu/bitcoin-edd70b52fcd7-x86_64-linux-gnu-debug.tar.gz ed2b459a06d5bc0f895402323fc5cae543a387d0621a55214feb37513210324e guix-build-edd70b52fcd7/output/x86_64-linux-gnu/bitcoin-edd70b52fcd7-x86_64-linux-gnu.tar.gz ``` ACKs for top commit: hebasto: ACK edd70b5, a Guix `bitcoin-edd70b52fcd7-osx-unsigned.dmg` tested on Mac mini (M1, 2020) + macOS Monterey 12.1 (21C52). Tree-SHA512: 00c55e139ce60c89900ef9365c380beec9474d8a0459bb584fd966d17715344fcf95b208b7222f665a5e3989aed51d3161641d3737aa6122a41b0c9f0d35906d
…-darwin edd70b5 doc: add arm macOS depends platform triplet (jarolrod) 0628815 guix: add arm64-apple-darwin triplet (fanquake) ca47f2e guix: use autoconf 2.71 (fanquake) 6fe5516 contrib: support arm64 darwin in security checks (fanquake) 446e73c build: use macOS 11 SDK (Xcode 12.2) (fanquake) Pull request description: With a few minor changes, our macOS toolchain, and Guix, supports building for `arm64-apple-darwin` (Apple M1). Newer [`config.guess`](https://git.savannah.gnu.org/cgit/config.git/plain/config.guess) and [`config.sub`](https://git.savannah.gnu.org/cgit/config.git/plain/config.sub) are required (via autoconf 2.71), as support for recognising `arm64-apple-darwin` [was only recently added](https://git.savannah.gnu.org/cgit/config.git/commit/?id=2593751ef276497e312d7c4ce7fd049614c7bf80). A newer version of the macOS SDK (11.0+) is also required; I've taken this from Xcode 12.2. Guix Build: ```bash bash-5.1# find guix-build-$(git rev-parse --short=12 HEAD)/output/ -type f -print0 | env LC_ALL=C sort -z | xargs -r0 sha256sum c9c64c6252927c82d4a96eb2b1a13c557ac230afa3640a7f2feec87fabe2b7e5 guix-build-edd70b52fcd7/output/aarch64-linux-gnu/SHA256SUMS.part b57dd461ec6a0b22ce9614ab824bcbff4c3e6e935c22367aff9a718f46c3fd39 guix-build-edd70b52fcd7/output/aarch64-linux-gnu/bitcoin-edd70b52fcd7-aarch64-linux-gnu-debug.tar.gz 66da9f57a6789108db3460d2b54a8bed68b7e147627404e51f66254d91500402 guix-build-edd70b52fcd7/output/aarch64-linux-gnu/bitcoin-edd70b52fcd7-aarch64-linux-gnu.tar.gz 13213523984837fc66fe27cc4dac269f6ef554ec56316af92337f4c4845f6acf guix-build-edd70b52fcd7/output/arm-linux-gnueabihf/SHA256SUMS.part e1a17b41c92879ccb6608a6a6f82aac22d999f10d798f40f96297d91ebd90d20 guix-build-edd70b52fcd7/output/arm-linux-gnueabihf/bitcoin-edd70b52fcd7-arm-linux-gnueabihf-debug.tar.gz 6a6d54081c2dd8f9a44293275aea875587b5fa7cc31a30925fb5156bb1e55bf3 guix-build-edd70b52fcd7/output/arm-linux-gnueabihf/bitcoin-edd70b52fcd7-arm-linux-gnueabihf.tar.gz 79f617a2f0a11e6a8fcf8058e05204a374ab5f689ccb15a2ff449ee2dd5e8216 guix-build-edd70b52fcd7/output/arm64-apple-darwin/SHA256SUMS.part 6fb04a18b17812174ec35a6788dce5745d6c2dec558056dcfdd356b57d089b99 guix-build-edd70b52fcd7/output/arm64-apple-darwin/bitcoin-edd70b52fcd7-arm64-apple-darwin.tar.gz a67e415631bea940adab86adec9899593fa35e1773d4ac1149efd249adf89996 guix-build-edd70b52fcd7/output/arm64-apple-darwin/bitcoin-edd70b52fcd7-osx-unsigned.dmg 2a41c273afb8bb621226de80302a7190eee057c2447d113999f40a9485430d44 guix-build-edd70b52fcd7/output/arm64-apple-darwin/bitcoin-edd70b52fcd7-osx-unsigned.tar.gz c8d9358471e1d72d56122ae6f53dc2a42ae0925628bf347acd1a3ae7110d8759 guix-build-edd70b52fcd7/output/dist-archive/bitcoin-edd70b52fcd7.tar.gz 1985a4869307418ec2ee85f5f0ccfea0944eba0792318a8676ae213a9589ed97 guix-build-edd70b52fcd7/output/powerpc64-linux-gnu/SHA256SUMS.part 6a1d3525011da84e24997c9ab39563b3597ae881b4d1af5a09a6e673cfc59808 guix-build-edd70b52fcd7/output/powerpc64-linux-gnu/bitcoin-edd70b52fcd7-powerpc64-linux-gnu-debug.tar.gz cda50c28b2920caef74de4b3e78fd4d81592d1aadc9758d1882f1cad16dc83ca guix-build-edd70b52fcd7/output/powerpc64-linux-gnu/bitcoin-edd70b52fcd7-powerpc64-linux-gnu.tar.gz ca92936cf796543d13250f94175af772a9fc985e834017d6af2cfb36eea289e6 guix-build-edd70b52fcd7/output/powerpc64le-linux-gnu/SHA256SUMS.part 64c46248065149a384c5710adeb4fd8e795ffee7d6441e4edb71468c6cf1478e guix-build-edd70b52fcd7/output/powerpc64le-linux-gnu/bitcoin-edd70b52fcd7-powerpc64le-linux-gnu-debug.tar.gz bb9c3400a1dd88fa5fd26e8e22ba0f3a43263bdbb762233ed1670394d3d69ed1 guix-build-edd70b52fcd7/output/powerpc64le-linux-gnu/bitcoin-edd70b52fcd7-powerpc64le-linux-gnu.tar.gz 926ef7a23ecdf02a6619f9dba0db814373c1a5105347e4c13b80a116910d1c8d guix-build-edd70b52fcd7/output/riscv64-linux-gnu/SHA256SUMS.part cdf7ce78f206378a44122865150784c20bf1bec6943b48e35a41633c0f1f05ed guix-build-edd70b52fcd7/output/riscv64-linux-gnu/bitcoin-edd70b52fcd7-riscv64-linux-gnu-debug.tar.gz 026bbb6662f4074054457ce1678fc1d697bfbf87c80f1c3d5d560bcc18e3bea6 guix-build-edd70b52fcd7/output/riscv64-linux-gnu/bitcoin-edd70b52fcd7-riscv64-linux-gnu.tar.gz 0190b93fc5d5202c6b252e51c34799fb153a284cc53b0e6f62d2521dc61866bd guix-build-edd70b52fcd7/output/x86_64-apple-darwin/SHA256SUMS.part 69aac1a962aeeec01c30015f79d93c57d68a484c3905b8a526f90e7e7382bc1b guix-build-edd70b52fcd7/output/x86_64-apple-darwin/bitcoin-edd70b52fcd7-osx-unsigned.dmg ea37691f1df0389afedbb9eece45660dcb2ee867ffda1d26b244a471af2a6230 guix-build-edd70b52fcd7/output/x86_64-apple-darwin/bitcoin-edd70b52fcd7-osx-unsigned.tar.gz f56ff4035e3bc70c2cbb04f8a7f5ef23096088189e84fa172cec97f59336cf2b guix-build-edd70b52fcd7/output/x86_64-apple-darwin/bitcoin-edd70b52fcd7-osx64.tar.gz a1c1bf47083c6f19bcc68761ea56bec666cfe128f4ba878a5655ae12874fe038 guix-build-edd70b52fcd7/output/x86_64-linux-gnu/SHA256SUMS.part cad26c66c764361b30c8204e7743dbb6c0e6dd1eb3af9418d77bd566e05e1470 guix-build-edd70b52fcd7/output/x86_64-linux-gnu/bitcoin-edd70b52fcd7-x86_64-linux-gnu-debug.tar.gz ed2b459a06d5bc0f895402323fc5cae543a387d0621a55214feb37513210324e guix-build-edd70b52fcd7/output/x86_64-linux-gnu/bitcoin-edd70b52fcd7-x86_64-linux-gnu.tar.gz ``` ACKs for top commit: hebasto: ACK edd70b5, a Guix `bitcoin-edd70b52fcd7-osx-unsigned.dmg` tested on Mac mini (M1, 2020) + macOS Monterey 12.1 (21C52). Tree-SHA512: 00c55e139ce60c89900ef9365c380beec9474d8a0459bb584fd966d17715344fcf95b208b7222f665a5e3989aed51d3161641d3737aa6122a41b0c9f0d35906d


With a few minor changes, our macOS toolchain, and Guix, supports building for
arm64-apple-darwin(Apple M1). Newerconfig.guessandconfig.subare required (via autoconf 2.71), as support for recognisingarm64-apple-darwinwas only recently added. A newer version of the macOS SDK (11.0+) is also required; I've taken this from Xcode 12.2.Guix Build: