Skip to content

Conversation

@fanquake
Copy link
Member

@fanquake fanquake commented May 9, 2021

Addresses the macOS portion of #21888.

Build Boost with -fcf-protection when targeting Darwin. This should be ok, because our cross-compiler (Clang 10) supports the option, and I'd expect all versions of Apple Clang being used to compile Core would also support it. Building Boost with this option is required so that the main provided to test_bitcoin has instrumentation.

Note that the presence of instrumentation does not mean it will be used, as that is determined at runtime by the CPU.
From the Intel control flow enforcement documentation:

The ENDBR32 and ENDBR64 instructions will have the same effect as the NOP instruction on Intel 64 processors that do not support CET. On processors supporting CET, these instructions do not change register or flag state. This allows CET instrumented programs to execute on processors that do not support CET. Even when CET is supported and enabled, these NOP–like instructions do not affect the execution state of the program, do not cause any additional register pressure, and are minimally intrusive from power and performance perspectives.

Follow up from #21135.

Guix builds:

663df8471400f06d4da739e39a886aa17f56a36d66e0ff7cc290686294ef39c9  guix-build-42b589d18fed/output/dist-archive/bitcoin-42b589d18fed.tar.gz
45e841661e1659a634468b6f8c9fb0a7956c31ba296f1fd0c02cd880736d6127  guix-build-42b589d18fed/output/x86_64-apple-darwin18/bitcoin-42b589d18fed-osx-unsigned.dmg
0ea85c99fef35429a5048fa14850bce6b900eaa887aeea419b019852f8d2be78  guix-build-42b589d18fed/output/x86_64-apple-darwin18/bitcoin-42b589d18fed-osx-unsigned.tar.gz
85857a5a4a5d4d3a172d6c361c12c4a94f6505fc12b527ea63b75bfe54ee1001  guix-build-42b589d18fed/output/x86_64-apple-darwin18/bitcoin-42b589d18fed-osx64.tar.gz

Gitian builds:

# macOS:
bdfd677a6b88273a741b433e1e7f554af50cc76b3342d44ab0c441e2b40efc96  bitcoin-42b589d18fed-osx-unsigned.dmg
f3b2d09f3bea7a5cc489b02e8e53dd76a9922338500fae79cad0506655af56f9  bitcoin-42b589d18fed-osx-unsigned.tar.gz
29d5ad5e46bc9fb0056922a8b47c026e5e9f71e6cf447203b74644587d6fb6f7  bitcoin-42b589d18fed-osx64.tar.gz
663df8471400f06d4da739e39a886aa17f56a36d66e0ff7cc290686294ef39c9  src/bitcoin-42b589d18fed.tar.gz
366f8d7a2fc1f3e22cb1018043099126a71ce65380cc27b1c3280cce42d06c98  bitcoin-core-osx-22-res.yml

fanquake added 2 commits May 9, 2021 13:38
The LLVM Clang we use for cross-compilation supports this option, and it's expected
that any builders on macOS will also be using an Apple Clang that supports it.
@laanwj
Copy link
Member

laanwj commented May 10, 2021

Code review ACK 42b589d
Haven't tested on the target platform.

@DrahtBot
Copy link
Contributor

Guix builds

File commit 94f8353
(master)
commit 4cba262997f809f9b4e175cdf3cf35195aaa971b
(master and this pull)
*-aarch64-linux-gnu-debug.tar.gz 0b78d0b45eac19aa... 027280bd748a227d...
*-aarch64-linux-gnu.tar.gz 1c27cc723ab9f9e3... 537291551ce8b0da...
*-arm-linux-gnueabihf-debug.tar.gz d830ae03907ba6fc... f45e3b95f79222f7...
*-arm-linux-gnueabihf.tar.gz efad98cedf9aeeb4... 3e2cb840d5a134f7...
*-osx-unsigned.dmg ba2522001444832f... 033a30eb25fbbd84...
*-osx-unsigned.tar.gz 03e8ee2628e35992... b51dc120e87e2c30...
*-osx64.tar.gz 11f6e49b98bec565... d89b72d273766c16...
*-powerpc64-linux-gnu-debug.tar.gz 341bd50f2647e026... 7ebbf2ffb1d1793d...
*-powerpc64-linux-gnu.tar.gz 4cac48951290e5e6... 99411c346d63e064...
*-powerpc64le-linux-gnu-debug.tar.gz 2677a745e2c477e6... e744a1bbff4dbf4b...
*-powerpc64le-linux-gnu.tar.gz 0803688359f7e1c1... 898cdff0375b1656...
*-riscv64-linux-gnu-debug.tar.gz 89174fb80a0b85b5... b811aaf3785abe08...
*-riscv64-linux-gnu.tar.gz 4fc6e6327cd53b6d... 92a0bc60ddd87c35...
*-win-unsigned.tar.gz 26f6dfbe7455593e... adbbb227f50d7f86...
*-win64-debug.zip 266a6c069f92480d... a0207b0891498436...
*-win64-setup-unsigned.exe b882ae611908201c... 8abd315eb0dbcd87...
*-win64.zip 164abbb8e2202757... 23b255d08df39f09...
*-x86_64-linux-gnu-debug.tar.gz 51bd810a000ac7cd... dd30d48594146909...
*-x86_64-linux-gnu.tar.gz 14d835d0a52c6c9b... df9f814aeaa0f62e...
*.tar.gz 4f55dbaa72aabd90... ac6fd5e8686532bd...
guix_build.log 6c4adc4d9ddc6602... c5fc2117cbc60a27...
guix_build.log.diff 0d71b8d4d587e164...

@DrahtBot
Copy link
Contributor

Gitian builds

File commit 0ab6ff5
(master)
commit e8ac496d81fe10a80b723e4e1f46e1e7d88f44ce
(master and this pull)
*-aarch64-linux-gnu-debug.tar.gz 10442e5275903baa... 257ae749a72eb05c...
*-aarch64-linux-gnu.tar.gz 171a90a906b68348... a32570cf568c3d6b...
*-arm-linux-gnueabihf-debug.tar.gz a075851790a21449... a55435dd2ae95d4d...
*-arm-linux-gnueabihf.tar.gz 8259d1b0596d5ff9... afab063b3fcf4801...
*-osx-unsigned.dmg d6cb4b066ff41405... 552bdc8e10e9b1e1...
*-osx64.tar.gz 186fe3809281e57d... 4f1379bb0ad7946a...
*-powerpc64-linux-gnu-debug.tar.gz a1dffc1c57b5c384... bfbc0d01fa023e50...
*-powerpc64-linux-gnu.tar.gz 6ecee04c0bc2e00b... 1c7576c45e45b041...
*-powerpc64le-linux-gnu-debug.tar.gz 81d6b95b2b25ab28... 23aea234cbb5f15e...
*-powerpc64le-linux-gnu.tar.gz 0d91dad61dcb3d65... 7e0dbdb4bc8b92bd...
*-riscv64-linux-gnu-debug.tar.gz f125c9dc325d2f53... 50e20b2fbe085e66...
*-riscv64-linux-gnu.tar.gz 87508d19dd213953... f349b38176f287e6...
*-win64-debug.zip 343d83b89a2fc0ff... 562a57ae1bddbaf1...
*-win64-setup-unsigned.exe 2fcc54f8972073a1... 02afc1f1325233f9...
*-win64.zip e5d4b95aa9815f17... 7bb379b2f9803830...
*-x86_64-linux-gnu-debug.tar.gz e3078f770897a759... a97f2e2378cea84f...
*-x86_64-linux-gnu.tar.gz 33ae0d9de8e26286... f79fc7418ad87740...
*.tar.gz 7db02c25c2b1172e... a6fb9b71e6a38405...
bitcoin-core-linux-22-res.yml 7a7b99a6ea1b3931... 6e0fa72ab2bfc563...
bitcoin-core-osx-22-res.yml 41fd9d2399c6f2a1... 55101bbe02c87dcf...
bitcoin-core-win-22-res.yml a342ec0348d40731... f38ac47d82d7ad77...
linux-build.log e5298cf8c0180cda... 836023908a9b196f...
osx-build.log 54be9e858aaa62f8... 8bbf5cd7c2a08eb1...
win-build.log 0233da121cadcb26... 701b344c5931765a...
bitcoin-core-linux-22-res.yml.diff 6e0a3331fcc4c8e5...
bitcoin-core-osx-22-res.yml.diff e8c5bc641e38b702...
bitcoin-core-win-22-res.yml.diff d9561bbc20008455...
linux-build.log.diff ef587607efa6976e...
osx-build.log.diff 8c186de35e8afffc...
win-build.log.diff a62670d231961ba7...

@laanwj
Copy link
Member

laanwj commented May 14, 2021

As a future direction I'm wondering if FCF checking should check every function in the program, not just the outer function. This would make sure that all code was built with it enabled not just a single object.

E.g. the only reason we discovered the lack of it in boost is because the test uses it for main.

That said, checking one is enough to know if the gcc flag was enabled, and we do the same kind of spot check for the other checks (like stack protection), so this is good to get started.

@laanwj laanwj merged commit ecf5f2c into bitcoin:master May 14, 2021
sidhujag pushed a commit to syscoin/syscoin that referenced this pull request May 14, 2021
42b589d scripts: test for MACHO control flow instrumentation (fanquake)
469a5bc build: build Boost with -fcf-protection when targeting Darwin (fanquake)

Pull request description:

  Addresses the macOS portion of bitcoin#21888.

  Build Boost with `-fcf-protection` when targeting Darwin. This should be ok, because our cross-compiler (Clang 10) supports the option, and I'd expect all versions of Apple Clang being used to compile Core would also support it. Building Boost with this option is required so that the `main` provided to `test_bitcoin` has instrumentation.

  Note that the presence of instrumentation does not mean it will be used, as that is determined at runtime by the CPU.
  From the Intel control flow enforcement documentation:

  > The ENDBR32 and ENDBR64 instructions will have the same effect as the NOP instruction on Intel 64 processors that do not support CET. On processors supporting CET, these instructions do not change register or flag state. This allows CET instrumented programs to execute on processors that do not support CET. Even when CET is supported and enabled, these NOP–like instructions do not affect the execution state of the program, do not cause any additional register pressure, and are minimally intrusive from power and performance perspectives.

  Follow up from bitcoin#21135.

  Guix builds:
  ```bash
  663df8471400f06d4da739e39a886aa17f56a36d66e0ff7cc290686294ef39c9  guix-build-42b589d18fed/output/dist-archive/bitcoin-42b589d18fed.tar.gz
  45e841661e1659a634468b6f8c9fb0a7956c31ba296f1fd0c02cd880736d6127  guix-build-42b589d18fed/output/x86_64-apple-darwin18/bitcoin-42b589d18fed-osx-unsigned.dmg
  0ea85c99fef35429a5048fa14850bce6b900eaa887aeea419b019852f8d2be78  guix-build-42b589d18fed/output/x86_64-apple-darwin18/bitcoin-42b589d18fed-osx-unsigned.tar.gz
  85857a5a4a5d4d3a172d6c361c12c4a94f6505fc12b527ea63b75bfe54ee1001  guix-build-42b589d18fed/output/x86_64-apple-darwin18/bitcoin-42b589d18fed-osx64.tar.gz
  ```

  Gitian builds:
  ```bash
  # macOS:
  bdfd677a6b88273a741b433e1e7f554af50cc76b3342d44ab0c441e2b40efc96  bitcoin-42b589d18fed-osx-unsigned.dmg
  f3b2d09f3bea7a5cc489b02e8e53dd76a9922338500fae79cad0506655af56f9  bitcoin-42b589d18fed-osx-unsigned.tar.gz
  29d5ad5e46bc9fb0056922a8b47c026e5e9f71e6cf447203b74644587d6fb6f7  bitcoin-42b589d18fed-osx64.tar.gz
  663df8471400f06d4da739e39a886aa17f56a36d66e0ff7cc290686294ef39c9  src/bitcoin-42b589d18fed.tar.gz
  366f8d7a2fc1f3e22cb1018043099126a71ce65380cc27b1c3280cce42d06c98  bitcoin-core-osx-22-res.yml
  ```

ACKs for top commit:
  laanwj:
    Code review ACK 42b589d

Tree-SHA512: 12cb8d462d64d845b9fe48c5c6978892adff8bf5b5572bb29f35df1f6176e47b32a68bcb6e4883c7d9454e76e8868851005a7325916852a2d0d32659ac7dae3f
@fanquake fanquake deleted the build_boost_fcf_protectcion branch May 17, 2021 00:53
gwillen pushed a commit to ElementsProject/elements that referenced this pull request Jun 1, 2022
@bitcoin bitcoin locked as resolved and limited conversation to collaborators Aug 16, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants