[CMake][Release] Build with -ffat-lto-objects#140381
Conversation
|
@llvm/pr-subscribers-clang Author: Tom Stellard (tstellar) ChangesFixes #133580 Full diff: https://github.com/llvm/llvm-project/pull/140381.diff 1 Files Affected:
diff --git a/clang/cmake/caches/Release.cmake b/clang/cmake/caches/Release.cmake
index fb12dfcdcb5a5..592829e01206d 100644
--- a/clang/cmake/caches/Release.cmake
+++ b/clang/cmake/caches/Release.cmake
@@ -102,7 +102,7 @@ if (LLVM_RELEASE_ENABLE_LTO)
# FIXME: We can't use LLVM_ENABLE_LTO=Thin here, because it causes the CMake
# step for the libcxx build to fail. CMAKE_INTERPROCEDURAL_OPTIMIZATION does
# enable ThinLTO, though.
- set(RUNTIMES_CMAKE_ARGS "-DCMAKE_INTERPROCEDURAL_OPTIMIZATION=ON -DLLVM_ENABLE_LLD=ON" CACHE STRING "")
+ set(RUNTIMES_CMAKE_ARGS "-DCMAKE_INTERPROCEDURAL_OPTIMIZATION=ON -DLLVM_ENABLE_LLD=ON -DLLVM_ENABLE_FATLTO=ON" CACHE STRING "")
endif()
# Stage 1 Common Config
@@ -144,3 +144,6 @@ set_final_stage_var(CPACK_GENERATOR "TXZ" STRING)
set_final_stage_var(CPACK_ARCHIVE_THREADS "0" STRING)
set_final_stage_var(LLVM_USE_STATIC_ZSTD "ON" BOOL)
+if (LLVM_RELEASE_ENABLE_LTO)
+ set_final_stage_var(LLVM_ENABLE_FATLTO "ON" BOOL)
+endif()
|
nikic
left a comment
There was a problem hiding this comment.
This is an improvement, but should we also strip the bitcode? Otherwise people will still get errors if they try to use the archives if their host clang version is older.
I think we should strip it. I've been trying to figure out a way to have cmake do this automatically, but I can't get anything to work. We may have to add an extra step to the actions workflow to strip the bitcode. |
|
We can copy the magic of brp-strip-lto :) |
abf0d25 to
24975b0
Compare
|
I figured out how to strip the binaries, so I've updated the patch. Though, you could argue that people should only be using these libraries with the compiler that comes with the release package, and in that case the LTO bitcode would be useful. However, the release package is already huge, so I think overall it's better to strip LTO bitcode and keep the packages smaller. We can always revisit the stripping later. |
| @@ -0,0 +1,5 @@ | |||
| file(GLOB files ${CPACK_TEMPORARY_INSTALL_DIRECTORY}/lib/*.a ${CPACK_TEMPORARY_INSTALL_DIRECTORY}/lib/*.so*) | |||
There was a problem hiding this comment.
I don't think we need *.so here? Those shouldn't contain LTO bitcode, right?
There was a problem hiding this comment.
I checked and they don't, so I've dropped this part.
|
How much slower is this? Last time I tested fatlto in our toolchain it took a VERY long time to link. |
You're probably thinking about "full" LTO, this is fat thin LTO. Our naming is truly amazing. See https://llvm.org/docs/FatLTO.html. It should add about 10% overhead. |
Oh great. In that case this seems fine! |
|
/cherry-pick cff9ae7 |
|
/pull-request #151245 |
…t-lto-objects (#200764) Building with --fat-lto-objects was added in #140381 (cff9ae7). On macOS the tests are failing when building release binaries with many "The file was not recognized as a valid object file" errors, e.g.: 2026-01-16T12:54:26.0928880Z /Users/runner/work/llvm-project/llvm-project/build/tools/clang/stage2-instrumented-bins/tools/clang/stage2-bins/_CPack_Packages/Darwin/TXZ/LLVM-22.1.0-rc1-macOS-ARM64/bin/llvm-strip: error: '/Users/runner/work/llvm-project/llvm-project/build/tools/clang/stage2-instrumented-bins/tools/clang/stage2-bins/_CPack_Packages/Darwin/TXZ/LLVM-22.1.0-rc1-macOS-ARM64/lib/libLLVMAArch64AsmParser.a(AArch64AsmParser.cpp.o)': The file was not recognized as a valid object file It's assuming bitcode is embedded in section .llvm.lto (llvm/lib/Object/ObjectFile.cpp:80) but on MachO it's in __LLVM,__bitcode (llvm/lib/Object/MachOObjectFile.cpp:2184) llvm-bitcode-strip is a driver for the MachO bitcode_strip utility which handles this. Use this instead. Fixes #176398. Assisted-by: codex
…t-lto-objects (#200764) Building with --fat-lto-objects was added in #140381 (cff9ae7). On macOS the tests are failing when building release binaries with many "The file was not recognized as a valid object file" errors, e.g.: 2026-01-16T12:54:26.0928880Z /Users/runner/work/llvm-project/llvm-project/build/tools/clang/stage2-instrumented-bins/tools/clang/stage2-bins/_CPack_Packages/Darwin/TXZ/LLVM-22.1.0-rc1-macOS-ARM64/bin/llvm-strip: error: '/Users/runner/work/llvm-project/llvm-project/build/tools/clang/stage2-instrumented-bins/tools/clang/stage2-bins/_CPack_Packages/Darwin/TXZ/LLVM-22.1.0-rc1-macOS-ARM64/lib/libLLVMAArch64AsmParser.a(AArch64AsmParser.cpp.o)': The file was not recognized as a valid object file It's assuming bitcode is embedded in section .llvm.lto (llvm/lib/Object/ObjectFile.cpp:80) but on MachO it's in __LLVM,__bitcode (llvm/lib/Object/MachOObjectFile.cpp:2184) llvm-bitcode-strip is a driver for the MachO bitcode_strip utility which handles this. Use this instead. Fixes #176398. Assisted-by: codex (cherry picked from commit b8e768f)
…t-lto-objects (llvm#200764) Building with --fat-lto-objects was added in llvm#140381 (cff9ae7). On macOS the tests are failing when building release binaries with many "The file was not recognized as a valid object file" errors, e.g.: 2026-01-16T12:54:26.0928880Z /Users/runner/work/llvm-project/llvm-project/build/tools/clang/stage2-instrumented-bins/tools/clang/stage2-bins/_CPack_Packages/Darwin/TXZ/LLVM-22.1.0-rc1-macOS-ARM64/bin/llvm-strip: error: '/Users/runner/work/llvm-project/llvm-project/build/tools/clang/stage2-instrumented-bins/tools/clang/stage2-bins/_CPack_Packages/Darwin/TXZ/LLVM-22.1.0-rc1-macOS-ARM64/lib/libLLVMAArch64AsmParser.a(AArch64AsmParser.cpp.o)': The file was not recognized as a valid object file It's assuming bitcode is embedded in section .llvm.lto (llvm/lib/Object/ObjectFile.cpp:80) but on MachO it's in __LLVM,__bitcode (llvm/lib/Object/MachOObjectFile.cpp:2184) llvm-bitcode-strip is a driver for the MachO bitcode_strip utility which handles this. Use this instead. Fixes llvm#176398. Assisted-by: codex
…t-lto-objects (llvm#200764) Building with --fat-lto-objects was added in llvm#140381 (cff9ae7). On macOS the tests are failing when building release binaries with many "The file was not recognized as a valid object file" errors, e.g.: 2026-01-16T12:54:26.0928880Z /Users/runner/work/llvm-project/llvm-project/build/tools/clang/stage2-instrumented-bins/tools/clang/stage2-bins/_CPack_Packages/Darwin/TXZ/LLVM-22.1.0-rc1-macOS-ARM64/bin/llvm-strip: error: '/Users/runner/work/llvm-project/llvm-project/build/tools/clang/stage2-instrumented-bins/tools/clang/stage2-bins/_CPack_Packages/Darwin/TXZ/LLVM-22.1.0-rc1-macOS-ARM64/lib/libLLVMAArch64AsmParser.a(AArch64AsmParser.cpp.o)': The file was not recognized as a valid object file It's assuming bitcode is embedded in section .llvm.lto (llvm/lib/Object/ObjectFile.cpp:80) but on MachO it's in __LLVM,__bitcode (llvm/lib/Object/MachOObjectFile.cpp:2184) llvm-bitcode-strip is a driver for the MachO bitcode_strip utility which handles this. Use this instead. Fixes llvm#176398. Assisted-by: codex (cherry picked from commit b8e768f)
Fixes #133580