cmake: rework LLVM/Clang cmake by MaskRay · Pull Request #3149 · ziglang/zig

MaskRay · 2019-09-01T08:04:07Z

Use {LLVM,CLANG}_INCLUDE_DIRS from {LLVM,Clang}Config.cmake
This makes it possible to use an LLVM/Clang source tree.
Delete ZIG_STATIC_LLVM cmake: allow user to select static vs dynamic LLVM #2858

Building both libLLVM*.a and libLLVM*.so is unsupported in LLVM. The
LLVM libraries can be configured in 3 ways:

-DLLVM_LINK_LLVM_DYLIB=ON => link against libLLVM.so
-DLLVM_LINK_LLVM_DYLIB=OFF -DBUILD_SHARED_LIBS=OFF => link against libLLVM*.a
-DLLVM_LINK_LLVM_DYLIB=OFF -DBUILD_SHARED_LIBS=ON => link against libLLVM*.so

If libLLVM*.a are unavailable, ZIG_STATIC_LLVM will not work.
If libLLVM*.a are available, linking against libLLVM*.a is automatic.
So ZIG_STATIC_LLVM is not useful and can only be confusing.

If CLANG_LINK_CLANG_DYLIB (available since clang 9) is ON,
libclang-cpp.so instead of libclang*.so will be linked. It currently
does not work because zig_cpp/libembedded_lld_lib.a has global
constructors that conflict with libclang-cpp.so

Succeeded to build zig with

releases.llvm.org clang+llvm 8 prebuilt archive -DBUILD_SHARED_LIBS=off
prereleases.llvm.org/ clang+llvm 9 sources -DBUILD_SHARED_LIBS=off
prereleases.llvm.org/ clang+llvm 9 sources -DBUILD_SHARED_LIBS=off -DLLVM_LINK_LLVM_DYLIB=on -DCLANG_LINK_CLANG_DYLIB=on
Succeeded to link zig0 but hit the conflicting global constructor issue when running zig0.
clang+llvm git master -DBUILD_SHARED_LIBS=on if I remove version requirement

daurnimator · 2019-09-01T08:05:54Z

cmake/Findclang.cmake

I don't think this option is supported by clang < 9. Perhaps target this PR at the zig llvm9 branch?

No. Users have to specify -DCLANG_LINK_CLANG_DYLIB explicitly to enable this code path. The code is here so that:

after the global constructor conflict caused by the embedded lld is fixed

after the upstream exports CLANG_LINK_CLANG_DYLIB as cmake variable

This configuration can work automatically.

MaskRay · 2019-09-01T08:19:00Z

https://dev.azure.com/ziglang/zig/_build/results?buildId=3140

CMake Error at /usr/lib/llvm-6.0/lib/cmake/clang/ClangTargets.cmake:490 (message):

The build configuration needs something like -DCMAKE_PREFIX_PATH=/usr/lib/llvm-8.0

Or probably uninstall libclang-6-dev

cmake/Findlld.cmake is not touched in this patch. The upstream does not provide LLDConfig.cmake.

In llvm/MC/MCTargetOptionsCommandFlags.inc:

static cl::opt<bool> RelaxAll("mc-relax-all",
                       cl::desc("When used with filetype=obj, "
                                "relax all fixups in the emitted object file"));

Both libclang-cpp.so and libembedded_lld_lib.a have this global constructor.

% readelf -s zig_cpp/libembedded_lld_lib.a | grep _ZL8RelaxAll
   150: 0000000000000000   160 OBJECT  LOCAL  DEFAULT 1615 _ZL8RelaxAll
% readelf -s ~/Dev/llvm-9.0.0rc2/Dylib/lib/libclang-cpp.so | grep _ZL8RelaxAll
 23221: 00000000025e7120   160 OBJECT  LOCAL  DEFAULT   24 _ZL8RelaxAll
% ./zig0 
: CommandLine Error: Option 'mc-relax-all' registered more than once!
LLVM ERROR: inconsistency in registered CommandLine options

The problem is that embedded_lld_lib is built as a static archive:

add_library(embedded_lld_lib STATIC ${EMBEDDED_LLD_LIB_SOURCES})

andrewrk · 2019-09-01T15:48:33Z

What problem are you trying to solve?

andrewrk · 2019-09-01T15:50:27Z

CMakeLists.txt

The main point of this cmake file and the stage1 source code is to bootstrap from C++. So supporting a wide range of C++ compiler versions and cmake versions is part what this accomplishes. Requiring a significantly newer cmake is a big problem.

1. Use {LLVM,CLANG}_INCLUDE_DIRS from {LLVM,Clang}Config.cmake This makes it possible to use an LLVM/Clang source tree. 2. Delete ZIG_STATIC_LLVM Building both libLLVM*.a and libLLVM*.so is unsupported in LLVM. The LLVM libraries can be configured in 3 ways: * -DLLVM_LINK_LLVM_DYLIB=ON => link against libLLVM.so * -DLLVM_LINK_LLVM_DYLIB=OFF -DBUILD_SHARED_LIBS=OFF => link against libLLVM*.a * -DLLVM_LINK_LLVM_DYLIB=OFF -DBUILD_SHARED_LIBS=ON => link against libLLVM*.so If libLLVM*.a are unavailable, ZIG_STATIC_LLVM will not work. If libLLVM*.a are available, linking against libLLVM*.a is automatic. So ZIG_STATIC_LLVM is not useful and can only be confusing. 3. If CLANG_LINK_CLANG_DYLIB (available since clang 9) is ON, libclang-cpp.so instead of libclang*.so will be linked. It currently does not work because zig_cpp/libembedded_lld_lib.a has global constructors that conflict with libclang-cpp.so

MaskRay · 2019-09-01T16:56:54Z

What problem are you trying to solve?

This makes it possible to use an LLVM/Clang source tree.

Zig cannot be built by using a LLVM/Clang source tree.

Delete ZIG_STATIC_LLVM which does not make lots of sense.

andrewrk · 2019-09-01T21:28:10Z

What is a LLVM/clang source tree?

MaskRay · 2019-09-02T03:15:43Z

What is a LLVM/clang source tree?

Point -DCMAKE_PREFIX_PATH to a LLVM/Clang source tree where ninja install has not been executed.

% cd ~/Dev/llvm-9.0.0rc2
# Unpack cfe-9.0.0rc2.src.tar.xz at tools/clang
% cmake -GNinja -H. -BRelease -DCMAKE_BUILD_TYPE=Release
% ninja -C Release all
# In Release/lib/cmake/clang/ClangConfig.cmake, set(CLANG_INCLUDE_DIRS "/home/user/Dev/llvm-9.0.0rc2/tools/clang/include;/home/user/Dev/llvm-9.0.0rc2/Release/tools/clang/include")
% ~/Dev/llvm-9.0.0rc2/Release/bin/llvm-config --includedir
/home/user/Dev/llvm-9.0.0rc2/include
# llvm-config --includedir output does not include Release/tools/clang/include, so it is not suitable for use


% cd ~/Dev/zig
% LLVM=~/Dev/llvm-9.0.0rc2
% cmake -H. -BRelease -G Ninja -DCMAKE_PREFIX_PATH="$LLVM/Release;$LLVM/Release/tools/clang;$LLVM;$LLVM/tools/clang"
...
fatal error: llvm/Config/llvm-config.h: No such file or directory
#include "llvm/Config/llvm-config.h"

I know you'll call it an "unsupported" configuration, but this basically makes it very difficult to debug if you suspect there are bugs in LLVM/Clang.

shawnl · 2019-09-17T11:58:19Z

@andrewrk

What is a LLVM/clang source tree?

https://github.com/llvm/llvm-project

What problem are you trying to solve?

With this it becomes easier to bisect problems with llvm (or less likely the use of libclang). This would mainly only be useful on the llvm-9/truck branches.

XVilka · 2019-09-24T04:39:59Z

What about adding these two different types of the builds in the CI? So if any will be broken the problem will be immediately caught.

shawnl · 2019-09-24T07:46:22Z

Yes, but a different CI run so our already long runs don't get longer. El mar., 24 sept. 2019 0:40, Anton Kochkov <notifications@github.com> escribió:

…

What about adding these two different types of the builds in the CI? So if any will be broken the problem will be immediately caught. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#3149?email_source=notifications&email_token=AAD4W4ULX6WPRAUDHNK4RYDQLGK2PA5CNFSM4ISV72TKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD7NBGJY#issuecomment-534385447>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAD4W4SO2DSURFYDKDIIC6LQLGK2PANCNFSM4ISV72TA> .

shawnl · 2019-11-07T18:50:42Z

I am still interested in this. turns out zig built against llvm 9 can't link against llvm 9.0.1

undefined symbol: _ZTIN4llvm17DiagnosticHandlerE, version LLVM_9

andrewrk · 2019-11-26T04:27:56Z

Sorry, I'm not willing to support this use case. For this use case (working on LLVM itself, testing using the Zig frontend), I suggest a third-party, community-maintained project which is a fork of Zig with modified build system to tightly couple with LLVM's build system. But I am not willing to maintain such a project in the ziglang/zig repository.

MaskRay · 2019-11-26T04:50:53Z

One would not say this if they looked at the clean-up of the build system. Though, it was your choice, and I respect that. Bye.

andrewrk · 2019-11-26T05:03:55Z

I certainly looked at it. It does look cleaner, but supports less system environments than status quo, mainly because it introduces a cmake dependency on a Clang cmake file, which is not always available. That's also not a viable way for the self-hosted compiler to depend on libclang.

But the bottom line is that the thing you're trying to do, is too much of a burden for me and other zig team members to maintain in this repository. I humbly ask for this maintenance work to be done externally.

If your goal was simply to fix the build for a particular system, in which the build was failing, I would be of course be happy to merge such changes.

daurnimator suggested changes Sep 1, 2019

View reviewed changes

MaskRay force-pushed the cmake branch from c416c79 to 10c54a8 Compare September 1, 2019 09:13

andrewrk requested changes Sep 1, 2019

View reviewed changes

MaskRay force-pushed the cmake branch from 10c54a8 to 681ec91 Compare September 1, 2019 16:55

andrewrk added this to the 0.6.0 milestone Sep 3, 2019

andrewrk removed this from the 0.6.0 milestone Oct 1, 2019

andrewrk closed this Nov 26, 2019

MaskRay deleted the cmake branch November 26, 2019 04:50

andrewrk mentioned this pull request Apr 10, 2020

rework llvm and clang detection in cmake configuration #4992

Closed

Uh oh!

Conversation

MaskRay commented Sep 1, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

daurnimator Sep 1, 2019

Choose a reason for hiding this comment

Uh oh!

MaskRay Sep 1, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

MaskRay commented Sep 1, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

andrewrk commented Sep 1, 2019

Uh oh!

andrewrk Sep 1, 2019

Choose a reason for hiding this comment

Uh oh!

MaskRay commented Sep 1, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

andrewrk commented Sep 1, 2019

Uh oh!

MaskRay commented Sep 2, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

shawnl commented Sep 17, 2019

Uh oh!

XVilka commented Sep 24, 2019

Uh oh!

shawnl commented Sep 24, 2019 via email

Uh oh!

shawnl commented Nov 7, 2019

Uh oh!

andrewrk commented Nov 26, 2019

Uh oh!

MaskRay commented Nov 26, 2019

Uh oh!

andrewrk commented Nov 26, 2019

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

MaskRay commented Sep 1, 2019 •

edited

Loading

MaskRay Sep 1, 2019 •

edited

Loading

MaskRay commented Sep 1, 2019 •

edited

Loading

MaskRay commented Sep 1, 2019 •

edited

Loading

MaskRay commented Sep 2, 2019 •

edited

Loading