-
Notifications
You must be signed in to change notification settings - Fork 4.4k
Wrong cc toolchain solib path in rpath for external repos with sibling layout #16956
Description
Description of the bug:
We are setting up bazel for Android Emulator. The cc toolchain we've setup:
- Uses a hermetic clang compiler.
- Enables the
static_link_cpp_runtimesfeature and feeds libc++ to the(dynamic | static)_runtime_libattributes of thecc_toolchainrule.
And due to the layout of the source tree (a top-level "external" directory), we have to pass --experimental_sibling_repository_layout to solve the conflict with external repositories.
With this setup, we find that we are unable to correctly build dynamically linked binaries in external repositories. The rpath for dynamic_runtime_lib (libc++) is set incorrectly. Since cc_test is always linked dynamically, this is prohibiting us to run any cc_test in external repos.
e.g., with the following setup:
- The custom toolchain is located at
@//toolchains/cc:linux_clang_x64 - The binary target is
@zlib//:zlib_test - CPU is
k8
The following rpaths are set by bazel:
❯ objdump -p zlib_test
...
Dynamic Section:
RUNPATH $ORIGIN/_solib_k8/
:$ORIGIN/zlib_test.runfiles/zlib/_solib_k8/
:$ORIGIN/_solib__toolchains_Scc_Clinux_Uclang_Ux64/
:$ORIGIN/zlib_test.runfiles/zlib/_solib__toolchains_Scc_Clinux_Uclang_Ux64/
NEEDED liblibzlib.so
NEEDED libc++.so.1
...
However, what bazel generates in the execroot file tree is:
❯ tree ~/.cache/bazel/_bazel_<username>/<md5>/execroot/__main__/bazel-out/zlib/k8-fastbuild/bin
bin
├── libzlib.a
├── libzlib.so
├── libzlib.so-2.params
├── _solib_k8
│ └── liblibzlib.so -> ~/.cache/bazel/_bazel_<username>/<md5>/execroot/__main__/bazel-out/zlib/k8-fastbuild/bin/libzlib.so
├── zlib_test
├── zlib_test-2.params
├── zlib_test.runfiles
│ ├── __main__
│ │ ├── external
│ │ │ └── zlib
│ │ │ ├── _solib_k8
│ │ │ │ └── liblibzlib.so -> ~/.cache/bazel/_bazel_<username>/<md5>/execroot/__main__/bazel-out/zlib/k8-fastbuild/bin/_solib_k8/liblibzlib.so
│ │ │ └── zlib_test -> ~/.cache/bazel/_bazel_<username>/<md5>/execroot/__main__/bazel-out/zlib/k8-fastbuild/bin/zlib_test
│ │ └── _solib__toolchains_Scc_Clinux_Uclang_Ux64
│ │ ├── libc++.so -> ~/.cache/bazel/_bazel_<username>/<md5>/execroot/__main__/bazel-out/k8-fastbuild/bin/_solib__toolchains_Scc_Clinux_Uclang_Ux64/libc++.so
│ │ └── libc++.so.1 -> ~/.cache/bazel/_bazel_<username>/<md5>/execroot/__main__/bazel-out/k8-fastbuild/bin/_solib__toolchains_Scc_Clinux_Uclang_Ux64/libc++.so.1
│ └── zlib
│ ├── _solib_k8
│ │ └── liblibzlib.so -> ~/.cache/bazel/_bazel_<username>/<md5>/execroot/__main__/bazel-out/zlib/k8-fastbuild/bin/_solib_k8/liblibzlib.so
│ └── zlib_test -> ~/.cache/bazel/_bazel_<username>/<md5>/execroot/__main__/bazel-out/zlib/k8-fastbuild/bin/zlib_test
...
Form the execroot tree, it seems that the correct rpath to find libc++ should be either:
$ORIGIN/../../../k8-fastbuild/bin/_solib__toolchains_Scc_Clinux_Uclang_Ux64, or$ORIGIN/zlib_test.runfiles/__main__/_solib__toolchains_Scc_Clinux_Uclang_Ux64
However bazel generates:
$ORIGIN/_solib__toolchains_Scc_Clinux_Uclang_Ux64$ORIGIN/zlib_test.runfiles/zlib/_solib__toolchains_Scc_Clinux_Uclang_Ux64
What's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.
- Create a toolchain config and a toolchain that feeds libs into the
dynamic_runtime_libattribute, and enable featurestatic_link_cpp_runtimes. - Define an external repo and create a cc binary target in it.
- Turn on
--experimental_sibling_repository_layoutflag and build the target - Inspect the created binary or the linking command to check rpath.
Which operating system are you running Bazel on?
Linux
What is the output of bazel info release?
release 7.0.0-pre.20221111.3
If bazel info release returns development version or (@non-git), tell us how you built Bazel.
No response
What's the output of git remote get-url origin; git rev-parse master; git rev-parse HEAD ?
No response
Have you found anything relevant by searching the web?
No.
Any other information, logs, or outputs that you want to share?
No response