configure_make: provide (DY)LD_LIBRARY_PATH based on the dependencies by chouquette · Pull Request #1452 · bazel-contrib/rules_foreign_cc

chouquette · 2025-12-08T15:34:46Z

Some software might require to be able to load their dependencies at build time.

In my case, python3 will attempt to load the freshly built modules to ensure they are functional. If the dependency can't be found, the module is either deactivated or the build fails, depending on the module.

This change will automatically populate LD_LIBRARY_PATH on linux & freebsb, DYLD_LIBRARY_PATH on macOS, and resorts to no-op for windows.

I'm not sure about the windows flavors, besides the fact that not returning a string from increment_ld_library_path would cause this error:

Traceback (most recent call last):
	File "C:/bzl/bazel/d42blgaa/external/rules_foreign_cc+/foreign_cc/configure.bzl", line 70, column 33, in _configure_make
		return cc_external_rule_impl(ctx, attrs)
	File "C:/bzl/bazel/d42blgaa/external/rules_foreign_cc+/foreign_cc/private/framework.bzl", line 513, column 29, in cc_external_rule_impl
		convert_shell_script(ctx, script_lines),
	File "C:/bzl/bazel/d42blgaa/external/rules_foreign_cc+/foreign_cc/private/framework/helpers.bzl", line 74, column 43, in convert_shell_script
		return convert_shell_script_by_context(create_context(ctx), script)
	File "C:/bzl/bazel/d42blgaa/external/rules_foreign_cc+/foreign_cc/private/framework/helpers.bzl", line 90, column 31, in convert_shell_script_by_context
		script = [do_function_call(line, shell_context) for line in script]
	File "C:/bzl/bazel/d42blgaa/external/rules_foreign_cc+/foreign_cc/private/framework/helpers.bzl", line 186, column 46, in do_function_call
		(funname, after) = get_function_name(text.strip(" "))
Error: 'NoneType' value has no field or method 'strip'

chouquette · 2026-01-15T10:21:24Z

Hello 👋
Gentle ping for review :)

Switch rules_foreign_cc from single_version_override on 0.15.1 to a git_override on main (e4068330) to pick up three upstream fixes: - bazel-contrib/rules_foreign_cc#1451: out_data_dirs in output groups (our patch 0001 verbatim — drop it) - bazel-contrib/rules_foreign_cc#1465: resource_set integration, which lets Bazel's scheduler respect CPU/RAM budgets and avoids build action overcommitting - bazel-contrib/rules_foreign_cc#1470: forward -isystem flags to configure scripts Patch 0002 (LD_LIBRARY_PATH propagation) is kept locally as bazel-contrib/rules_foreign_cc#1452 is still open. bazel-contrib/rules_foreign_cc#1462 bumps bazel_lib to 3.2.0; follow up to 3.2.2 to get path-mapping compatibility with Bazel 9. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Switch rules_foreign_cc from single_version_override on 0.15.1 to a git_override on main (e4068330) to pick up three upstream fixes: - bazel-contrib/rules_foreign_cc#1451: out_data_dirs in output groups (our patch 0001 verbatim — drop it) - bazel-contrib/rules_foreign_cc#1465: resource_set integration, which lets Bazel's scheduler respect CPU/RAM budgets and avoids build action overcommitting - bazel-contrib/rules_foreign_cc#1470: forward -isystem flags to configure scripts Patch 0002 (LD_LIBRARY_PATH propagation) is kept locally as bazel-contrib/rules_foreign_cc#1452 is still open. bazel-contrib/rules_foreign_cc#1462 bumps bazel_lib to 3.2.0; follow up to 3.2.2 to get path-mapping compatibility with Bazel 9.

### What does this PR do? Switch `rules_foreign_cc` from `single_version_override` on 0.15.1 to a `git_override` on `main`. Implies to bump `bazel_lib` to 3.2+ (see bazel-contrib/rules_foreign_cc#1462). ### Motivation Pick up three upstream fixes: 1. bazel-contrib/rules_foreign_cc#1451 (allows to **drop our corresponding patch**), 2. bazel-contrib/rules_foreign_cc#1465 (avoids build action **over-committing**), 3. bazel-contrib/rules_foreign_cc#1470. ### Additional Notes Patch 0002 (LD_LIBRARY_PATH propagation) is kept locally as bazel-contrib/rules_foreign_cc#1452 is still open. Co-authored-by: regis.desgroppes <regis.desgroppes@datadoghq.com>

JSGette · 2026-03-16T09:02:43Z

Gentle reminder

### What does this PR do? Switch `rules_foreign_cc` from `single_version_override` on 0.15.1 to a `git_override` on `main`. Implies to bump `bazel_lib` to 3.2+ (see bazel-contrib/rules_foreign_cc#1462). ### Motivation Pick up three upstream fixes: 1. bazel-contrib/rules_foreign_cc#1451 (allows to **drop our corresponding patch**), 2. bazel-contrib/rules_foreign_cc#1465 (avoids build action **over-committing**), 3. bazel-contrib/rules_foreign_cc#1470. ### Additional Notes Patch 0002 (LD_LIBRARY_PATH propagation) is kept locally as bazel-contrib/rules_foreign_cc#1452 is still open. Co-authored-by: regis.desgroppes <regis.desgroppes@datadoghq.com>

Copilot

Pull request overview

This PR extends the foreign_cc configure+make script generation to automatically populate the platform-specific dynamic library search path environment variable from the external build dependencies so freshly-built binaries/modules can load their dependent shared libraries at build time.

Changes:

Added an ldpath_script() fragment and injected it into create_configure_script() to set up (DY)LD_LIBRARY_PATH from EXT_BUILD_DEPS.
Introduced new shell-toolchain commands: increment_ld_library_path and export_ld_library_path_var, with implementations for Linux, FreeBSD, macOS, and no-op stubs for Windows.
Registered the new commands in the toolchain command registry (PLATFORM_COMMANDS).

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 6 comments.

Show a summary per file

File	Description
foreign_cc/private/make_script.bzl	Adds `ldpath_script()` helper that emits toolchain command markers for ld path setup.
foreign_cc/private/configure_script.bzl	Injects `ldpath_script()` into the generated configure script.
foreign_cc/private/framework/toolchains/commands.bzl	Registers new `export_ld_library_path_var` and `increment_ld_library_path` commands.
foreign_cc/private/framework/toolchains/linux_commands.bzl	Implements LD_LIBRARY_PATH export + increment logic for Linux.
foreign_cc/private/framework/toolchains/freebsd_commands.bzl	Implements LD_LIBRARY_PATH export + increment logic for FreeBSD.
foreign_cc/private/framework/toolchains/macos_commands.bzl	Implements DYLD_LIBRARY_PATH export + increment logic for macOS.
foreign_cc/private/framework/toolchains/windows_commands.bzl	Adds no-op implementations to avoid conversion errors on Windows.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

foreign_cc/private/framework/toolchains/linux_commands.bzl

+    local children=$(find "$1" -name '*.so')
+    LD_LIBRARY_PATH=""
+    for child in $children; do
+      export LD_LIBRARY_PATH="$${LD_LIBRARY_PATH:-}$$:$(dirname $child)"
+    done


foreign_cc/private/framework/toolchains/linux_commands.bzl


+def increment_ld_library_path(_source):
+    text = """\
+    local children=$(find "$1" -name '*.so')


foreign_cc/private/framework/toolchains/freebsd_commands.bzl

+    local children=$(find "$1" -name '*.so')
+    LD_LIBRARY_PATH=""
+    for child in $children; do
+      export LD_LIBRARY_PATH="$${LD_LIBRARY_PATH:-}$$:$(dirname $child)"
+    done


foreign_cc/private/framework/toolchains/freebsd_commands.bzl


+def increment_ld_library_path(_source):
+    text = """\
+    local children=$(find "$1" -name '*.so')


foreign_cc/private/framework/toolchains/macos_commands.bzl

+def increment_ld_library_path(_source):
+    text = """\
+    local children=$(find "$1" -name '*.dylib')
+    for child in $children; do
+      export DYLD_LIBRARY_PATH="$${DYLD_LIBRARY_PATH:-}$$:$(dirname $child)"
+    done


foreign_cc/private/configure_script.bzl

    script = pkgconfig_script(ext_build_dirs)
+    script.extend(ldpath_script())



novas0x2a · 2026-03-19T06:17:52Z

Note that the gentle reminders unfortunately do not actually show up anywhere for me or the other maintainers, sorry about that. I've just become a maintainer recently.

Could you add some sort of integration test that uses this in practice and would fail without it? (I unfortunately think the EXT_BUILD_DEPS linking approach isn't that great of an approach, and I want to start looking into doing something about it, but without an integration test that relies on the sort of library search behavior that you wanted, it would be hard to preserve the behavior).

I'd also say that maybe this should not be enabled by default, because LD_LIBRARY_PATH affects both the build tools and the actually-built targets, and needing to set LD_LIBRARY_PATH during a build almost always represents a bug in the build system itself. One of the more insidious ways this can show up is this pseudocode that represents a real scenario we ran into:

cmake(name = "curl", deps = ["openssl1"])
cmake(name = "mytarget", deps = ["curl"])

The trap is:

the cmake binary depends on curl which depends on openssl3
mytarget depends on curl which depends on openssl1

This is going to cause the cmake instance to load the wrong curl and openssl, which will cause random crashes (depending on what functions actually get called).

06393993 · 2026-03-19T16:15:34Z

I have one question on this PR, does the change works for cross compilation? It seems to add the shared object of deps to the dynamic library search path instead of the build_data's runfiles:

def ldpath_script():
    script = []
    script.append("##increment_ld_library_path## $$EXT_BUILD_DEPS$$/")
    script.append("##export_ld_library_path_var##")
    return script

IIUC, $$EXT_BUILD_DEPS$$ contains the deps, which are binaries of the target platform (in bazel's term, or host platform in GNU Autotool's term), and can be different from the execution platform (in bazel's term, or build platform in GNU Autotool's term).

Depends on #48082. ### Motivation Moving the unreleased tip of main allows to address Bazel 9 related issues instead of patching locally: - bazel-contrib/rules_foreign_cc#1493 - bazel-contrib/rules_foreign_cc#1492 (`CcInfo` and other `Cc*` symbols) Other notable commits since: - bazel-contrib/rules_foreign_cc#1483 - bazel-contrib/rules_foreign_cc#1490 - bazel-contrib/rules_foreign_cc#1496 ### Additional Notes Patches 0002 (bazel-contrib/rules_foreign_cc#1452) and 0003 (bazel-contrib/rules_foreign_cc#1491) are still carried locally as neither has landed upstream yet (but we're working on it). Co-authored-by: regis.desgroppes <regis.desgroppes@datadoghq.com>

chouquette added 3 commits December 8, 2025 16:33

configure_make: provide (DY)LD_LIBRARY_PATH based on the dependencies

4d48a36

buildifier

e4b312a

Empty commit for CI

00206b0

chouquette mentioned this pull request Jan 9, 2026

Update rules_foreign_cc patch set DataDog/datadog-agent#43893

Merged

rdesgroppes mentioned this pull request Mar 12, 2026

Bump rules_foreign_cc for 3 improvements DataDog/datadog-agent#47799

Merged

novas0x2a requested a review from Copilot March 19, 2026 06:03

Copilot AI reviewed Mar 19, 2026

View reviewed changes

novas0x2a self-requested a review March 19, 2026 06:24

rdesgroppes mentioned this pull request Mar 21, 2026

Bump rules_foreign_cc for Bazel 9 fixes DataDog/datadog-agent#48071

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

configure_make: provide (DY)LD_LIBRARY_PATH based on the dependencies#1452

configure_make: provide (DY)LD_LIBRARY_PATH based on the dependencies#1452
chouquette wants to merge 3 commits intobazel-contrib:mainfrom
chouquette:chouquette/ld_library_path

chouquette commented Dec 8, 2025

Uh oh!

chouquette commented Jan 15, 2026

Uh oh!

JSGette commented Mar 16, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

novas0x2a commented Mar 19, 2026

Uh oh!

06393993 commented Mar 19, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

		script = pkgconfig_script(ext_build_dirs)
		script.extend(ldpath_script())

Conversation

chouquette commented Dec 8, 2025

Uh oh!

chouquette commented Jan 15, 2026

Uh oh!

JSGette commented Mar 16, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

novas0x2a commented Mar 19, 2026

Uh oh!

06393993 commented Mar 19, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants