Skip to content

Fix static linking with alwayslink#2584

Open
cstrahan wants to merge 6 commits intobazel-contrib:release-0.23from
cstrahan:fix-static-linking
Open

Fix static linking with alwayslink#2584
cstrahan wants to merge 6 commits intobazel-contrib:release-0.23from
cstrahan:fix-static-linking

Conversation

@cstrahan
Copy link
Copy Markdown

@cstrahan cstrahan commented Jul 24, 2020

What type of PR is this?

Bug fix

What does this PR do? Why is it needed?

Fixes static linking against libraries that have alwayslink specified (where the linker must be directed to link the whole archive).

Which issues(s) does this PR fix?

Fixes #1486

Other notes for review

N/A


I'd be more than happy to perform any clean up / refactoring / bug fixes necessary to get this merged in. I'm also willing to block out time at your convenience to talk design/implementation in real time (whether text chat or video conferencing). Please advise.

Copy link
Copy Markdown
Collaborator

@jayconrod jayconrod left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A quick initial look, but I'll be on vacation next week and won't be able to review again for a while.

Also related is #2294, though I think that PR is now defunct.

"darwin": ("-Wl,-force_load", ""),
"js": ("", ""),
"linux": ("-Wl,--whole-archive", "-Wl,--no-whole-archive"),
"windows": ("-Wl,--whole-archive", "-Wl,--no-whole-archive"),
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why are these flags OS-specific? I think the assumption here is that clang is always used on darwin, and gcc is always used on linux and windows, but that's not a safe assumption.

Is there any way you can get the right flags out of cc_common from the toolchain? If not, I think it would be better to look at the name of the compiler or linker binary to guess which flags will be accepted.

Why is "js" included here by the way? What toolchain is expected there?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why are these flags OS-specific? I think the assumption here is that clang is always used on darwin, and gcc is always used on linux and windows, but that's not a safe assumption.
[...]
Why is "js" included here by the way? What toolchain is expected there?

I cribbed that from #1879.

Is there any way you can get the right flags out of cc_common from the toolchain? If not, I think it would be better to look at the name of the compiler or linker binary to guess which flags will be accepted.

If there is a way to do that, that would be great; I have no idea, though. There doesn't appear to be any sort of sort of sanctioned forum to ask the devs and others in the know, and I have no ties with Google or the Bazel devs, so I don't know how to inquire. Any advice here would be great.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see anything directly in cc_common that would help.

I think a "good enough" solution would be to check go.env["CC"], which will be the CC environment variable passed to actions. If the string after the last slash contains clang, let's use the clang flags. Otherwise, let's use the gcc flags. I'm not 100% sure that will always be set, so let's assume gcc if it's not.

(For asking questions by the way, the bazel-discuss mailing list is usually pretty good. There's also a public Slack, though I'm not sure if it's still active).

inputs_direct.append(lib)
else:
if library_to_link.alwayslink:
lib_opts.append(_WHOLE_ARCHIVE_FLAGS_BY_GOOS[go.mode.goos][0])
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will crash if go.mode.goos isn't in the dict above.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a good point. I'd love to scrap _WHOLE_ARCHIVE_FLAGS_BY_GOOS and replace this code with something more robust, though I don't (yet) know how I would do that.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, let's replace _WHOLE_ARCHIVE_FLAGS_BY_GOOS with a function:

def _library_args(go, library_to_link):

If alwayslink is false, it can just return a list with the file path. Otherwise, it can return a list with the file path surrounded by the appropriate flags. It can test go.env["CC"] to decide which flags to use.

return libs

def _cc_libs(library_to_link):
libs = []
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why return a list from this function instead of a single value?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why return a list from this function instead of a single value?

When I extracted that from the original source, I misread the code -- I thought I had read a set of four distinct if conditions (implying that library_to_link might have multiple files that need to be handled), not one chain of if ... elif. I'll change that.

cc_transitive_headers = d[CcInfo].compilation_context.headers
inputs_transitive.append(cc_transitive_headers)
cc_libs = _cc_libs(d)
cc_libs = _cc_libs_for_target(d)
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This list is now distinct from the libraries returned by _cc_libs, but the names are really similar. What's the distinction? Won't there be an error if there are differences?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The old _cc_libs fished out the archive/dso path from each LibraryToLink in d, giving back a list of those paths. At that point, we no longer have LibraryToLink.alwayslink to inspect.

So, as the code stands now:

  • _cc_libs(library_to_link): gets the archive/dso path for the given library (currently as singleton list, but as you pointed out, this ought to just return the path itself).
  • _cc_libs_for_target(target): returns the list of all lib paths for a given target (as _cc_libs used to).
  • _cc_libraries_to_link(target): returns the list of all LibraryToLinks for a given target.

The names are terrible, I will admit. Ultimately, my changes were intended to address the fact I need the respective LibraryToLink so I can inspect alwayslink.

With that said, I noticed something odd about the code before I got involved (my notes as comments):

            
            # cc_libs has the path of all libs for target `d`
            cc_libs = _cc_libs(d)
            # the existing implementation immediately extends inputs_direct and deps_direct with cc_libs
            inputs_direct.extend(cc_libs)
            deps_direct.extend(cc_libs)
            # [... elided ...]
            for lib in cc_libs:
                if (lib.basename.startswith("lib") and
                    libname = lib.basename[len("lib"):lib.basename.rindex(".")]
                    clinkopts.extend(["-L", lib.dirname, "-l", libname])
                    inputs_direct.append(lib) # <===== isn't this superfluous, since we extended inputs_direct with all of cc_libs above???
                elif (lib.basename.startswith("lib") and
                      has_versioned_shared_lib_extension(lib.basename)):
                    libname = ":%s" % lib.basename
                    clinkopts.extend(["-L", lib.dirname, "-l", libname])
                    inputs_direct.append(lib) # <===== ditto
                else:
                    lib_opts.append(lib.path)

Assuming I'm tracking how the code is intended to work, my next commit (which I will push shortly) will greatly clean things up.

@cstrahan
Copy link
Copy Markdown
Author

My last commit (50acc1a) attempts to clean things up a bit, as I described earlier. How does that look?

@cstrahan
Copy link
Copy Markdown
Author

Sorry, day is getting late and my blood sugar is low. Made a bunch of mistakes in 50acc1a that I fixed in 97e8445.

@cstrahan
Copy link
Copy Markdown
Author

@jayconrod Any suggestions for how to move this along? I'd love to fix the remaining problems and get this fixed.

@cstrahan
Copy link
Copy Markdown
Author

cstrahan commented Aug 24, 2020

@jayconrod I tried implementing your suggestions, as you can see in this diff:

diff --git a/go/private/rules/cgo.bzl b/go/private/rules/cgo.bzl
index 863d7240..0407d216 100644
--- a/go/private/rules/cgo.bzl
+++ b/go/private/rules/cgo.bzl
@@ -31,13 +31,6 @@ load(
     "cc_library",
 )
 
-_WHOLE_ARCHIVE_FLAGS_BY_GOOS = {
-    "darwin": ("-Wl,-force_load", ""),
-    "js": ("", ""),
-    "linux": ("-Wl,--whole-archive", "-Wl,--no-whole-archive"),
-    "windows": ("-Wl,--whole-archive", "-Wl,--no-whole-archive"),
-}
-
 def cgo_configure(go, srcs, cdeps, cppopts, copts, cxxopts, clinkopts):
     """cgo_configure returns the inputs and compile / link options
     that are required to build a cgo archive.
@@ -147,11 +140,7 @@ def cgo_configure(go, srcs, cdeps, cppopts, copts, cxxopts, clinkopts):
                         libname = ":%s" % lib_file.basename
                         clinkopts.extend(["-L", lib_file.dirname, "-l", libname])
                     else:
-                        if library_to_link.alwayslink:
-                            lib_opts.append(_WHOLE_ARCHIVE_FLAGS_BY_GOOS[go.mode.goos][0])
-                        lib_opts.append(lib_file.path)
-                        if library_to_link.alwayslink:
-                            lib_opts.append(_WHOLE_ARCHIVE_FLAGS_BY_GOOS[go.mode.goos][1])
+                        lib_opts.extend(_library_args(go, lib_file, library_to_link.alwayslink))
             cc_link_flags = d[CcInfo].linking_context.user_link_flags
             clinkopts.extend(cc_link_flags)
 
@@ -196,7 +185,7 @@ def _cc_libraries_to_link(target):
     libraries_to_link = as_iterable(target[CcInfo].linking_context.libraries_to_link)
     return libraries_to_link
 
-# Returns the library path to link for the given LibraryToLink.
+# Returns the library File to link for the given LibraryToLink.
 def _cc_lib_file(library_to_link):
     if library_to_link.static_library != None:
         return library_to_link.static_library
@@ -208,6 +197,17 @@ def _cc_lib_file(library_to_link):
         return library_to_link.dynamic_library
     return None
 
+# Returns the linker flags to link the given static library File.
+def _library_args(go, lib_file, alwayslink):
+    if not alwayslink:
+        return [lib_file.path]
+
+    cc_basename = go.env["CC"].rpartition("/")[-1]
+    if cc_basename == "clang":
+        return ["-Wl,-force_load", lib_file.path]
+    else:
+        return ["-Wl,--whole-archive", lib_file.path, "-Wl,--no-whole-archive"]
+
 _DEFAULT_PLATFORM_COPTS = select({
     "@io_bazel_rules_go//go/platform:darwin": [],
     "@io_bazel_rules_go//go/platform:windows_amd64": ["-mthreads"],

But I run into this error when I try to use it:

ERROR: /home/cstrahan/src/rules_go_repro/BUILD:34:1: in go_binary rule //:demogo: 
Traceback (most recent call last):
	File "/home/cstrahan/src/rules_go_repro/BUILD", line 34
		go_binary(name = 'demogo')
	File "/home/cstrahan/.cache/bazel/_bazel_cstrahan/548f10c1937b9439c20336a9e6e4ebdc/external/io_bazel_rules_go/go/private/rules/binary.bzl", line 56, in _go_binary_impl
		go.binary(go, name = name, <5 more arguments>)
	File "/home/cstrahan/.cache/bazel/_bazel_cstrahan/548f10c1937b9439c20336a9e6e4ebdc/external/io_bazel_rules_go/go/private/actions/binary.bzl", line 41, in go.binary
		go.archive(go, source)
	File "/home/cstrahan/.cache/bazel/_bazel_cstrahan/548f10c1937b9439c20336a9e6e4ebdc/external/io_bazel_rules_go/go/private/actions/archive.bzl", line 85, in go.archive
		cgo_configure(go, <6 more arguments>)
	File "/home/cstrahan/.cache/bazel/_bazel_cstrahan/548f10c1937b9439c20336a9e6e4ebdc/external/io_bazel_rules_go/go/private/rules/cgo.bzl", line 143, in cgo_configure
		lib_opts.extend(<1 more arguments>)
	File "/home/cstrahan/.cache/bazel/_bazel_cstrahan/548f10c1937b9439c20336a9e6e4ebdc/external/io_bazel_rules_go/go/private/rules/cgo.bzl", line 143, in lib_opts.extend
		_library_args(go, lib_file, <1 more arguments>)
	File "/home/cstrahan/.cache/bazel/_bazel_cstrahan/548f10c1937b9439c20336a9e6e4ebdc/external/io_bazel_rules_go/go/private/rules/cgo.bzl", line 205, in _library_args
		go.env["CC"]
key "CC" not found in dictionary
ERROR: Analysis of target '//:demogo' failed; build aborted: Analysis of target '//:demogo' failed; build aborted
INFO: Elapsed time: 0.919s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (12 packages loaded, 6547 targets configured)

Is there something I'm doing wrong?

@jayconrod
Copy link
Copy Markdown
Collaborator

Just to set expectations, I probably won't have time to debug this issue this week, so I can't offer direct advice on what to do at the moment.

The C compiler path should be available somewhere within the go object. I'd expect go.env["CC"] to always be set when cgo is enabled and a C compiler is configured, but it sounds like that might not be the case here. go.cgo_tools.c_compiler_path might be another thing to look at.

Use basename(c_compiler_path) to choose between clang or gcc style
linker flags.
@cstrahan
Copy link
Copy Markdown
Author

@jayconrod

Just to set expectations, I probably won't have time to debug this issue this week, so I can't offer direct advice on what to do at the moment.

No worries -- thanks for the heads up, and all of the help!

go.cgo_tools.c_compiler_path might be another thing to look at.

I just gave that a go, and with the latest commits I just pushed, my test project works with both gcc and clang! 🎊

Whenever you get a chance, let me know what next steps you'd like to see, and I'd be more than happy to push this along.

@cstrahan
Copy link
Copy Markdown
Author

Actually, I take back what I said in that last comment about working with clang. I just realized I had set CC=clang-7, so that check for clang was not satisfied.

If I set CC=clang (on Linux):

ERROR: /home/cstrahan/src/rules_go_repro/BUILD:34:1: GoCompilePkg demogo.a failed (Exit 1) builder failed: error executing command bazel-out/host/bin/external/go_sdk/builder compilepkg -sdk external/go_sdk -installsuffix linux_amd64 -src main.go -p main -package_list bazel-out/host/bin/external/go_sdk/packages.txt -o ... (remaining 17 argument(s) skipped)

Use --sandbox_debug to see verbose messages from the sandbox
/usr/bin/ld.gold: fatal error: -f/--auxiliary may not be used without -shared
clang: error: linker command failed with exit code 1 (use -v to see invocation)
compilepkg: error running subcommand: exit status 1
Target //:demogo failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 1.662s, Critical Path: 0.40s
INFO: 5 processes: 5 linux-sandbox.
FAILED: Build did NOT complete successfully

I have some more work to do. I'll see if I can sort things out later today when I have more time.

@robfig
Copy link
Copy Markdown
Contributor

robfig commented Sep 23, 2021

Hi @cstrahan , thanks for working on this! I'm doing a maintenance pass on all our PRs and I wanted to see if this was still an area of active development. Unfortunately I don't have Jay's knowledge to help with the implementation, just trying to clean up shop a bit.

yushan26 pushed a commit to yushan26/rules_go that referenced this pull request Jun 16, 2025
…g packages (bazel-contrib#2514) (bazel-contrib#2584)

This reverts commit fbf8bc1 (bazel-contrib#2514)

Also, update the CHANGELOG about the reverting.

Fixes bazel-contrib#908, which is about the `pip-compile` not using the right files
for performing the locking. It seems that the `pip` upgrade regressed
this error.
fmeum pushed a commit that referenced this pull request Sep 5, 2025
…hen using alwayslink = True (#4438)

<!-- Thanks for sending a PR! Before submitting:

1. If this is your first PR, please read CONTRIBUTING.md and sign the
CLA
   first. We cannot review code without a signed CLA.
2. Please file an issue *first*. All features and most bug fixes should
have
an associated issue with a design discussed and decided upon. Small bug
   fixes and documentation improvements don't need issues.
3. New features and bug fixes must have tests. Documentation may need to
be updated. If you're unsure what to update, send the PR, and we'll
discuss
   in review.
4. Note that PRs updating dependencies and new Go versions are not
accepted.
   Please file an issue instead.
-->

**What type of PR is this?**

Bug fix

**What does this PR do? Why is it needed?**

Fixes static linking against libraries that have `alwayslink = True`
specified. Fixes C++'s dynamic initialization of static variables when
using cgo

**Which issues(s) does this PR fix?**

Fixes #1486

---

Related to #2584 and
#2294
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants