Skip to content

Pass large ldflags to cgo via response file instead of env variable.#4386

Merged
fmeum merged 9 commits intobazel-contrib:masterfrom
dgoel:cgo_ldflags_file
Jul 3, 2025
Merged

Pass large ldflags to cgo via response file instead of env variable.#4386
fmeum merged 9 commits intobazel-contrib:masterfrom
dgoel:cgo_ldflags_file

Conversation

@dgoel
Copy link
Copy Markdown
Contributor

@dgoel dgoel commented Jun 30, 2025

What type of PR is this?

Bug fix

What does this PR do? Why is it needed?

Pass ldflags to cgo tool via a response file instead of CGO_LDFLAGS env variable for go 1.23+. This avoids the "argument list too long" error, which can happen if CGO_LDFLAGS is so large that it exceeds the system limits.

Support to accept ldflags via file was added to go more than a year ago. References:

Which issues(s) does this PR fix?

Fixes #2654

Copy link
Copy Markdown
Contributor

@mering mering left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: The indentation look off. Note that golang uses tabs for indentation.

@dgoel
Copy link
Copy Markdown
Contributor Author

dgoel commented Jun 30, 2025

Nit: The indentation look off. Note that golang uses tabs for indentation.

Done! Thanks for pointing out.

@dgoel dgoel changed the title Pass ldflags to cgo via response file instead of env variable. Pass large ldflags to cgo via response file instead of env variable. Jun 30, 2025
@fmeum fmeum requested a review from jayconrod June 30, 2025 18:12
@jayconrod
Copy link
Copy Markdown
Collaborator

Also, needs a test in tests/core/cgo. Just a buildable target that will trigger this functionality to make sure it still links.

@jayconrod
Copy link
Copy Markdown
Collaborator

So far we've tried to support EOLed versions of Go, do you think we should keep doing that?

I think we can keep supporting them as long as it's convenient. In this case, we can gate the new functionality on runtime.Version.

- Use tempfile for ldflags response file.
- Only use response file for go 1.23+.
- Add a test to confirm that ldflags are progapated correctly.
@dgoel
Copy link
Copy Markdown
Contributor Author

dgoel commented Jul 1, 2025

Also, needs a test in tests/core/cgo. Just a buildable target that will trigger this functionality to make sure it still links.

Added a new test though some cgo existing tests were also sufficient.

@dgoel dgoel requested a review from fmeum July 1, 2025 17:10
@fmeum fmeum requested a review from jayconrod July 1, 2025 18:06
@dgoel dgoel force-pushed the cgo_ldflags_file branch from e93e565 to 31d9f4c Compare July 2, 2025 06:27
Dhiraj Goel added 2 commits July 2, 2025 18:00
It fails with "The filename or extension is too long" error, which is
due to windows compiler and unrelated to cgo.
@jayconrod
Copy link
Copy Markdown
Collaborator

The change looks good to me. However, CI is failing on Windows due to the command-line length limit.

It looks like we're getting the Go version out of go.mod where we have go 1.22.0 and toolchain go1.23.6. So I'd guess we're actually building with standard library version 1.22.0, disabling this new feature. @fmeum Do you know if we support the toolchain directive? I'm not all that familiar with it, but it seems like that should control the version of the compiler and standard library we download, if it's higher than the go version.

@dgoel
Copy link
Copy Markdown
Contributor Author

dgoel commented Jul 2, 2025

The change looks good to me. However, CI is failing on Windows due to the command-line length limit.

My hunch is that the new cgo plumbing is working correctly (because cgo execution is not failing). The error is happening in the hand-off from cgo to the windows compiler where arguments are being passed via command line vs params/response file. That seems unrelated to this specific change.

In the latest commit, I restricted the test target to macos and linux. We can relax it if we are able to find and fix the underlying issue.

@fmeum
Copy link
Copy Markdown
Member

fmeum commented Jul 2, 2025

@jayconrod We should be preferring toolchain over go for the SDK version as per

version = state["toolchain"]
.

@jayconrod
Copy link
Copy Markdown
Collaborator

So then @dgoel may be right that there is a problem between cgo and the Windows compiler. I don't like that we're disabling the test on Windows, but if that's true, then it can't work on Windows.

Unfortunately I don't have more bandwidth this week to confirm this. Let's merge this now.

@dgoel
Copy link
Copy Markdown
Contributor Author

dgoel commented Jul 3, 2025

@fmeum could you enable squash + automerge for this PR if you are okay with the final set of changes?

@fmeum fmeum merged commit 7bf46c1 into bazel-contrib:master Jul 3, 2025
1 check passed
@dgoel dgoel deleted the cgo_ldflags_file branch July 5, 2025 07:29
@dgoel
Copy link
Copy Markdown
Contributor Author

dgoel commented Jul 7, 2025

Thanks @fmeum!

Would it be possible to push a new release to bcr?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

cgo fails when CGO_LDFLAGS too large

4 participants