Skip to content

ci: add scripts to build patched versions of Go runtime#84867

Merged
craig[bot] merged 1 commit intocockroachdb:masterfrom
rickystewart:patchedgo
Aug 2, 2022
Merged

ci: add scripts to build patched versions of Go runtime#84867
craig[bot] merged 1 commit intocockroachdb:masterfrom
rickystewart:patchedgo

Conversation

@rickystewart
Copy link
Copy Markdown
Collaborator

@rickystewart rickystewart commented Jul 21, 2022

The script simply uses our existing cross toolchains and applies a patch
to the Go sources for the fine-grained CPU attribution work. We make a
custom go executable for all supported platforms except FreeBSD, for
which we have no cross-compilers, and M1/M2 Macs, for which we don't
have a code-signing pipeline set up yet.

Also update build/README.MD to capture the new process of building
dependencies.

Part of #82625.

Release note: None

@rickystewart rickystewart added the do-not-merge bors won't merge a PR with this label. label Jul 21, 2022
@cockroach-teamcity
Copy link
Copy Markdown
Member

This change is Reviewable

@rickystewart rickystewart force-pushed the patchedgo branch 20 times, most recently from bbd59a5 to 8da42a3 Compare July 22, 2022 17:01
@rickystewart rickystewart requested review from irfansharif and rail July 22, 2022 17:01
@rickystewart rickystewart marked this pull request as ready for review July 22, 2022 17:02
@rickystewart rickystewart requested a review from a team July 22, 2022 17:02
@rickystewart rickystewart requested review from a team as code owners July 22, 2022 17:02
@rickystewart rickystewart requested review from a team July 22, 2022 17:02
@rickystewart
Copy link
Copy Markdown
Collaborator Author

Hmm, still have a failing test. :(

/home/roach/.cache/bazel/_bazel_roach/cc377fc379544923cc7508dd261e4a48/sandbox/processwrapper-sandbox/1233/execroot/com_github_cockroachdb_cockroach/bazel-out/aarch64-fastbuild/bin/pkg/testutils/lint/passes/loopvarcapture/loopvarcapture_test_/loopvarcapture_test.runfiles/go_sdk/src/runtime/cgo/cgo.go:34:8: could not import C (no metadata for C)
/home/roach/.cache/bazel/_bazel_roach/cc377fc379544923cc7508dd261e4a48/sandbox/processwrapper-sandbox/1233/execroot/com_github_cockroachdb_cockroach/bazel-out/aarch64-fastbuild/bin/pkg/testutils/lint/passes/loopvarcapture/loopvarcapture_test_/loopvarcapture_test.runfiles/go_sdk/src/net/cgo_linux.go:12:8: could not import C (no metadata for C)
--- FAIL: TestAnalyzer (0.39s)
    analysistest.go:298: error analyzing loopvarcapture@p: analysis skipped due to errors in package

@irfansharif
Copy link
Copy Markdown
Contributor

irfansharif commented Jul 31, 2022

I don't entirely understand what's happening and what I'm doing is likely unsound, but this patch fixed the test:

diff --git i/pkg/testutils/lint/passes/loopvarcapture/loopvarcapture.go w/pkg/testutils/lint/passes/loopvarcapture/loopvarcapture.go
index 402c8c3fcb..cf919d0bb2 100644
--- i/pkg/testutils/lint/passes/loopvarcapture/loopvarcapture.go
+++ w/pkg/testutils/lint/passes/loopvarcapture/loopvarcapture.go
@@ -51,10 +51,11 @@ var (
 	// Analyzer implements this linter, looking for loop variables
 	// captured by reference in closures called in Go routines
 	Analyzer = &analysis.Analyzer{
-		Name:     name,
-		Doc:      doc,
-		Requires: []*analysis.Analyzer{inspect.Analyzer},
-		Run:      run,
+		Name:             name,
+		Doc:              doc,
+		Requires:         []*analysis.Analyzer{inspect.Analyzer},
+		Run:              run,
+		RunDespiteErrors: true,
 	}
 
 	// GoRoutineFunctions is a collection of functions that are known to

Reproduced using:

$ ./dev builder
$ ./dev test pkg/testutils/lint/passes/loopvarcapture

From what I understand, it's something to do with golang/go#36547, some funkiness in packages.Load expecting metadata for the pseudo C package with some combination of GOOS/CGO_ENABLED env vars. By specifying RunDespiteErrors: true, we're defeating this analysis/internal/checker check, which doesn't look relevant for this unit test and its test data files.

The script simply uses our existing cross toolchains and applies a patch
to the Go sources for the fine-grained CPU attribution work. We make a
custom `go` executable for all supported platforms except FreeBSD, for
which we have no cross-compilers, and M1/M2 Macs, for which we don't
have a code-signing pipeline set up yet.

Also update `build/README.MD` to capture the new process of building
dependencies.

Part of cockroachdb#82625.

Release note: None
@rickystewart
Copy link
Copy Markdown
Collaborator Author

I updated the test in question to use a version of the analyzer with RunDespiteErrors = true instead of updating the actual analyzer in-place. If this passes CI I'll bors today.

@rickystewart
Copy link
Copy Markdown
Collaborator Author

bors r=irfansharif,rail

@craig
Copy link
Copy Markdown
Contributor

craig bot commented Aug 2, 2022

Build failed:

@rickystewart
Copy link
Copy Markdown
Collaborator Author

image

bors r=irfansharif,rail

@craig
Copy link
Copy Markdown
Contributor

craig bot commented Aug 2, 2022

Build failed (retrying...):

@craig
Copy link
Copy Markdown
Contributor

craig bot commented Aug 2, 2022

Build succeeded:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants