Skip to content

build: use mode default and valid_archive=False in bazel proto gen#74313

Merged
craig[bot] merged 2 commits intocockroachdb:masterfrom
dt:valid-archive
Jan 4, 2022
Merged

build: use mode default and valid_archive=False in bazel proto gen#74313
craig[bot] merged 2 commits intocockroachdb:masterfrom
dt:valid-archive

Conversation

@dt
Copy link
Copy Markdown
Contributor

@dt dt commented Dec 29, 2021

Prior to this change, there were some known-broken go targets in the tree, defined by go_proto_library rules, that were unbuildable for reasons explained below. They existed only to be embedded in other go_library targets which could then be built, tested, etc and were not intended to be built or tested on their own, but their presence in the tree still caused problems for workflows that inadvertently tried to build them, such as build //some/prefix/... or test-changed-packages.

Specifically, some options used in protos to control code generation can result in a generated file that cannot be compiled on its own. For example, several of our proto messages use gogo.stringer=false which causes the generated message to not include a String() method, but it still asserts that it implements the Message interface which requires that method. It is expected that the generated file is passed to the go compiler along with a hand-written file that adds the missing method to the message type, so that it them implements the interface.

That hand-written file depends on the generated file, e.g. the type is appears in the receiver when defining the String() method, so it too cannot be built on its own. Thus simply wrapping it in a go_library of its own to embed in the go_proto_library doesn't solve this; now that target fails to build.

Instead, we can set the valid_archive=False flag on the proto compiler used to generate these go_proto_library targets. Doing so causes these targets to no longer yeidld a GoArchive, which believes it should be able to be built, and instead only yeild the GoLibrary and GoSource that can then be embedded in go_library to complete it.

To use the above, we also switch gazelle's proto mode from package to default; the latter instructs it to generate these go_library wrapper targets that embded the go_proto_library.

In this mode it generates a go_library target as well that embeds the
go_proto_library target and resolves users to depend on it instead.

On its own this changes very little, but paves the way for a follow-up
change to how go_proto_library targets behave.

Release note: none.
@cockroach-teamcity

This comment has been minimized.

@dt
Copy link
Copy Markdown
Contributor Author

dt commented Dec 29, 2021

Before this change:

➜ bazel build //pkg/util/tracing/...
...
ERROR:.../pkg/util/tracing/tracingpb/BUILD.bazel:36:17: GoCompilePkg pkg/util/tracing/tracingpb/tracingpb_go_proto.a failed: (Exit 1): builder failed: error executing command bazel-out/host/bin/external/go_sdk/builder compilepkg -sdk external/go_sdk -installsuffix darwin_arm64 -tags bazel,gss,bazel,gss -src ... (remaining 29 argument(s) skipped)

Use --sandbox_debug to see verbose messages from the sandbox
[...]/util/tracing/tracingpb/recorded_span.pb.go:149:10: undefined: TraceID
[...]/util/tracing/tracingpb/recorded_span.pb.go:151:9: undefined: SpanID
[...]/util/tracing/tracingpb/recorded_span.pb.go:153:15: undefined: SpanID
[...]
compilepkg: error running subcommand external/go_sdk/pkg/tool/darwin_arm64/compile: exit status 2
INFO: Elapsed time: 5.285s, Critical Path: 0.96s
INFO: 20 processes: 15 internal, 5 darwin-sandbox.
FAILED: Build did NOT complete successfully

After this change:

➜  bazel build //pkg/util/tracing/...
...
INFO: 40 processes: 1 internal, 39 darwin-sandbox.
INFO: Build completed successfully, 40 total actions

Prior to this change, there were some known-broken go targets in the
tree, defined by `go_proto_library` rules, that were unbuildable for
reasons explained below. They existed only to be embedded in other
`go_library` targets which could then be built, tested, etc and were not
_intended_ to be built or tested on their own, but their presence in the
tree still caused problems for workflows that inadvertently tried to
build them, such as `build //some/prefix/...` or test-changed-packages.

Specifically, some options used in protos to control code generation can
result in a generated file that cannot be compiled on its own. For
example, several of our proto messages use `gogo.stringer=false` which
causes the generated message to not include a String() method, but it
still asserts that it implements the Message interface which requires
that method. It is expected that the generated file is passed to the go
compiler along with a hand-written file that adds the missing method to
the message type, so that it them implements the interface.

That hand-written file depends on the generated file, e.g. the type is
appears in the receiver when defining the String() method, so it too
cannot be built on its own. Thus simply wrapping it in a go_library of
its own to `embed` in the `go_proto_library` doesn't solve this; now
*that* target fails to build.

Instead, we can set the `valid_archive=False` flag on the proto compiler
used to generate these go_proto_library targets. Doing so causes these
targets to no longer yeidld a `GoArchive`, which believes it should be
able to be built, and instead only yeild the `GoLibrary` and `GoSource`
that can then be embedded in `go_library` to complete it.

Release note: none.
Copy link
Copy Markdown
Collaborator

@rickystewart rickystewart left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

works for me, thanks!

@dt
Copy link
Copy Markdown
Contributor Author

dt commented Jan 4, 2022

TFTR!

bors r+

@craig
Copy link
Copy Markdown
Contributor

craig bot commented Jan 4, 2022

This PR was included in a batch that was canceled, it will be automatically retried

@irfansharif
Copy link
Copy Markdown
Contributor

Cool!

@craig
Copy link
Copy Markdown
Contributor

craig bot commented Jan 4, 2022

Build failed (retrying...):

@craig
Copy link
Copy Markdown
Contributor

craig bot commented Jan 4, 2022

Build succeeded:

@craig craig bot merged commit 798a3e5 into cockroachdb:master Jan 4, 2022
@dt dt deleted the valid-archive branch January 4, 2022 23:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants