Skip to content

builds: make bazelw support arm64 builds#2096

Merged
jpsim merged 3 commits intoenvoyproxy:mainfrom
crockeo:ch/build-arm64
Mar 14, 2022
Merged

builds: make bazelw support arm64 builds#2096
jpsim merged 3 commits intoenvoyproxy:mainfrom
crockeo:ch/build-arm64

Conversation

@crockeo
Copy link
Copy Markdown
Contributor

@crockeo crockeo commented Mar 14, 2022 •
edited
Loading

For an explanation of how to fill out the fields, please see the relevant section
in PULL_REQUESTS.md

Description: Modifies the bazelw shell script to bootstrap bazelisk for arm64 on both Darwin and Linux. This unblocks work to re-enable Python builds for py-envoy-mobile internally. Note that this may not work until #2085 is in, as (iirc) it includes fixes to arm64 builds for Envoy.
Risk Level: Low
Testing: No unit testing, but tested on arm64 both on macOS (natively) and Linux (Docker). See below. I don't have an x86_64 laptop on hand at the moment, so if someone wouldn't mind checking that this still works on x86_64, that'd be much appreciated!
Docs Changes: N/A
Release Notes: N/A

macOS 
~/src/envoy-mobile$ ./bazelw
Installing bazelisk...
[bazel output omitted]
~/src/envoy-mobile$ ls tmp/bazel/versions/
bazelisk-1.10.1-darwin-arm64

Note that there is no "Extracting Bazel installation..." step as I've been running this already.

Linux 
/code/envoy-mobile$ ./bazelw
Installing bazelisk...
2022/03/14 03:42:12 Downloading https://releases.bazel.build/5.0.0/release/bazel-5.0.0-linux-arm64...
Extracting Bazel installation...
Starting local Bazel server and connecting to it...
[bazel output omitted]
/code/envoy-mobile$ ls tmp/bazel/versions/
bazelisk-1.10.1-linux-arm64

crockeo added 3 commits March 13, 2022 20:35
@crockeo
make bazelw support arm64 builds
e12734c 
Signed-off-by: Cerek Hillen <cerekh@gmail.com>
@crockeo
go back to using OSTYPE
293591a 
Signed-off-by: Cerek Hillen <cerekh@gmail.com>
@crockeo
move variable down a bit
7851250 
Signed-off-by: Cerek Hillen <cerekh@gmail.com>
@jpsim
Copy link
Copy Markdown
Contributor

jpsim commented Mar 14, 2022

#2094 updated Envoy a few days ago so #2085 shouldn't be needed.

@jpsim
Copy link
Copy Markdown
Contributor

jpsim commented Mar 14, 2022

I'm seeing the same issue on my M1 Mac as I did when I reverted a similar change in January: #1998

Here's what I'm seeing now:

./bazelw build //library/common:envoy_main_interface_lib
Installing bazelisk...
2022/03/14 12:26:05 Downloading https://releases.bazel.build/5.0.0/release/bazel-5.0.0-darwin-arm64...
Extracting Bazel installation...
Starting local Bazel server and connecting to it...
DEBUG: /private/var/tmp/_bazel_jsimard/7140f038a5c13b01a171c93151a000da/external/rules_proto_grpc/repositories.bzl:302:18: Dependency 'bazel_skylib' has already been declared with a different version. Found urls=("https://github.com/bazelbuild/bazel-skylib/releases/download/1.1.1/bazel-skylib-1.1.1.tar.gz",), expected ("https://mirror.bazel.build/github.com/bazelbuild/bazel-skylib/archive/be3b1fc838386bdbea39d9750ea4411294870575.tar.gz", "https://github.com/bazelbuild/bazel-skylib/archive/be3b1fc838386bdbea39d9750ea4411294870575.tar.gz")
DEBUG: /private/var/tmp/_bazel_jsimard/7140f038a5c13b01a171c93151a000da/external/rules_proto_grpc/repositories.bzl:302:18: Dependency 'com_google_protobuf' has already been declared with a different version. Found urls=("https://github.com/protocolbuffers/protobuf/releases/download/v3.16.0/protobuf-all-3.16.0.tar.gz",), expected ("https://mirror.bazel.build/github.com/protocolbuffers/protobuf/archive/19ca196e4dcc8ff03136c950cf1bc298d5e1ce1e.tar.gz", "https://github.com/protocolbuffers/protobuf/archive/19ca196e4dcc8ff03136c950cf1bc298d5e1ce1e.tar.gz")
DEBUG: /private/var/tmp/_bazel_jsimard/7140f038a5c13b01a171c93151a000da/external/rules_proto_grpc/repositories.bzl:311:18: Dependency 'zlib' has already been declared with a different rule kind. Found bind, expected http_archive
DEBUG: /private/var/tmp/_bazel_jsimard/7140f038a5c13b01a171c93151a000da/external/rules_proto_grpc/repositories.bzl:302:18: Dependency 'bazel_skylib' has already been declared with a different version. Found urls=("https://github.com/bazelbuild/bazel-skylib/releases/download/1.1.1/bazel-skylib-1.1.1.tar.gz",), expected ("https://mirror.bazel.build/github.com/bazelbuild/bazel-skylib/archive/be3b1fc838386bdbea39d9750ea4411294870575.tar.gz", "https://github.com/bazelbuild/bazel-skylib/archive/be3b1fc838386bdbea39d9750ea4411294870575.tar.gz")
DEBUG: /private/var/tmp/_bazel_jsimard/7140f038a5c13b01a171c93151a000da/external/rules_proto_grpc/repositories.bzl:302:18: Dependency 'com_google_protobuf' has already been declared with a different version. Found urls=("https://github.com/protocolbuffers/protobuf/releases/download/v3.16.0/protobuf-all-3.16.0.tar.gz",), expected ("https://mirror.bazel.build/github.com/protocolbuffers/protobuf/archive/19ca196e4dcc8ff03136c950cf1bc298d5e1ce1e.tar.gz", "https://github.com/protocolbuffers/protobuf/archive/19ca196e4dcc8ff03136c950cf1bc298d5e1ce1e.tar.gz")
DEBUG: /private/var/tmp/_bazel_jsimard/7140f038a5c13b01a171c93151a000da/external/rules_proto_grpc/repositories.bzl:311:18: Dependency 'zlib' has already been declared with a different rule kind. Found bind, expected http_archive
DEBUG: /private/var/tmp/_bazel_jsimard/7140f038a5c13b01a171c93151a000da/external/rules_proto_grpc/repositories.bzl:302:18: Dependency 'io_grpc_grpc_java' has already been declared with a different version. Found urls=("https://github.com/grpc/grpc-java/archive/v1.24.0.tar.gz",), expected ("https://mirror.bazel.build/github.com/grpc/grpc-java/archive/e341d4c6554ddac8f883c03357e2b9646c090ff1.tar.gz", "https://github.com/grpc/grpc-java/archive/e341d4c6554ddac8f883c03357e2b9646c090ff1.tar.gz")
ERROR: /private/var/tmp/_bazel_jsimard/7140f038a5c13b01a171c93151a000da/external/com_envoyproxy_protoc_gen_validate/BUILD:11:10: While resolving toolchains for target @com_envoyproxy_protoc_gen_validate//:protoc-gen-validate: no matching toolchains found for types @io_bazel_rules_go//go:toolchain
ERROR: Analysis of target '//library/common:envoy_main_interface_lib' failed; build aborted: 
INFO: Elapsed time: 10.572s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (326 packages loaded, 4709 targets configured)
    currently loading: @org_brotli// ... (2 packages)

That command builds cleanly with the x86_64 bazelisk binary.

@jpsim
Copy link
Copy Markdown
Contributor

jpsim commented Mar 14, 2022

I'd be supportive if you wanted to narrow this change to support Linux arm64/amd64.

@crockeo
Copy link
Copy Markdown
Contributor Author

crockeo commented Mar 14, 2022

Strange! I had tested it locally and it works.

bazelw output 
./bazelw build //library/common:envoy_main_interface_lib
DEBUG: /private/var/tmp/_bazel_chillen/2489a5718b9c4d75cf74403596848d28/external/rules_proto_grpc/repositories.bzl:302:18: Dependency 'bazel_skylib' has already been declared with a different version. Found urls=("https://github.com/bazelbuild/bazel-skylib/releases/download/1.1.1/bazel-skylib-1.1.1.tar.gz",), expected ("https://mirror.bazel.build/github.com/bazelbuild/bazel-skylib/archive/be3b1fc838386bdbea39d9750ea4411294870575.tar.gz", "https://github.com/bazelbuild/bazel-skylib/archive/be3b1fc838386bdbea39d9750ea4411294870575.tar.gz")
DEBUG: /private/var/tmp/_bazel_chillen/2489a5718b9c4d75cf74403596848d28/external/rules_proto_grpc/repositories.bzl:302:18: Dependency 'com_google_protobuf' has already been declared with a different version. Found urls=("https://github.com/protocolbuffers/protobuf/releases/download/v3.16.0/protobuf-all-3.16.0.tar.gz",), expected ("https://mirror.bazel.build/github.com/protocolbuffers/protobuf/archive/19ca196e4dcc8ff03136c950cf1bc298d5e1ce1e.tar.gz", "https://github.com/protocolbuffers/protobuf/archive/19ca196e4dcc8ff03136c950cf1bc298d5e1ce1e.tar.gz")
DEBUG: /private/var/tmp/_bazel_chillen/2489a5718b9c4d75cf74403596848d28/external/rules_proto_grpc/repositories.bzl:311:18: Dependency 'zlib' has already been declared with a different rule kind. Found bind, expected http_archive
DEBUG: /private/var/tmp/_bazel_chillen/2489a5718b9c4d75cf74403596848d28/external/rules_proto_grpc/repositories.bzl:302:18: Dependency 'bazel_skylib' has already been declared with a different version. Found urls=("https://github.com/bazelbuild/bazel-skylib/releases/download/1.1.1/bazel-skylib-1.1.1.tar.gz",), expected ("https://mirror.bazel.build/github.com/bazelbuild/bazel-skylib/archive/be3b1fc838386bdbea39d9750ea4411294870575.tar.gz", "https://github.com/bazelbuild/bazel-skylib/archive/be3b1fc838386bdbea39d9750ea4411294870575.tar.gz")
DEBUG: /private/var/tmp/_bazel_chillen/2489a5718b9c4d75cf74403596848d28/external/rules_proto_grpc/repositories.bzl:302:18: Dependency 'com_google_protobuf' has already been declared with a different version. Found urls=("https://github.com/protocolbuffers/protobuf/releases/download/v3.16.0/protobuf-all-3.16.0.tar.gz",), expected ("https://mirror.bazel.build/github.com/protocolbuffers/protobuf/archive/19ca196e4dcc8ff03136c950cf1bc298d5e1ce1e.tar.gz", "https://github.com/protocolbuffers/protobuf/archive/19ca196e4dcc8ff03136c950cf1bc298d5e1ce1e.tar.gz")
DEBUG: /private/var/tmp/_bazel_chillen/2489a5718b9c4d75cf74403596848d28/external/rules_proto_grpc/repositories.bzl:311:18: Dependency 'zlib' has already been declared with a different rule kind. Found bind, expected http_archive
DEBUG: /private/var/tmp/_bazel_chillen/2489a5718b9c4d75cf74403596848d28/external/rules_proto_grpc/repositories.bzl:302:18: Dependency 'io_grpc_grpc_java' has already been declared with a different version. Found urls=("https://github.com/grpc/grpc-java/archive/v1.24.0.tar.gz",), expected ("https://mirror.bazel.build/github.com/grpc/grpc-java/archive/e341d4c6554ddac8f883c03357e2b9646c090ff1.tar.gz", "https://github.com/grpc/grpc-java/archive/e341d4c6554ddac8f883c03357e2b9646c090ff1.tar.gz")
DEBUG: Rule 'com_envoyproxy_protoc_gen_validate' indicated that a canonical reproducible form can be obtained by modifying arguments shallow_since = "1643413839 -0700"
DEBUG: Repository com_envoyproxy_protoc_gen_validate instantiated at:
  /Users/chillen/src/envoy-mobile-private/envoy-mobile/WORKSPACE:15:26: in 
  /private/var/tmp/_bazel_chillen/2489a5718b9c4d75cf74403596848d28/external/envoy_mobile/bazel/envoy_mobile_repositories.bzl:15:17: in envoy_mobile_repositories
  /private/var/tmp/_bazel_chillen/2489a5718b9c4d75cf74403596848d28/external/envoy_mobile/bazel/envoy_mobile_repositories.bzl:31:19: in override_pgv
Repository rule git_repository defined at:
  /private/var/tmp/_bazel_chillen/2489a5718b9c4d75cf74403596848d28/external/bazel_tools/tools/build_defs/repo/git.bzl:199:33: in 
INFO: Analyzed target //library/common:envoy_main_interface_lib (414 packages loaded, 17569 targets configured).
INFO: Found 1 target...
INFO: From Generating C++ proto_library //library/common/extensions/retry/options/network_configuration:predicate:
library/common/extensions/retry/options/network_configuration/predicate.proto:5:1: warning: Import validate/validate.proto is unused.
INFO: From Generating Descriptor Set proto_library //library/common/extensions/retry/options/network_configuration:predicate:
library/common/extensions/retry/options/network_configuration/predicate.proto:5:1: warning: Import validate/validate.proto is unused.
Target //library/common:envoy_main_interface_lib up-to-date (nothing to build)
INFO: Elapsed time: 394.108s, Critical Path: 44.43s
INFO: 2146 processes: 63 internal, 2083 local.
INFO: Build completed successfully, 2146 total actions

I'm happy to downscope, but I'd also love to root-cause! I remember running into a similar golang toolchain issue when using the x86_64 bazelisk before using the arm64 bazelisk. A couple of questions:

  • What happens if you bazelw clean --expunge and then try to build the target again?
  • Is this the same issue that was happening in the internal builders or was it something else?

@jpsim
Copy link
Copy Markdown
Contributor

jpsim commented Mar 14, 2022

Ran ./bazelw clean --expunge and tested again with ./bazelw build //library/common:envoy_main_interface_lib and it worked. Maybe that's all we need to transition.

I'll do some more testing with some of our downstream integrations before signing off.

jpsim
jpsim approved these changes Mar 14, 2022
View reviewed changes
Copy link
Copy Markdown
Contributor

@jpsim jpsim left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some light downstream testing didn't surface any issues that couldn't be resolved with a ./bazelw clean --expunge. 👍

@jpsim jpsim merged commit 82aec9b into envoyproxy:main Mar 14, 2022
@jpsim jpsim mentioned this pull request Mar 15, 2022
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@jpsim jpsim jpsim approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@crockeo @jpsim