We introduced docker image builds in #517. Initially the goal was to generate linux-amd64 and linux-arm64 images, but downscoped to linux-amd64 for the v1. Here are some notes for when we pick this up again. Mechanically, to enable the build, it's just a matter of either copy/pasting the entire amd64 step and replacing instances of amd64 with arm64, or editing the platforms property of the existing docker/build-push-action@v6 step to include ,linux-arm64.
On my local M3 Macbook Air, the build (for linux-arm64/v8) works without an issue. On GitHub Actions, however, we ran into the following issues:
nsjail didn't compile
...
> [linux/arm64 builder 4/7] RUN git clone https://github.com/google/nsjail.git nsjail-checkout && cd nsjail-checkout && make && mv nsjail .. && cd .. && rm -rf nsjail-checkout:
60.42 objdump -t libkafel_r.o | grep '.hidden' | grep -v 'get_pc_thunk' | sed -e 's/^.*\.hidden //' > libkafel.localize_symbols
60.62 objcopy -w --localize-symbols=libkafel.localize_symbols libkafel_r.o libkafel.o
60.72 rm -f libkafel.localize_symbols
60.76 rm -f libkafel_r.o
60.80 ar rcs ../libkafel.a libkafel.o
60.88 rm -f libkafel.o
60.92 make[2]: Leaving directory '/nsjail-checkout/kafel/src'
60.92 make[1]: Leaving directory '/nsjail-checkout/kafel'
61.08 g++ -O2 -c -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -fPIE -Wformat -Wformat-security -Wno-format-nonliteral -Wall -Wextra -Werror -Ikafel/include -std=c++17 -fno-exceptions -Wno-unused -Wno-unused-parameter -I/usr/include/libnl3 -c -o caps.o caps.cc
61.22 make: *** [<builtin>: caps.o] Segmentation fault (core dumped)
zstd didn't compile
...
#14 1442.1 Compiling precomputed-hash v0.1.1
#14 1444.7 Compiling actix-router v0.5.3
#14 1458.6 Compiling h2 v0.4.7
#14 1462.9 Compiling chrono v0.4.39
#14 1472.5 warning: zstd-sys@2.0.13+zstd.1.5.6: ToolExecError: Command LC_ALL="C" "cc" "-O3" "-ffunction-sections" "-fdata-sections" "-fPIC" "-I" "zstd/lib/" "-I" "zstd/lib/common" "-I" "zstd/lib/legacy" "-fvisibility=hidden" "-ffunction-sections" "-fdata-sections" "-fmerge-all-constants" "-DZSTD_LIB_DEPRECATED=0" "-DXXH_PRIVATE_API=" "-DZSTDLIB_VISIBILITY=" "-DZDICTLIB_VISIBILITY=" "-DZSTDERRORLIB_VISIBILITY=" "-DZSTD_LEGACY_SUPPORT=1" "-o" "/ayb/target/release/build/zstd-sys-2d0cce38a029843b/out/44ff4c55aa9e5133-error_private.o" "-c" "zstd/lib/common/error_private.c" with args cc did not execute successfully (status code signal: 11 (SIGSEGV) (core dumped)).
#14 1472.5 error: failed to run custom build command for `zstd-sys v2.0.13+zstd.1.5.6`
#14 1472.5
#14 1472.5 Caused by:
#14 1472.5 process didn't exit successfully: `/ayb/target/release/build/zstd-sys-03958aac7f190a8d/build-script-build` (exit status: 1)
#14 1472.5 --- stdout
#14 1472.5 cargo:rerun-if-env-changed=ZSTD_SYS_USE_PKG_CONFIG
#14 1472.5 OUT_DIR = Some(/ayb/target/release/build/zstd-sys-2d0cce38a029843b/out)
#14 1472.5 OPT_LEVEL = Some(3)
#14 1472.5 TARGET = Some(aarch64-unknown-linux-gnu)
#14 1472.5 HOST = Some(aarch64-unknown-linux-gnu)
#14 1472.5 cargo:rerun-if-env-changed=CC_aarch64-unknown-linux-gnu
#14 1472.5 CC_aarch64-unknown-linux-gnu = None
#14 1472.5 cargo:rerun-if-env-changed=CC_aarch64_unknown_linux_gnu
#14 1472.5 CC_aarch64_unknown_linux_gnu = None
#14 1472.5 cargo:rerun-if-env-changed=HOST_CC
#14 1472.5 HOST_CC = None
#14 1472.5 cargo:rerun-if-env-changed=CC
#14 1472.5 CC = None
#14 1472.5 cargo:rerun-if-env-changed=CC_ENABLE_DEBUG_OUTPUT
#14 1472.5 RUSTC_WRAPPER = None
#14 1472.5 cargo:rerun-if-env-changed=CRATE_CC_NO_DEFAULTS
#14 1472.5 CRATE_CC_NO_DEFAULTS = None
#14 1472.5 DEBUG = Some(false)
#14 1472.5 CARGO_CFG_TARGET_FEATURE = Some(neon)
#14 1472.5 cargo:rerun-if-env-changed=CFLAGS_aarch64-unknown-linux-gnu
#14 1472.5 CFLAGS_aarch64-unknown-linux-gnu = None
#14 1472.5 cargo:rerun-if-env-changed=CFLAGS_aarch64_unknown_linux_gnu
#14 1472.5 CFLAGS_aarch64_unknown_linux_gnu = None
#14 1472.5 cargo:rerun-if-env-changed=HOST_CFLAGS
#14 1472.5 HOST_CFLAGS = None
#14 1472.5 cargo:rerun-if-env-changed=CFLAGS
#14 1472.5 CFLAGS = None
#14 1472.5 CARGO_ENCODED_RUSTFLAGS = Some()
#14 1472.5 OUT_DIR = Some(/ayb/target/release/build/zstd-sys-2d0cce38a029843b/out)
#14 1472.5 cargo:rerun-if-env-changed=CC_ENABLE_DEBUG_OUTPUT
#14 1472.5 cargo:rerun-if-env-changed=CRATE_CC_NO_DEFAULTS
#14 1472.5 CRATE_CC_NO_DEFAULTS = None
#14 1472.5 TARGET = Some(aarch64-unknown-linux-gnu)
#14 1472.5 CARGO_CFG_TARGET_FEATURE = Some(neon)
#14 1472.5 HOST = Some(aarch64-unknown-linux-gnu)
#14 1472.5 cargo:rerun-if-env-changed=CFLAGS_aarch64-unknown-linux-gnu
#14 1472.5 CFLAGS_aarch64-unknown-linux-gnu = None
#14 1472.5 cargo:rerun-if-env-changed=CFLAGS_aarch64_unknown_linux_gnu
#14 1472.5 CFLAGS_aarch64_unknown_linux_gnu = None
#14 1472.5 cargo:rerun-if-env-changed=HOST_CFLAGS
#14 1472.5 HOST_CFLAGS = None
#14 1472.5 cargo:rerun-if-env-changed=CFLAGS
#14 1472.5 CFLAGS = None
#14 1472.5 OUT_DIR = Some(/ayb/target/release/build/zstd-sys-2d0cce38a029843b/out)
#14 1472.5 cargo:rerun-if-env-changed=CC_ENABLE_DEBUG_OUTPUT
#14 1472.5 cargo:rerun-if-env-changed=CRATE_CC_NO_DEFAULTS
#14 1472.5 CRATE_CC_NO_DEFAULTS = None
#14 1472.5 TARGET = Some(aarch64-unknown-linux-gnu)
#14 1472.5 CARGO_CFG_TARGET_FEATURE = Some(neon)
#14 1472.5 HOST = Some(aarch64-unknown-linux-gnu)
#14 1472.5 cargo:rerun-if-env-changed=CFLAGS_aarch64-unknown-linux-gnu
#14 1472.5 CFLAGS_aarch64-unknown-linux-gnu = None
#14 1472.5 cargo:rerun-if-env-changed=CFLAGS_aarch64_unknown_linux_gnu
#14 1472.5 CFLAGS_aarch64_unknown_linux_gnu = None
#14 1472.5 cargo:rerun-if-env-changed=HOST_CFLAGS
#14 1472.5 HOST_CFLAGS = None
#14 1472.5 cargo:rerun-if-env-changed=CFLAGS
#14 1472.5 CFLAGS = None
#14 1472.5 OUT_DIR = Some(/ayb/target/release/build/zstd-sys-2d0cce38a029843b/out)
#14 1472.5 cargo:rerun-if-env-changed=CC_ENABLE_DEBUG_OUTPUT
#14 1472.5 cargo:rerun-if-env-changed=CRATE_CC_NO_DEFAULTS
#14 1472.5 CRATE_CC_NO_DEFAULTS = None
#14 1472.5 TARGET = Some(aarch64-unknown-linux-gnu)
#14 1472.5 CARGO_CFG_TARGET_FEATURE = Some(neon)
#14 1472.5 HOST = Some(aarch64-unknown-linux-gnu)
#14 1472.5 cargo:rerun-if-env-changed=CFLAGS_aarch64-unknown-linux-gnu
#14 1472.5 CFLAGS_aarch64-unknown-linux-gnu = None
#14 1472.5 cargo:rerun-if-env-changed=CFLAGS_aarch64_unknown_linux_gnu
#14 1472.5 CFLAGS_aarch64_unknown_linux_gnu = None
#14 1472.5 cargo:rerun-if-env-changed=HOST_CFLAGS
#14 1472.5 HOST_CFLAGS = None
#14 1472.5 cargo:rerun-if-env-changed=CFLAGS
#14 1472.5 CFLAGS = None
#14 1472.5 exit status: 0
#14 1472.5 exit status: 0
#14 1472.5 signal: 11 (SIGSEGV) (core dumped)
#14 1472.5 cargo:warning=ToolExecError: Command LC_ALL="C" "cc" "-O3" "-ffunction-sections" "-fdata-sections" "-fPIC" "-I" "zstd/lib/" "-I" "zstd/lib/common" "-I" "zstd/lib/legacy" "-fvisibility=hidden" "-ffunction-sections" "-fdata-sections" "-fmerge-all-constants" "-DZSTD_LIB_DEPRECATED=0" "-DXXH_PRIVATE_API=" "-DZSTDLIB_VISIBILITY=" "-DZDICTLIB_VISIBILITY=" "-DZSTDERRORLIB_VISIBILITY=" "-DZSTD_LEGACY_SUPPORT=1" "-o" "/ayb/target/release/build/zstd-sys-2d0cce38a029843b/out/44ff4c55aa9e5133-error_private.o" "-c" "zstd/lib/common/error_private.c" with args cc did not execute successfully (status code signal: 11 (SIGSEGV) (core dumped)).
#14 1472.5 exit status: 0
The linux-arm64 build is way slower
On GitHub Actions, building linux-amd64 takes ~5 minutes, whereas linux-arm64 takes ~27 minutes to get to the failure above. I'm guessing since all of this builds on my beefier M3 Macbook Air with no segfaults and way faster, that the segfaults in the previous steps and the slowness here are due to some sort of out of memory/swap space problem.
We introduced docker image builds in #517. Initially the goal was to generate
linux-amd64andlinux-arm64images, but downscoped tolinux-amd64for the v1. Here are some notes for when we pick this up again. Mechanically, to enable the build, it's just a matter of either copy/pasting the entireamd64step and replacing instances ofamd64witharm64, or editing theplatformsproperty of the existingdocker/build-push-action@v6step to include,linux-arm64.On my local M3 Macbook Air, the build (for
linux-arm64/v8) works without an issue. On GitHub Actions, however, we ran into the following issues:nsjaildidn't compilezstddidn't compileThe
linux-arm64build is way slowerOn GitHub Actions, building
linux-amd64takes ~5 minutes, whereaslinux-arm64takes ~27 minutes to get to the failure above. I'm guessing since all of this builds on my beefier M3 Macbook Air with no segfaults and way faster, that the segfaults in the previous steps and the slowness here are due to some sort of out of memory/swap space problem.