Skip to content

clang won't build for bpf target #176128

Closed
#176152
Closed
clang won't build for bpf target#176128
#176152
Labels
0.kind: bugSomething is broken6.topic: cross-compilationBuilding packages on a different platform than they will be used on
@mmlb

Description

@mmlb
mmlb
opened on Jun 3, 2022

Describe the bug

I'd like to build bpf object (clang -target bpf -g ...) files but am unable to do so.

Steps To Reproduce

Steps to reproduce the behavior:

  1. clone https://github.com/libbpf/libbpf-bootstrap
  2. cd libbpf-bootstrap
  3. nix-shell -p clang libbpf
  4. clang -O2 -target bpfel -g -c minimal.bpf.c
  5. readelf -h minimal.bpf.o | e:grep Machine:
  6. get: Machine: Advanced Micro Devices X86-64, want: Machine: Linux BPF

Expected behavior

I expect clang to honor the -target I specify so that I can generate bpf object files. I was able to verify that just -target ... and -fstack-protector-strong are getting in the way of generating valid eBPF files. I did so by copying the wrapper script that is clang and seeing the final full command line that is actually invoking the real clang binary:

Unmodified exec line (doesn't work) 
#!/usr/bin/env bash
out=minimal.bpf.unmod.o
/nix/store/p6dr62inbahm1q40xf4hl7l5p1ivghfj-clang-11.1.0/bin/clang \
	-fPIC \
	-Wformat \
	-Wformat-security \
	-Werror=format-security \
	-fstack-protector-strong \
	--param ssp-buffer-size=4 \
	-O2 \
	-D_FORTIFY_SOURCE=2 \
	-fno-strict-overflow \
	-O2 \
	-target bpf \
	-g \
	-c minimal.bpf.c \
	-o $out \
	-B/nix/store/lyl6nysc3i3aqhj6shizjgj0ibnf1pvg-glibc-2.34-210/lib/ \
	-idirafter /nix/store/a0s1vq35j3g6aa757v7yg0a001l01ng4-glibc-2.34-210-dev/include \
	-target x86_64-unknown-linux-gnu \
	-B/nix/store/d7q0qfm12hb59wj63wyjs0hrdhmmapfz-gcc-11.3.0/lib/gcc/x86_64-unknown-linux-gnu/11.3.0 \
	--gcc-toolchain=/nix/store/d7q0qfm12hb59wj63wyjs0hrdhmmapfz-gcc-11.3.0 \
	-B/nix/store/a469w4h9bs21y3499i9sa0fzzjqfwx2m-clang-11.1.0-lib/lib \
	-resource-dir=/nix/store/r2h2gkjsr0xmxk1s0bx6p4v2czpc3km8-clang-wrapper-11.1.0/resource-root \
	-B/nix/store/r2h2gkjsr0xmxk1s0bx6p4v2czpc3km8-clang-wrapper-11.1.0/bin/ \
	-frandom-seed=4f2mdhgyns \
	-isystem /nix/store/74qg7k17vf31v4pckf8j31q0rdrzpgcp-libbpf-0.7.0/include \
	-isystem /nix/store/2lfvz4d6a4iqriqakw4qpg94bzr3c8z5-compiler-rt-libc-11.1.0-dev/include \
	-isystem /nix/store/74qg7k17vf31v4pckf8j31q0rdrzpgcp-libbpf-0.7.0/include \
	-isystem /nix/store/2lfvz4d6a4iqriqakw4qpg94bzr3c8z5-compiler-rt-libc-11.1.0-dev/include \
	-isystem /nix/store/74qg7k17vf31v4pckf8j31q0rdrzpgcp-libbpf-0.7.0/include \
	-isystem /nix/store/2lfvz4d6a4iqriqakw4qpg94bzr3c8z5-compiler-rt-libc-11.1.0-dev/include

readelf -h $out | grep Machine:
Modified exec (works) 
#!/usr/bin/env bash
out=minimal.bpf.mod.o
/nix/store/p6dr62inbahm1q40xf4hl7l5p1ivghfj-clang-11.1.0/bin/clang \
	-fPIC \
	-Wformat \
	-Wformat-security \
	-Werror=format-security \
	--param ssp-buffer-size=4 \
	-O2 \
	-D_FORTIFY_SOURCE=2 \
	-fno-strict-overflow \
	-O2 \
	-target bpf \
	-g \
	-c minimal.bpf.c \
	-o $out \
	-B/nix/store/lyl6nysc3i3aqhj6shizjgj0ibnf1pvg-glibc-2.34-210/lib/ \
	-idirafter /nix/store/a0s1vq35j3g6aa757v7yg0a001l01ng4-glibc-2.34-210-dev/include \
	-B/nix/store/d7q0qfm12hb59wj63wyjs0hrdhmmapfz-gcc-11.3.0/lib/gcc/x86_64-unknown-linux-gnu/11.3.0 \
	--gcc-toolchain=/nix/store/d7q0qfm12hb59wj63wyjs0hrdhmmapfz-gcc-11.3.0 \
	-B/nix/store/a469w4h9bs21y3499i9sa0fzzjqfwx2m-clang-11.1.0-lib/lib \
	-resource-dir=/nix/store/r2h2gkjsr0xmxk1s0bx6p4v2czpc3km8-clang-wrapper-11.1.0/resource-root \
	-B/nix/store/r2h2gkjsr0xmxk1s0bx6p4v2czpc3km8-clang-wrapper-11.1.0/bin/ \
	-frandom-seed=4f2mdhgyns \
	-isystem /nix/store/74qg7k17vf31v4pckf8j31q0rdrzpgcp-libbpf-0.7.0/include \
	-isystem /nix/store/2lfvz4d6a4iqriqakw4qpg94bzr3c8z5-compiler-rt-libc-11.1.0-dev/include \
	-isystem /nix/store/74qg7k17vf31v4pckf8j31q0rdrzpgcp-libbpf-0.7.0/include \
	-isystem /nix/store/2lfvz4d6a4iqriqakw4qpg94bzr3c8z5-compiler-rt-libc-11.1.0-dev/include \
	-isystem /nix/store/74qg7k17vf31v4pckf8j31q0rdrzpgcp-libbpf-0.7.0/include \
	-isystem /nix/store/2lfvz4d6a4iqriqakw4qpg94bzr3c8z5-compiler-rt-libc-11.1.0-dev/include

readelf -h $out | grep Machine:
runs 
[11:13:57]-[~/cloned/github.com/libbpf/libbpf-bootstrap/examples/c]─[manny@zennix]> 
./bad.sh 
  Machine:                           Advanced Micro Devices X86-64
                                                                                                                                                                      
[11:13:58]-[~/cloned/github.com/libbpf/libbpf-bootstrap/examples/c]─[manny@zennix]> 
./good.sh 
clang-11: warning: argument unused during compilation: '--gcc-toolchain=/nix/store/d7q0qfm12hb59wj63wyjs0hrdhmmapfz-gcc-11.3.0' [-Wunused-command-line-argument]
readelf: Warning: unable to apply unsupported reloc type 10 to section .debug_info
readelf: Warning: unable to apply unsupported reloc type 1 to section .debug_info
readelf: Warning: unable to apply unsupported reloc type 10 to section .debug_info
readelf: Warning: unable to apply unsupported reloc type 1 to section .debug_info
readelf: Warning: unable to apply unsupported reloc type 10 to section .debug_info
readelf: Warning: unable to apply unsupported reloc type 1 to section .debug_info
readelf: Warning: unable to apply unsupported reloc type 10 to section .debug_info
readelf: Warning: unable to apply unsupported reloc type 1 to section .debug_info
readelf: Warning: unable to apply unsupported reloc type 10 to section .debug_info
readelf: Warning: unable to apply unsupported reloc type 1 to section .debug_info
readelf: Warning: unable to apply unsupported reloc type 10 to section .debug_info
  Machine:                           Linux BPF
                                                                                                                                                                      
[11:14:00]-[~/cloned/github.com/libbpf/libbpf-bootstrap/examples/c]─[manny@zennix]> 
diff -u bad.sh good.sh 
--- bad.sh	2022-06-03 11:13:39.685645805 -0400
+++ good.sh	2022-06-03 11:13:55.424661960 -0400
@@ -1,11 +1,10 @@
 #!/usr/bin/env bash
-out=minimal.bpf.unmod.o
+out=minimal.bpf.mod.o
 /nix/store/p6dr62inbahm1q40xf4hl7l5p1ivghfj-clang-11.1.0/bin/clang \
 	-fPIC \
 	-Wformat \
 	-Wformat-security \
 	-Werror=format-security \
-	-fstack-protector-strong \
 	--param ssp-buffer-size=4 \
 	-O2 \
 	-D_FORTIFY_SOURCE=2 \
@@ -17,7 +16,6 @@
 	-o $out \
 	-B/nix/store/lyl6nysc3i3aqhj6shizjgj0ibnf1pvg-glibc-2.34-210/lib/ \
 	-idirafter /nix/store/a0s1vq35j3g6aa757v7yg0a001l01ng4-glibc-2.34-210-dev/include \
-	-target x86_64-unknown-linux-gnu \
 	-B/nix/store/d7q0qfm12hb59wj63wyjs0hrdhmmapfz-gcc-11.3.0/lib/gcc/x86_64-unknown-linux-gnu/11.3.0 \
 	--gcc-toolchain=/nix/store/d7q0qfm12hb59wj63wyjs0hrdhmmapfz-gcc-11.3.0 \
 	-B/nix/store/a469w4h9bs21y3499i9sa0fzzjqfwx2m-clang-11.1.0-lib/lib \

Additional context

I tracked this down to cc-wrapper inserting the -target via clang-wrapper-*/nix-support/cc-cflags which ends up living in extraAfter. I'd like for it be be in extraBefore instead so that I can override it. This is deep in the guts of CC support so wanted to open up an issue for discussion for this before trying to mess around with a PR.

I can see why the -target was added to the wrapper, I suspect to require use of crossSystem or similar to get a clang that will build for that arch/system but I'm not so sure thats possible for bpf (wasm too?) that don't have a libc and other common tools (binutils?)...

Notify maintainers

@Ericson2314 (for cc-wrapper)
@lovek323 @dtzWill @primeos (for clang)

Metadata

Please run nix-shell -p nix-info --run "nix-info -m" and paste the result.

nix-shell -p nix-info --run "nix-info -m"
 - system: `"x86_64-linux"`
 - host os: `Linux 5.17.7, NixOS, 22.05 (Quokka), 22.05pre377845.43ff6cb1c02`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.8.0`
 - channels(root): `"nixos"`
 - channels(manny): `"home-manager, nixos-21.11-21.11"`
 - nixpkgs: `/nix/var/nix/profiles/per-user/root/channels/nixos`

Metadata

Metadata

Assignees

No one assigned

    Labels

    0.kind: bugSomething is broken6.topic: cross-compilationBuilding packages on a different platform than they will be used on

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions