Heap checker compiled on clang-10 crashes on SysCalls::wait4() on arm64 #1241
Description
Any program or test running while HEAPCHECK=normal crashes at the end in SysCalls::wait4(). This happens if compiled with clang-10 using either -O0 or -O1, but not if using -O2 or -O3. This also does not happen if compiled with GCC 7.5. Looking at the disassembly (below) it seems the errno value is being written to the memory address 0x104, which is the syscall number of wait4 on arm64. So this appears to be a bug in clang-10, see the filed clang Bugzilla report: https://bugs.llvm.org/show_bug.cgi?id=48798. The reproducer in the report was reduced from gperftool code.
Reporting this here as this affects users of gperftools running on arm64. Symptoms of this include all bazel tests failing on arm64 when compiled on clang-10, as tests are typically compiled without optimization.
$ env HEAPCHECK=normal bazel-bin/tests/accesslog_test
WARNING: Perftools heap leak checker is active -- Performance may suffer
[==========] Running 1 test from 1 test suite.
[----------] Global test environment set-up.
[----------] 1 test from CiliumTest
[ RUN ] CiliumTest.AccessLog
TestRandomGenerator running with seed -1303473560
[2021-01-18 05:18:48.367][325665][error][misc] [tests/accesslog_test.cc:42] source_address: 5.6.7.8:45678
[2021-01-18 05:18:48.367][325665][error][misc] [tests/accesslog_test.cc:43] destination_address: 1.2.3.4:80
[ OK ] CiliumTest.AccessLog (447 ms)
[----------] 1 test from CiliumTest (447 ms total)
[----------] Global test environment tear-down
[==========] 1 test from 1 test suite ran. (447 ms total)
[ PASSED ] 1 test.
Segmentation fault (core dumped)
$ lldb-10 bazel-bin/tests/accesslog_test -c core
(lldb) target create "bazel-bin/tests/accesslog_test" --core "core"
Core file '/home/ubuntu/go/src/github.com/cilium/proxy/core' (aarch64) was loaded.
(lldb) dis -n SysCalls::wait4
accesslog_test`(anonymous namespace)::SysCalls::wait4:
0x6e084ac <+0>: sub sp, sp, #0x60 ; =0x60
0x6e084b0 <+4>: str x0, [sp, #0x58]
0x6e084b4 <+8>: str w1, [sp, #0x54]
0x6e084b8 <+12>: str x2, [sp, #0x48]
0x6e084bc <+16>: str w3, [sp, #0x44]
0x6e084c0 <+20>: str x4, [sp, #0x38]
0x6e084c4 <+24>: ldr x8, [sp, #0x58]
0x6e084c8 <+28>: ldrsw x9, [sp, #0x54]
0x6e084cc <+32>: str x9, [sp, #0x30]
0x6e084d0 <+36>: ldr x9, [sp, #0x48]
0x6e084d4 <+40>: str x9, [sp, #0x28]
0x6e084d8 <+44>: ldrsw x9, [sp, #0x44]
0x6e084dc <+48>: str x9, [sp, #0x20]
0x6e084e0 <+52>: ldr x9, [sp, #0x38]
0x6e084e4 <+56>: str x9, [sp, #0x18]
0x6e084e8 <+60>: ldr x0, [sp, #0x30]
0x6e084ec <+64>: ldr x1, [sp, #0x28]
0x6e084f0 <+68>: ldr x2, [sp, #0x20]
0x6e084f4 <+72>: ldr x3, [sp, #0x18]
0x6e084f8 <+76>: mov x8, #0x104
0x6e084fc <+80>: svc #0
0x6e08500 <+84>: str x0, [sp, #0x10]
0x6e08504 <+88>: ldr x9, [sp, #0x10]
0x6e08508 <+92>: str x9, [sp, #0x8]
0x6e0850c <+96>: str x8, [sp]
0x6e08510 <+100>: b 0x6e08514 ; <+104> at linux_syscall_support.h:2605:16
0x6e08514 <+104>: ldr x8, [sp, #0x8]
0x6e08518 <+108>: adds x8, x8, #0xfff ; =0xfff
0x6e0851c <+112>: b.lo 0x6e08548 ; <+156> at linux_syscall_support.h:2605:16
0x6e08520 <+116>: b 0x6e08524 ; <+120> at linux_syscall_support.h:2605:16
0x6e08524 <+120>: ldr w8, [sp, #0x8]
0x6e08528 <+124>: mov w9, wzr
0x6e0852c <+128>: subs w8, w9, w8
0x6e08530 <+132>: mov w0, w8
0x6e08534 <+136>: ldr x10, [sp]
-> 0x6e08538 <+140>: str w0, [x10]
0x6e0853c <+144>: mov x11, #-0x1
0x6e08540 <+148>: str x11, [sp, #0x8]
0x6e08544 <+152>: b 0x6e08548 ; <+156> at linux_syscall_support.h:2605:16
0x6e08548 <+156>: ldr w0, [sp, #0x8]
0x6e0854c <+160>: add sp, sp, #0x60 ; =0x60
0x6e08550 <+164>: ret
(lldb) bt
* thread #1, name = 'accesslog_test', stop reason = signal SIGSEGV
* frame #0: 0x0000000006e08538 accesslog_test`(anonymous namespace)::SysCalls::wait4(this=0x0000ffffd0281ba8, p=220, s=0x0000ffffd0281ba0, o=1073741824, r=0x0000000000000000)::SysCalls::kernel_rusage*) at linux_syscall_support.h:2605:16
frame #1: 0x0000000006e07274 accesslog_test`(anonymous namespace)::SysCalls::waitpid(this=0x0000ffffd0281ba8, pid=220, status=0x0000ffffd0281ba0, options=1073741824) at linux_syscall_support.h:2609:14
frame #2: 0x0000000006e063b4 accesslog_test`::TCMalloc_ListAllProcessThreads(parameter=0x0000000000000000, callback=(accesslog_test`HeapLeakChecker::IgnoreLiveThreadsLocked(void*, int, int*, std::__va_list) at heap-checker.cc:1024)) at linuxthreads.cc:645:22
frame #3: 0x0000000006df9e0c accesslog_test`HeapLeakChecker::IgnoreAllLiveObjectsLocked(self_stack_top=0x0000ffffd02841ec) at heap-checker.cc:1313:15
frame #4: 0x0000000006dfab90 accesslog_test`HeapLeakChecker::DoNoLeaks(this=0x000000001b9fb140, should_symbolize=SYMBOLIZE) at heap-checker.cc:1769:5
frame #5: 0x0000000006dfc3dc accesslog_test`HeapLeakChecker::NoGlobalLeaksMaybeSymbolize(should_symbolize=SYMBOLIZE) at heap-checker.cc:2147:21
frame #6: 0x0000000006dfb8f8 accesslog_test`HeapLeakChecker::DoMainHeapCheck() at heap-checker.cc:2169:8
frame #7: 0x0000000006dfc864 accesslog_test`HeapLeakChecker_AfterDestructors() at heap-checker.cc:2317:9
frame #8: 0x0000000006e086f8 accesslog_test`HeapLeakCheckerGlobalPrePost::~HeapLeakCheckerGlobalPrePost(this=0x00000000071ccba0) at heap-checker-bcad.cc:82:23
frame #9: 0x0000ffffac20074c libc.so.6`___lldb_unnamed_symbol101$$libc.so.6 + 268
frame #10: 0x0000ffffac2008dc libc.so.6`exit + 28
frame #11: 0x0000ffffac1eb094 libc.so.6`__libc_start_main + 236
frame #12: 0x00000000033857b4 accesslog_test`_start + 52
(lldb) q