Skip to content

test/run-cli-tests: inherit ASAN_OPTIONS and LSAN_OPTIONS#64521

Merged
tchaikov merged 1 commit intoceph:mainfrom
tchaikov:wip-run-cli-tests-with-asan-supp
Aug 4, 2025
Merged

test/run-cli-tests: inherit ASAN_OPTIONS and LSAN_OPTIONS#64521
tchaikov merged 1 commit intoceph:mainfrom
tchaikov:wip-run-cli-tests-with-asan-supp

Conversation

@tchaikov
Copy link
Contributor

@tchaikov tchaikov commented Jul 16, 2025

Previously, run-cli-tests ignored all environment variables from the parent process to ensure a clean test environment. However, this also dropped sanitizer settings (ASAN_OPTIONS and LSAN_OPTIONS) needed when AddressSanitizer is enabled.

This causes test failures with TCMalloc due to false-positive leak reports from TCMalloc's internal objects, which is a known issue documented in Google's C++ style guide. While recent gperftools releases have fixed this, Ubuntu Jammy still ships with an older version that triggers these warnings.

This change preserves sanitizer environment variables while maintaining the clean test environment for other variables.

Note: Once we upgrade to newer gperftools, we can remove the related suppression rule in qa/lsan.supp.

The test failure with TCMalloc looks like:

/home/jenkins-build/build/workspace/ceph-pull-requests/src/test/cli/ceph-kvstore-tool/help.t: failed
--- /home/jenkins-build/build/workspace/ceph-pull-requests/src/test/cli/ceph-kvstore-tool/help.t
+++ /home/jenkins-build/build/workspace/ceph-pull-requests/src/test/cli/ceph-kvstore-tool/help.t.err
@@ -21,3 +21,19 @@
     stats
     histogram [prefix]

+
+  =================================================================
+  ==87908==ERROR: LeakSanitizer: detected memory leaks
+
+  Direct leak of 45 byte(s) in 1 object(s) allocated from:
+      #0 0x562fd797265d in operator new(unsigned long) (/home/jenkins-build/build/workspace/ceph-pull-requests/build/bin/ceph-kvstore-tool+0xe5e65d) (BuildId: 7eb56077b615aeb3c7aedafa0818ad89fdf3702d)
+      #1 0x562fd79815c8 in std::__new_allocator<char>::allocate(unsigned long, void const*) /usr/lib/gcc/x86_64-linux-gnu/12/../../../../include/c++/12/bits/new_allocator.h:137:27
+      #2 0x562fd7981520 in std::allocator<char>::allocate(unsigned long) /usr/lib/gcc/x86_64-linux-gnu/12/../../../../include/c++/12/bits/allocator.h:188:32
+      #3 0x562fd7981520 in std::allocator_traits<std::allocator<char>>::allocate(std::allocator<char>&, unsigned long) /usr/lib/gcc/x86_64-linux-gnu/12/../../../../include/c++/12/bits/alloc_traits.h:464:20
+      #4 0x562fd798115a in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>::_M_create(unsigned long&, unsigned long) /usr/lib/gcc/x86_64-linux-gnu/12/../../../../include/c++/12/bits/basic_string.tcc:155:14
+      #5 0x562fd798787f in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>::_M_mutate(unsigned long, unsigned long, char const*, unsigned long) /usr/lib/gcc/x86_64-linux-gnu/12/../../../../include/c++/12/bits/basic_string.tcc:328:21
+      #6 0x562fd79876a7 in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>::_M_append(char const*, unsigned long) /usr/lib/gcc/x86_64-linux-gnu/12/../../../../include/c++/12/bits/basic_string.tcc:420:8
+      #7 0x7fa1aa0286f0 in MallocExtension::Initialize() (/lib/x86_64-linux-gnu/libtcmalloc.so.4+0x2a6f0) (BuildId: eeef3d1257388a806e122398dbce3157ee568ef4)
+
+  SUMMARY: AddressSanitizer: 45 byte(s) leaked in 1 allocation(s).

Contribution Guidelines

  • To sign and title your commits, please refer to Submitting Patches to Ceph.

  • If you are submitting a fix for a stable branch (e.g. "quincy"), please refer to Submitting Patches to Ceph - Backports for the proper workflow.

  • When filling out the below checklist, you may click boxes directly in the GitHub web UI. When entering or editing the entire PR message in the GitHub web UI editor, you may also select a checklist item by adding an x between the brackets: [x]. Spaces and capitalization matter when checking off items this way.

Checklist

  • Tracker (select at least one)
    • References tracker ticket
    • Very recent bug; references commit where it was introduced
    • New feature (ticket optional)
    • Doc update (no ticket needed)
    • Code cleanup (no ticket needed)
  • Component impact
    • Affects Dashboard, opened tracker ticket
    • Affects Orchestrator, opened tracker ticket
    • No impact that needs to be tracked
  • Documentation (select at least one)
    • Updates relevant documentation
    • No doc update is appropriate
  • Tests (select at least one)
Show available Jenkins commands

Previously, run-cli-tests ignored all environment variables from the parent
process to ensure a clean test environment. However, this also dropped
sanitizer settings (ASAN_OPTIONS and LSAN_OPTIONS) needed when AddressSanitizer
is enabled.

This causes test failures with TCMalloc due to false-positive leak reports
from TCMalloc's internal objects, which is a known issue documented in
Google's C++ style guide. While recent gperftools releases have fixed this,
Ubuntu Jammy still ships with an older version that triggers these warnings.

This change preserves sanitizer environment variables while maintaining
the clean test environment for other variables.

Note: Once we upgrade to newer gperftools, we can remove the related
suppression rule in qa/lsan.supp.

The test failure with TCMalloc looks like:

```
/home/jenkins-build/build/workspace/ceph-pull-requests/src/test/cli/ceph-kvstore-tool/help.t: failed
--- /home/jenkins-build/build/workspace/ceph-pull-requests/src/test/cli/ceph-kvstore-tool/help.t
+++ /home/jenkins-build/build/workspace/ceph-pull-requests/src/test/cli/ceph-kvstore-tool/help.t.err
@@ -21,3 +21,19 @@
     stats
     histogram [prefix]

+
+  =================================================================
+  ==87908==ERROR: LeakSanitizer: detected memory leaks
+
+  Direct leak of 45 byte(s) in 1 object(s) allocated from:
+      #0 0x562fd797265d in operator new(unsigned long) (/home/jenkins-build/build/workspace/ceph-pull-requests/build/bin/ceph-kvstore-tool+0xe5e65d) (BuildId: 7eb56077b615aeb3c7aedafa0818ad89fdf3702d)
+      #1 0x562fd79815c8 in std::__new_allocator<char>::allocate(unsigned long, void const*) /usr/lib/gcc/x86_64-linux-gnu/12/../../../../include/c++/12/bits/new_allocator.h:137:27
+      #2 0x562fd7981520 in std::allocator<char>::allocate(unsigned long) /usr/lib/gcc/x86_64-linux-gnu/12/../../../../include/c++/12/bits/allocator.h:188:32
+      #3 0x562fd7981520 in std::allocator_traits<std::allocator<char>>::allocate(std::allocator<char>&, unsigned long) /usr/lib/gcc/x86_64-linux-gnu/12/../../../../include/c++/12/bits/alloc_traits.h:464:20
+      #4 0x562fd798115a in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>::_M_create(unsigned long&, unsigned long) /usr/lib/gcc/x86_64-linux-gnu/12/../../../../include/c++/12/bits/basic_string.tcc:155:14
+      #5 0x562fd798787f in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>::_M_mutate(unsigned long, unsigned long, char const*, unsigned long) /usr/lib/gcc/x86_64-linux-gnu/12/../../../../include/c++/12/bits/basic_string.tcc:328:21
+      #6 0x562fd79876a7 in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>::_M_append(char const*, unsigned long) /usr/lib/gcc/x86_64-linux-gnu/12/../../../../include/c++/12/bits/basic_string.tcc:420:8
+      #7 0x7fa1aa0286f0 in MallocExtension::Initialize() (/lib/x86_64-linux-gnu/libtcmalloc.so.4+0x2a6f0) (BuildId: eeef3d1257388a806e122398dbce3157ee568ef4)
+
+  SUMMARY: AddressSanitizer: 45 byte(s) leaked in 1 allocation(s).
```

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
@github-actions github-actions bot added the tests label Jul 16, 2025
@tchaikov tchaikov mentioned this pull request Jul 16, 2025
14 tasks
@tchaikov
Copy link
Contributor Author

jenkins test make check

@tchaikov
Copy link
Contributor Author

jenkins test make check arm64

@tchaikov
Copy link
Contributor Author

jenkins test make check

1 similar comment
@tchaikov
Copy link
Contributor Author

jenkins test make check

@tchaikov
Copy link
Contributor Author

@ceph/core hello maintainers, could you please help review this change?

2 similar comments
@tchaikov
Copy link
Contributor Author

@ceph/core hello maintainers, could you please help review this change?

@tchaikov
Copy link
Contributor Author

tchaikov commented Aug 3, 2025

@ceph/core hello maintainers, could you please help review this change?

@tchaikov tchaikov merged commit abd4ec6 into ceph:main Aug 4, 2025
25 of 26 checks passed
@tchaikov tchaikov deleted the wip-run-cli-tests-with-asan-supp branch August 4, 2025 05:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants