Skip to content

~DynamicThreadPool() got some errors in in valgrind and TASN. #5088

@jiaqizho

Description

@jiaqizho

Bug Report

Please answer these questions before submitting your issue. Thanks!

1. Minimal reproduce step (Required)

  1. TASN get data race
[2022-06-07T18:07:38.675Z] WARNING: ThreadSanitizer: data race (pid=18280)
[2022-06-07T18:07:38.675Z]   Write of size 8 at 0x7b20000da228 by main thread:
[2022-06-07T18:07:38.675Z]     #0 operator delete(void*) /llvm-project/compiler-rt/lib/tsan/rtl/tsan_new_delete.cpp:126:3 (gtests_dbms+0x2538510)
[2022-06-07T18:07:38.675Z]     #1 void std::__1::__libcpp_operator_delete<void*>(void*) /usr/local/bin/../include/c++/v1/new:245:3 (gtests_dbms+0xd852b7e)
[2022-06-07T18:07:38.675Z]     #2 void std::__1::__do_deallocate_handle_size<>(void*, unsigned long) /usr/local/bin/../include/c++/v1/new:269:10 (gtests_dbms+0xd852b7e)
[2022-06-07T18:07:38.675Z]     #3 std::__1::__libcpp_deallocate(void*, unsigned long, unsigned long) /usr/local/bin/../include/c++/v1/new:285:14 (gtests_dbms+0xd852b7e)
[2022-06-07T18:07:38.675Z]     #4 std::__1::allocator<prometheus::Gauge*>::deallocate(prometheus::Gauge**, unsigned long) /usr/local/bin/../include/c++/v1/__memory/allocator.h:117:13 (gtests_dbms+0xd852b7e)
[2022-06-07T18:07:38.675Z]     #5 std::__1::allocator_traits<std::__1::allocator<prometheus::Gauge*> >::deallocate(std::__1::allocator<prometheus::Gauge*>&, prometheus::Gauge**, unsigned long) /usr/local/bin/../include/c++/v1/__memory/allocator_traits.h:282:13 (gtests_dbms+0xd852b7e)
[2022-06-07T18:07:38.675Z]     #6 std::__1::__vector_base<prometheus::Gauge*, std::__1::allocator<prometheus::Gauge*> >::~__vector_base() /usr/local/bin/../include/c++/v1/vector:488:9 (gtests_dbms+0xd852b7e)
[2022-06-07T18:07:38.675Z]     #7 std::__1::vector<prometheus::Gauge*, std::__1::allocator<prometheus::Gauge*> >::~vector() /usr/local/bin/../include/c++/v1/vector:579:5 (gtests_dbms+0xd852b7e)
[2022-06-07T18:07:38.675Z]     #8 DB::MetricFamily<prometheus::Gauge>::~MetricFamily() /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/dbms/src/Common/TiFlashMetrics.h:287:8 (gtests_dbms+0xd852b7e)
[2022-06-07T18:07:38.675Z]     #9 DB::TiFlashMetrics::~TiFlashMetrics() /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/dbms/src/Common/TiFlashMetrics.h:322:7 (gtests_dbms+0xd852b7e)
[2022-06-07T18:07:38.675Z]     #10 cxa_at_exit_wrapper(void*) /llvm-project/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:394:3 (gtests_dbms+0x24ffa9f)
[2022-06-07T18:07:38.675Z] 
[2022-06-07T18:07:38.675Z]   Previous read of size 8 at 0x7b20000da228 by thread T18:
[2022-06-07T18:07:38.675Z]     [failed to restore the stack]
[2022-06-07T18:07:38.675Z] 
[2022-06-07T18:07:38.675Z]   As if synchronized via sleep:
[2022-06-07T18:07:38.675Z]     #0 nanosleep /llvm-project/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:363:3 (gtests_dbms+0x24c0bef)
[2022-06-07T18:07:38.675Z]     #1 std::__1::this_thread::sleep_for(std::__1::chrono::duration<long long, std::__1::ratio<1l, 1000000000l> > const&) <null> (libc++.so.1+0x709ba)
[2022-06-07T18:07:38.675Z]     #2 DB::tests::(anonymous namespace)::DynamicThreadPoolTest_testAutoExpanding_Test::TestBody() /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/dbms/src/Common/tests/gtest_dynamic_thread_pool.cpp:61:1 (gtests_dbms+0x25883ec)
[2022-06-07T18:07:38.675Z]     #3 void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/contrib/googletest/googletest/src/gtest.cc:2401:10 (gtests_dbms+0x9985715)
[2022-06-07T18:07:38.675Z]     #4 void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/contrib/googletest/googletest/src/gtest.cc:2437:14 (gtests_dbms+0x9985715)
[2022-06-07T18:07:38.675Z]     #5 testing::Test::Run() /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/contrib/googletest/googletest/src/gtest.cc:2473:5 (gtests_dbms+0x9965435)
[2022-06-07T18:07:38.675Z]     #6 testing::TestInfo::Run() /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/contrib/googletest/googletest/src/gtest.cc:2655:11 (gtests_dbms+0x9966c01)
[2022-06-07T18:07:38.675Z]     #7 testing::TestCase::Run() /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/contrib/googletest/googletest/src/gtest.cc:2773:28 (gtests_dbms+0x99678bd)
[2022-06-07T18:07:38.675Z]     #8 testing::internal::UnitTestImpl::RunAllTests() /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/contrib/googletest/googletest/src/gtest.cc:4673:43 (gtests_dbms+0x9972c40)
[2022-06-07T18:07:38.675Z]     #9 bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/contrib/googletest/googletest/src/gtest.cc:2401:10 (gtests_dbms+0x99867d5)
[2022-06-07T18:07:38.675Z]     #10 bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/contrib/googletest/googletest/src/gtest.cc:2437:14 (gtests_dbms+0x99867d5)
[2022-06-07T18:07:38.675Z]     #11 testing::UnitTest::Run() /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/contrib/googletest/googletest/src/gtest.cc:4281:10 (gtests_dbms+0x99720c3)
[2022-06-07T18:07:38.675Z]     #12 RUN_ALL_TESTS() /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/contrib/googletest/googletest/include/gtest/gtest.h:2237:46 (gtests_dbms+0x3f6d827)
[2022-06-07T18:07:38.675Z]     #13 main /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/dbms/src/TestUtils/gtests_dbms_main.cpp:36:16 (gtests_dbms+0x3f6d827)
[2022-06-07T18:07:38.675Z] 
[2022-06-07T18:07:38.675Z]   Thread T18 (tid=18488, finished) created by main thread at:
[2022-06-07T18:07:38.675Z]     #0 pthread_create /llvm-project/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:977:3 (gtests_dbms+0x24c327d)
[2022-06-07T18:07:38.675Z]     #1 std::__1::__libcpp_thread_create(unsigned long*, void* (*)(void*), void*) /usr/local/bin/../include/c++/v1/__threading_support:514:10 (gtests_dbms+0xd7e6062)
[2022-06-07T18:07:38.676Z]     #2 std::__1::thread::thread<std::__1::thread DB::ThreadFactory::newThread<void (DB::DynamicThreadPool::*)(std::__1::unique_ptr<DB::IExecutableTask, std::__1::default_delete<DB::IExecutableTask> >), DB::DynamicThreadPool*, std::__1::unique_ptr<DB::IExecutableTask, std::__1::default_delete<DB::IExecutableTask> > >(bool, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, void (DB::DynamicThreadPool::*&&)(std::__1::unique_ptr<DB::IExecutableTask, std::__1::default_delete<DB::IExecutableTask> >), DB::DynamicThreadPool*&&, std::__1::unique_ptr<DB::IExecutableTask, std::__1::default_delete<DB::IExecutableTask> >&&)::'lambda'(auto&&...)&, DB::DynamicThreadPool*, std::__1::unique_ptr<DB::IExecutableTask, std::__1::default_delete<DB::IExecutableTask> >, void>(void (DB::DynamicThreadPool::*&&)(std::__1::unique_ptr<DB::IExecutableTask, std::__1::default_delete<DB::IExecutableTask> >), DB::DynamicThreadPool*&&, std::__1::unique_ptr<DB::IExecutableTask, std::__1::default_delete<DB::IExecutableTask> >&&) /usr/local/bin/../include/c++/v1/thread:307:16 (gtests_dbms+0xd7e6062)
[2022-06-07T18:07:38.676Z]     #3 std::__1::thread DB::ThreadFactory::newThread<void (DB::DynamicThreadPool::*)(std::__1::unique_ptr<DB::IExecutableTask, std::__1::default_delete<DB::IExecutableTask> >), DB::DynamicThreadPool*, std::__1::unique_ptr<DB::IExecutableTask, std::__1::default_delete<DB::IExecutableTask> > >(bool, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, void (DB::DynamicThreadPool::*&&)(std::__1::unique_ptr<DB::IExecutableTask, std::__1::default_delete<DB::IExecutableTask> >), DB::DynamicThreadPool*&&, std::__1::unique_ptr<DB::IExecutableTask, std::__1::default_delete<DB::IExecutableTask> >&&) /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/dbms/src/Common/ThreadFactory.h:49:16 (gtests_dbms+0xd7e32f5)
[2022-06-07T18:07:38.676Z]     #4 DB::DynamicThreadPool::scheduledToNewDynamicThread(std::__1::unique_ptr<DB::IExecutableTask, std::__1::default_delete<DB::IExecutableTask> >&) /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/dbms/src/Common/DynamicThreadPool.cpp:94:21 (gtests_dbms+0xd7e32f5)
[2022-06-07T18:07:38.676Z]     #5 DB::DynamicThreadPool::scheduleTask(std::__1::unique_ptr<DB::IExecutableTask, std::__1::default_delete<DB::IExecutableTask> >) /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/dbms/src/Common/DynamicThreadPool.cpp:64:9 (gtests_dbms+0xd7e3066)
[2022-06-07T18:07:38.676Z]     #6 auto DB::DynamicThreadPool::schedule<DB::tests::(anonymous namespace)::DynamicThreadPoolTest_testAutoExpanding_Test::TestBody()::$_1>(bool, DB::tests::(anonymous namespace)::DynamicThreadPoolTest_testAutoExpanding_Test::TestBody()::$_1&&) /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/dbms/src/Common/DynamicThreadPool.h:58:9 (gtests_dbms+0x2587f16)
[2022-06-07T18:07:38.676Z]     #7 DB::tests::(anonymous namespace)::DynamicThreadPoolTest_testAutoExpanding_Test::TestBody() /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/dbms/src/Common/tests/gtest_dynamic_thread_pool.cpp:48:20 (gtests_dbms+0x2587f16)
[2022-06-07T18:07:38.676Z]     #8 void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/contrib/googletest/googletest/src/gtest.cc:2401:10 (gtests_dbms+0x9985715)
[2022-06-07T18:07:38.676Z]     #9 void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/contrib/googletest/googletest/src/gtest.cc:2437:14 (gtests_dbms+0x9985715)
[2022-06-07T18:07:38.676Z]     #10 testing::Test::Run() /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/contrib/googletest/googletest/src/gtest.cc:2473:5 (gtests_dbms+0x9965435)
[2022-06-07T18:07:38.676Z]     #11 testing::TestInfo::Run() /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/contrib/googletest/googletest/src/gtest.cc:2655:11 (gtests_dbms+0x9966c01)
[2022-06-07T18:07:38.676Z]     #12 testing::TestCase::Run() /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/contrib/googletest/googletest/src/gtest.cc:2773:28 (gtests_dbms+0x99678bd)
[2022-06-07T18:07:38.676Z]     #13 testing::internal::UnitTestImpl::RunAllTests() /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/contrib/googletest/googletest/src/gtest.cc:4673:43 (gtests_dbms+0x9972c40)
[2022-06-07T18:07:38.676Z]     #14 bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/contrib/googletest/googletest/src/gtest.cc:2401:10 (gtests_dbms+0x99867d5)
[2022-06-07T18:07:38.676Z]     #15 bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/contrib/googletest/googletest/src/gtest.cc:2437:14 (gtests_dbms+0x99867d5)
[2022-06-07T18:07:38.676Z]     #16 testing::UnitTest::Run() /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/contrib/googletest/googletest/src/gtest.cc:4281:10 (gtests_dbms+0x99720c3)
[2022-06-07T18:07:38.676Z]     #17 RUN_ALL_TESTS() /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/contrib/googletest/googletest/include/gtest/gtest.h:2237:46 (gtests_dbms+0x3f6d827)
[2022-06-07T18:07:38.676Z]     #18 main /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/dbms/src/TestUtils/gtests_dbms_main.cpp:36:16 (gtests_dbms+0x3f6d827)
[2022-06-07T18:07:38.676Z] 
[2022-06-07T18:07:38.676Z] SUMMARY: ThreadSanitizer: data race /llvm-project/compiler-rt/lib/tsan/rtl/tsan_new_delete.cpp:126:3 in operator delete(void*)
[2022-06-07T18:07:38.676Z] ==================
[2022-06-07T18:07:38.676Z] ==================
[2022-06-07T18:07:38.676Z] WARNING: ThreadSanitizer: data race (pid=18280)
[2022-06-07T18:07:38.676Z]   Write of size 8 at 0x7b04000058d0 by main thread:
[2022-06-07T18:07:38.676Z]     #0 operator delete(void*) /llvm-project/compiler-rt/lib/tsan/rtl/tsan_new_delete.cpp:126:3 (gtests_dbms+0x2538510)
[2022-06-07T18:07:38.676Z]     #1 std::__1::default_delete<prometheus::Gauge>::operator()(prometheus::Gauge*) const /usr/local/bin/../include/c++/v1/__memory/unique_ptr.h:57:5 (gtests_dbms+0xdabce57)
[2022-06-07T18:07:38.676Z]     #2 std::__1::unique_ptr<prometheus::Gauge, std::__1::default_delete<prometheus::Gauge> >::reset(prometheus::Gauge*) /usr/local/bin/../include/c++/v1/__memory/unique_ptr.h:318:7 (gtests_dbms+0xdabce57)
[2022-06-07T18:07:38.676Z]     #3 std::__1::unique_ptr<prometheus::Gauge, std::__1::default_delete<prometheus::Gauge> >::~unique_ptr() /usr/local/bin/../include/c++/v1/__memory/unique_ptr.h:272:19 (gtests_dbms+0xdabce57)
[2022-06-07T18:07:38.676Z]     #4 std::__1::pair<unsigned long const, std::__1::unique_ptr<prometheus::Gauge, std::__1::default_delete<prometheus::Gauge> > >::~pair() /usr/local/bin/../include/c++/v1/__utility/pair.h:42:29 (gtests_dbms+0xdabce57)
[2022-06-07T18:07:38.676Z]     #5 void std::__1::allocator_traits<std::__1::allocator<std::__1::__hash_node<std::__1::__hash_value_type<unsigned long, std::__1::unique_ptr<prometheus::Gauge, std::__1::default_delete<prometheus::Gauge> > >, void*> > >::destroy<std::__1::pair<unsigned long const, std::__1::unique_ptr<prometheus::Gauge, std::__1::default_delete<prometheus::Gauge> > >, void, void>(std::__1::allocator<std::__1::__hash_node<std::__1::__hash_value_type<unsigned long, std::__1::unique_ptr<prometheus::Gauge, std::__1::default_delete<prometheus::Gauge> > >, void*> >&, std::__1::pair<unsigned long const, std::__1::unique_ptr<prometheus::Gauge, std::__1::default_delete<prometheus::Gauge> > >*) /usr/local/bin/../include/c++/v1/__memory/allocator_traits.h:319:15 (gtests_dbms+0xdabce57)
[2022-06-07T18:07:38.677Z]     #6 std::__1::__hash_table<std::__1::__hash_value_type<unsigned long, std::__1::unique_ptr<prometheus::Gauge, std::__1::default_delete<prometheus::Gauge> > >, std::__1::__unordered_map_hasher<unsigned long, std::__1::__hash_value_type<unsigned long, std::__1::unique_ptr<prometheus::Gauge, std::__1::default_delete<prometheus::Gauge> > >, std::__1::hash<unsigned long>, std::__1::equal_to<unsigned long>, true>, std::__1::__unordered_map_equal<unsigned long, std::__1::__hash_value_type<unsigned long, std::__1::unique_ptr<prometheus::Gauge, std::__1::default_delete<prometheus::Gauge> > >, std::__1::equal_to<unsigned long>, std::__1::hash<unsigned long>, true>, std::__1::allocator<std::__1::__hash_value_type<unsigned long, std::__1::unique_ptr<prometheus::Gauge, std::__1::default_delete<prometheus::Gauge> > > > >::__deallocate_node(std::__1::__hash_node_base<std::__1::__hash_node<std::__1::__hash_value_type<unsigned long, std::__1::unique_ptr<prometheus::Gauge, std::__1::default_delete<prometheus::Gauge> > >, void*>*>*) /usr/local/bin/../include/c++/v1/__hash_table:1580:9 (gtests_dbms+0xdabce57)
[2022-06-07T18:07:38.677Z]     #7 std::__1::__hash_table<std::__1::__hash_value_type<unsigned long, std::__1::unique_ptr<prometheus::Gauge, std::__1::default_delete<prometheus::Gauge> > >, std::__1::__unordered_map_hasher<unsigned long, std::__1::__hash_value_type<unsigned long, std::__1::unique_ptr<prometheus::Gauge, std::__1::default_delete<prometheus::Gauge> > >, std::__1::hash<unsigned long>, std::__1::equal_to<unsigned long>, true>, std::__1::__unordered_map_equal<unsigned long, std::__1::__hash_value_type<unsigned long, std::__1::unique_ptr<prometheus::Gauge, std::__1::default_delete<prometheus::Gauge> > >, std::__1::equal_to<unsigned long>, std::__1::hash<unsigned long>, true>, std::__1::allocator<std::__1::__hash_value_type<unsigned long, std::__1::unique_ptr<prometheus::Gauge, std::__1::default_delete<prometheus::Gauge> > > > >::~__hash_table() /usr/local/bin/../include/c++/v1/__hash_table:1519:5 (gtests_dbms+0xdabce57)
[2022-06-07T18:07:38.677Z]     #8 std::__1::unordered_map<unsigned long, std::__1::unique_ptr<prometheus::Gauge, std::__1::default_delete<prometheus::Gauge> >, std::__1::hash<unsigned long>, std::__1::equal_to<unsigned long>, std::__1::allocator<std::__1::pair<unsigned long const, std::__1::unique_ptr<prometheus::Gauge, std::__1::default_delete<prometheus::Gauge> > > > >::~unordered_map() /usr/local/bin/../include/c++/v1/unordered_map:1047:5 (gtests_dbms+0xdabce57)
[2022-06-07T18:07:38.677Z]     #9 prometheus::Family<prometheus::Gauge>::~Family() /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/contrib/prometheus-cpp/core/src/family.cc:101:43 (gtests_dbms+0xdabce57)
[2022-06-07T18:07:38.677Z]     #10 prometheus::Family<prometheus::Gauge>::~Family() /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/contrib/prometheus-cpp/core/src/family.cc:101:43 (gtests_dbms+0xdabce79)
[2022-06-07T18:07:38.677Z]     #11 std::__1::default_delete<prometheus::Family<prometheus::Gauge> >::operator()(prometheus::Family<prometheus::Gauge>*) const /usr/local/bin/../include/c++/v1/__memory/unique_ptr.h:57:5 (gtests_dbms+0xdaca6fc)
[2022-06-07T18:07:38.677Z]     #12 std::__1::unique_ptr<prometheus::Family<prometheus::Gauge>, std::__1::default_delete<prometheus::Family<prometheus::Gauge> > >::reset(prometheus::Family<prometheus::Gauge>*) /usr/local/bin/../include/c++/v1/__memory/unique_ptr.h:318:7 (gtests_dbms+0xdaca6fc)
[2022-06-07T18:07:38.677Z]     #13 std::__1::unique_ptr<prometheus::Family<prometheus::Gauge>, std::__1::default_delete<prometheus::Family<prometheus::Gauge> > >::~unique_ptr() /usr/local/bin/../include/c++/v1/__memory/unique_ptr.h:272:19 (gtests_dbms+0xdaca6fc)
[2022-06-07T18:07:38.677Z]     #14 std::__1::allocator<std::__1::unique_ptr<prometheus::Family<prometheus::Gauge>, std::__1::default_delete<prometheus::Family<prometheus::Gauge> > > >::destroy(std::__1::unique_ptr<prometheus::Family<prometheus::Gauge>, std::__1::default_delete<prometheus::Family<prometheus::Gauge> > >*) /usr/local/bin/../include/c++/v1/__memory/allocator.h:159:15 (gtests_dbms+0xdaca6fc)
[2022-06-07T18:07:38.677Z]     #15 void std::__1::allocator_traits<std::__1::allocator<std::__1::unique_ptr<prometheus::Family<prometheus::Gauge>, std::__1::default_delete<prometheus::Family<prometheus::Gauge> > > > >::destroy<std::__1::unique_ptr<prometheus::Family<prometheus::Gauge>, std::__1::default_delete<prometheus::Family<prometheus::Gauge> > >, void>(std::__1::allocator<std::__1::unique_ptr<prometheus::Family<prometheus::Gauge>, std::__1::default_delete<prometheus::Family<prometheus::Gauge> > > >&, std::__1::unique_ptr<prometheus::Family<prometheus::Gauge>, std::__1::default_delete<prometheus::Family<prometheus::Gauge> > >*) /usr/local/bin/../include/c++/v1/__memory/allocator_traits.h:309:13 (gtests_dbms+0xdaca6fc)
[2022-06-07T18:07:38.677Z]     #16 std::__1::__vector_base<std::__1::unique_ptr<prometheus::Family<prometheus::Gauge>, std::__1::default_delete<prometheus::Family<prometheus::Gauge> > >, std::__1::allocator<std::__1::unique_ptr<prometheus::Family<prometheus::Gauge>, std::__1::default_delete<prometheus::Family<prometheus::Gauge> > > > >::__destruct_at_end(std::__1::unique_ptr<prometheus::Family<prometheus::Gauge>, std::__1::default_delete<prometheus::Family<prometheus::Gauge> > >*) /usr/local/bin/../include/c++/v1/vector:450:9 (gtests_dbms+0xdaca6fc)
[2022-06-07T18:07:38.677Z]     #17 std::__1::__vector_base<std::__1::unique_ptr<prometheus::Family<prometheus::Gauge>, std::__1::default_delete<prometheus::Family<prometheus::Gauge> > >, std::__1::allocator<std::__1::unique_ptr<prometheus::Family<prometheus::Gauge>, std::__1::default_delete<prometheus::Family<prometheus::Gauge> > > > >::clear() /usr/local/bin/../include/c++/v1/vector:374:29 (gtests_dbms+0xdaca6fc)
[2022-06-07T18:07:38.677Z]     #18 std::__1::__vector_base<std::__1::unique_ptr<prometheus::Family<prometheus::Gauge>, std::__1::default_delete<prometheus::Family<prometheus::Gauge> > >, std::__1::allocator<std::__1::unique_ptr<prometheus::Family<prometheus::Gauge>, std::__1::default_delete<prometheus::Family<prometheus::Gauge> > > > >::~__vector_base() /usr/local/bin/../include/c++/v1/vector:487:9 (gtests_dbms+0xdaca6fc)
[2022-06-07T18:07:38.677Z]     #19 std::__1::vector<std::__1::unique_ptr<prometheus::Family<prometheus::Gauge>, std::__1::default_delete<prometheus::Family<prometheus::Gauge> > >, std::__1::allocator<std::__1::unique_ptr<prometheus::Family<prometheus::Gauge>, std::__1::default_delete<prometheus::Family<prometheus::Gauge> > > > >::~vector() /usr/local/bin/../include/c++/v1/vector:579:5 (gtests_dbms+0xdaca6fc)
[2022-06-07T18:07:38.677Z]     #20 prometheus::Registry::~Registry() /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/contrib/prometheus-cpp/core/src/registry.cc:39:21 (gtests_dbms+0xdaca6fc)
[2022-06-07T18:07:38.678Z]     #21 std::__1::__shared_ptr_emplace<prometheus::Registry, std::__1::allocator<prometheus::Registry> >::__on_zero_shared() /usr/local/bin/../include/c++/v1/__memory/shared_ptr.h:315:24 (gtests_dbms+0x26509a7)
[2022-06-07T18:07:38.678Z]     #22 std::__1::__shared_count::__release_shared() /usr/local/bin/../include/c++/v1/__memory/shared_ptr.h:177:9 (gtests_dbms+0xd8534d1)
[2022-06-07T18:07:38.678Z]     #23 std::__1::__shared_weak_count::__release_shared() /usr/local/bin/../include/c++/v1/__memory/shared_ptr.h:219:27 (gtests_dbms+0xd8534d1)
[2022-06-07T18:07:38.678Z]     #24 std::__1::shared_ptr<prometheus::Registry>::~shared_ptr() /usr/local/bin/../include/c++/v1/__memory/shared_ptr.h:959:19 (gtests_dbms+0xd8534d1)
[2022-06-07T18:07:38.678Z]     #25 DB::TiFlashMetrics::~TiFlashMetrics() /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/dbms/src/Common/TiFlashMetrics.h:322:7 (gtests_dbms+0xd8534d1)
[2022-06-07T18:07:38.678Z]     #26 cxa_at_exit_wrapper(void*) /llvm-project/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:394:3 (gtests_dbms+0x24ffa9f)
[2022-06-07T18:07:38.678Z] 
[2022-06-07T18:07:38.678Z]   Previous atomic write of size 8 at 0x7b04000058d0 by thread T18:
[2022-06-07T18:07:38.678Z]     [failed to restore the stack]
[2022-06-07T18:07:38.678Z] 
[2022-06-07T18:07:38.678Z]   As if synchronized via sleep:
[2022-06-07T18:07:38.678Z]     #0 nanosleep /llvm-project/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:363:3 (gtests_dbms+0x24c0bef)
[2022-06-07T18:07:38.678Z]     #1 std::__1::this_thread::sleep_for(std::__1::chrono::duration<long long, std::__1::ratio<1l, 1000000000l> > const&) <null> (libc++.so.1+0x709ba)
[2022-06-07T18:07:38.678Z]     #2 DB::tests::(anonymous namespace)::DynamicThreadPoolTest_testAutoExpanding_Test::TestBody() /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/dbms/src/Common/tests/gtest_dynamic_thread_pool.cpp:61:1 (gtests_dbms+0x25883ec)
[2022-06-07T18:07:38.678Z]     #3 void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/contrib/googletest/googletest/src/gtest.cc:2401:10 (gtests_dbms+0x9985715)
[2022-06-07T18:07:38.678Z]     #4 void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/contrib/googletest/googletest/src/gtest.cc:2437:14 (gtests_dbms+0x9985715)
[2022-06-07T18:07:38.678Z]     #5 testing::Test::Run() /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/contrib/googletest/googletest/src/gtest.cc:2473:5 (gtests_dbms+0x9965435)
[2022-06-07T18:07:38.678Z]     #6 testing::TestInfo::Run() /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/contrib/googletest/googletest/src/gtest.cc:2655:11 (gtests_dbms+0x9966c01)
[2022-06-07T18:07:38.678Z]     #7 testing::TestCase::Run() /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/contrib/googletest/googletest/src/gtest.cc:2773:28 (gtests_dbms+0x99678bd)
[2022-06-07T18:07:38.678Z]     #8 testing::internal::UnitTestImpl::RunAllTests() /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/contrib/googletest/googletest/src/gtest.cc:4673:43 (gtests_dbms+0x9972c40)
[2022-06-07T18:07:38.678Z]     #9 bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/contrib/googletest/googletest/src/gtest.cc:2401:10 (gtests_dbms+0x99867d5)
[2022-06-07T18:07:38.678Z]     #10 bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/contrib/googletest/googletest/src/gtest.cc:2437:14 (gtests_dbms+0x99867d5)
[2022-06-07T18:07:38.678Z]     #11 testing::UnitTest::Run() /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/contrib/googletest/googletest/src/gtest.cc:4281:10 (gtests_dbms+0x99720c3)
[2022-06-07T18:07:38.678Z]     #12 RUN_ALL_TESTS() /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/contrib/googletest/googletest/include/gtest/gtest.h:2237:46 (gtests_dbms+0x3f6d827)
[2022-06-07T18:07:38.678Z]     #13 main /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/dbms/src/TestUtils/gtests_dbms_main.cpp:36:16 (gtests_dbms+0x3f6d827)
[2022-06-07T18:07:38.678Z] 
[2022-06-07T18:07:38.678Z]   Thread T18 (tid=18488, finished) created by main thread at:
[2022-06-07T18:07:38.678Z]     #0 pthread_create /llvm-project/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:977:3 (gtests_dbms+0x24c327d)
[2022-06-07T18:07:38.678Z]     #1 std::__1::__libcpp_thread_create(unsigned long*, void* (*)(void*), void*) /usr/local/bin/../include/c++/v1/__threading_support:514:10 (gtests_dbms+0xd7e6062)
[2022-06-07T18:07:38.678Z]     #2 std::__1::thread::thread<std::__1::thread DB::ThreadFactory::newThread<void (DB::DynamicThreadPool::*)(std::__1::unique_ptr<DB::IExecutableTask, std::__1::default_delete<DB::IExecutableTask> >), DB::DynamicThreadPool*, std::__1::unique_ptr<DB::IExecutableTask, std::__1::default_delete<DB::IExecutableTask> > >(bool, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, void (DB::DynamicThreadPool::*&&)(std::__1::unique_ptr<DB::IExecutableTask, std::__1::default_delete<DB::IExecutableTask> >), DB::DynamicThreadPool*&&, std::__1::unique_ptr<DB::IExecutableTask, std::__1::default_delete<DB::IExecutableTask> >&&)::'lambda'(auto&&...)&, DB::DynamicThreadPool*, std::__1::unique_ptr<DB::IExecutableTask, std::__1::default_delete<DB::IExecutableTask> >, void>(void (DB::DynamicThreadPool::*&&)(std::__1::unique_ptr<DB::IExecutableTask, std::__1::default_delete<DB::IExecutableTask> >), DB::DynamicThreadPool*&&, std::__1::unique_ptr<DB::IExecutableTask, std::__1::default_delete<DB::IExecutableTask> >&&) /usr/local/bin/../include/c++/v1/thread:307:16 (gtests_dbms+0xd7e6062)
[2022-06-07T18:07:38.678Z]     #3 std::__1::thread DB::ThreadFactory::newThread<void (DB::DynamicThreadPool::*)(std::__1::unique_ptr<DB::IExecutableTask, std::__1::default_delete<DB::IExecutableTask> >), DB::DynamicThreadPool*, std::__1::unique_ptr<DB::IExecutableTask, std::__1::default_delete<DB::IExecutableTask> > >(bool, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, void (DB::DynamicThreadPool::*&&)(std::__1::unique_ptr<DB::IExecutableTask, std::__1::default_delete<DB::IExecutableTask> >), DB::DynamicThreadPool*&&, std::__1::unique_ptr<DB::IExecutableTask, std::__1::default_delete<DB::IExecutableTask> >&&) /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/dbms/src/Common/ThreadFactory.h:49:16 (gtests_dbms+0xd7e32f5)
[2022-06-07T18:07:38.678Z]     #4 DB::DynamicThreadPool::scheduledToNewDynamicThread(std::__1::unique_ptr<DB::IExecutableTask, std::__1::default_delete<DB::IExecutableTask> >&) /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/dbms/src/Common/DynamicThreadPool.cpp:94:21 (gtests_dbms+0xd7e32f5)
[2022-06-07T18:07:38.678Z]     #5 DB::DynamicThreadPool::scheduleTask(std::__1::unique_ptr<DB::IExecutableTask, std::__1::default_delete<DB::IExecutableTask> >) /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/dbms/src/Common/DynamicThreadPool.cpp:64:9 (gtests_dbms+0xd7e3066)
[2022-06-07T18:07:38.679Z]     #6 auto DB::DynamicThreadPool::schedule<DB::tests::(anonymous namespace)::DynamicThreadPoolTest_testAutoExpanding_Test::TestBody()::$_1>(bool, DB::tests::(anonymous namespace)::DynamicThreadPoolTest_testAutoExpanding_Test::TestBody()::$_1&&) /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/dbms/src/Common/DynamicThreadPool.h:58:9 (gtests_dbms+0x2587f16)
[2022-06-07T18:07:38.679Z]     #7 DB::tests::(anonymous namespace)::DynamicThreadPoolTest_testAutoExpanding_Test::TestBody() /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/dbms/src/Common/tests/gtest_dynamic_thread_pool.cpp:48:20 (gtests_dbms+0x2587f16)
[2022-06-07T18:07:38.679Z]     #8 void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/contrib/googletest/googletest/src/gtest.cc:2401:10 (gtests_dbms+0x9985715)
[2022-06-07T18:07:38.679Z]     #9 void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/contrib/googletest/googletest/src/gtest.cc:2437:14 (gtests_dbms+0x9985715)
[2022-06-07T18:07:38.679Z]     #10 testing::Test::Run() /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/contrib/googletest/googletest/src/gtest.cc:2473:5 (gtests_dbms+0x9965435)
[2022-06-07T18:07:38.679Z]     #11 testing::TestInfo::Run() /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/contrib/googletest/googletest/src/gtest.cc:2655:11 (gtests_dbms+0x9966c01)
[2022-06-07T18:07:38.679Z]     #12 testing::TestCase::Run() /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/contrib/googletest/googletest/src/gtest.cc:2773:28 (gtests_dbms+0x99678bd)
[2022-06-07T18:07:38.679Z]     #13 testing::internal::UnitTestImpl::RunAllTests() /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/contrib/googletest/googletest/src/gtest.cc:4673:43 (gtests_dbms+0x9972c40)
[2022-06-07T18:07:38.679Z]     #14 bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/contrib/googletest/googletest/src/gtest.cc:2401:10 (gtests_dbms+0x99867d5)
[2022-06-07T18:07:38.679Z]     #15 bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/contrib/googletest/googletest/src/gtest.cc:2437:14 (gtests_dbms+0x99867d5)
[2022-06-07T18:07:38.679Z]     #16 testing::UnitTest::Run() /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/contrib/googletest/googletest/src/gtest.cc:4281:10 (gtests_dbms+0x99720c3)
[2022-06-07T18:07:38.679Z]     #17 RUN_ALL_TESTS() /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/contrib/googletest/googletest/include/gtest/gtest.h:2237:46 (gtests_dbms+0x3f6d827)
[2022-06-07T18:07:38.679Z]     #18 main /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/dbms/src/TestUtils/gtests_dbms_main.cpp:36:16 (gtests_dbms+0x3f6d827)
[2022-06-07T18:07:38.679Z] 
[2022-06-07T18:07:38.679Z] SUMMARY: ThreadSanitizer: data race /llvm-project/compiler-rt/lib/tsan/rtl/tsan_new_delete.cpp:126:3 in operator delete(void*)
[2022-06-07T18:07:38.679Z] ==================
[2022-06-07T18:07:38.679Z] ThreadSanitizer: reported 2 warnings
  1. valgrind get free error
[ RUN      ] DynamicThreadPoolTest.testAutoExpanding
==21335== Invalid free() / delete / delete[] / realloc()
==21335==    at 0xB34D06D: free (vg_replace_malloc.c:540)
==21335==    by 0x46C45A: __gnu_cxx::new_allocator<boost::lockfree::queue<DB::MPMCQueue<std::unique_ptr<DB::IExecutableTask, std::default_delete<DB::IExecutableTask> > >*>::node>::deallocate(boost::lockfree::queue<DB::MPMCQueue<std::unique_ptr<DB::IExecutableTask, std::default_delete<DB::IExecutableTask> > >*>::node*, unsigned long) (new_allocator.h:121)
==21335==    by 0x46CD9F: boost::lockfree::detail::freelist_stack<boost::lockfree::queue<DB::MPMCQueue<std::unique_ptr<DB::IExecutableTask, std::default_delete<DB::IExecutableTask> > >*>::node, std::allocator<boost::lockfree::queue<DB::MPMCQueue<std::unique_ptr<DB::IExecutableTask, std::default_delete<DB::IExecutableTask> > >*>::node> >::~freelist_stack() (freelist.hpp:132)
==21335==    by 0x46E397: boost::lockfree::queue<DB::MPMCQueue<std::unique_ptr<DB::IExecutableTask, std::default_delete<DB::IExecutableTask> > >*>::~queue() (queue.hpp:281)
==21335==    by 0x5E60276: DB::DynamicThreadPool::~DynamicThreadPool() (DynamicThreadPool.cpp:20)
==21335==    by 0x464634: DB::tests::(anonymous namespace)::DynamicThreadPoolTest_testAutoExpanding_Test::TestBody() (gtest_dynamic_thread_pool.cpp:29)
==21335==    by 0x41C30C2: void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) (gtest.cc:2401)
==21335==    by 0x41CC9BB: void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) (gtest.cc:2437)
==21335==    by 0x41BDEA8: testing::Test::Run() (gtest.cc:2473)
==21335==    by 0x41BE05C: testing::TestInfo::Run() (gtest.cc:2655)
==21335==    by 0x41BE108: testing::TestCase::Run() (gtest.cc:2773)
==21335==    by 0x41BF5C8: testing::internal::UnitTestImpl::RunAllTests() (gtest.cc:4673)
==21335==  Address 0xa74c040 is in a rw- anonymous segment
==21335==

2. What did you expect to see? (Required)

3. What did you see instead (Required)

4. What is your TiFlash version? (Required)

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions