Skip to content

Race condition in drain sequence#3

Closed
aunu53 wants to merge 0 commit intomasterfrom
integration
Closed

Race condition in drain sequence#3
aunu53 wants to merge 0 commit intomasterfrom
integration

Conversation

@aunu53
Copy link
Copy Markdown
Owner

@aunu53 aunu53 commented May 18, 2020

No description provided.

@aunu53
Copy link
Copy Markdown
Owner Author

aunu53 commented May 18, 2020 •
edited
Loading

bazel test test/integration:protocol_integration_test --test_sharding_strategy=disabled --test_filter="*DrainClose/IPv4_Http2Downstream_HttpUpstream" --test_output=all --test_arg="-l trace"

[AUNI] waitFordDisconnect()
[2020-05-18 14:47:40.615][19][info][main] [source/server/drain_manager_impl.cc:68] shutting down parent after drain
[2020-05-18 14:47:43.562][19][debug][main] [source/server/server.cc:187] flushing stats
[2020-05-18 14:47:43.649][25][trace][http2] [source/common/http/http2/nghttp2.cc:25] nghttp2: stream: adjusting kept idle streams num_idle_streams=0, max=100
[2020-05-18 14:47:43.649][25][trace][http2] [source/common/http/http2/nghttp2.cc:25] nghttp2: send: next frame: payloadlen=8, type=7, flags=0x00, stream_id=0
[2020-05-18 14:47:43.649][25][trace][http2] [source/common/http/http2/codec_impl.cc:736] [C1] about to send frame type=7, flags=0
[2020-05-18 14:47:43.649][25][trace][http2] [source/common/http/http2/nghttp2.cc:25] nghttp2: send: start transmitting frame type=7, length=17
[2020-05-18 14:47:43.649][25][trace][http2] [source/common/http/http2/codec_impl.cc:790] [C1] send data: bytes=17
[2020-05-18 14:47:43.649][25][trace][connection] [source/common/network/connection_impl.cc:426] [C1] writing 17 bytes, end_stream false
[2020-05-18 14:47:43.649][25][trace][http2] [source/common/http/http2/nghttp2.cc:25] nghttp2: stream: adjusting kept idle streams num_idle_streams=0, max=100
[2020-05-18 14:47:43.649][25][trace][http2] [source/common/http/http2/nghttp2.cc:25] nghttp2: send: end transmission of a frame
[2020-05-18 14:47:43.649][25][trace][http2] [source/common/http/http2/codec_impl.cc:679] [C1] sent frame type=7
[2020-05-18 14:47:43.649][25][debug][http2] [source/common/http/http2/codec_impl.cc:682] [C1] sent goaway code=0
[2020-05-18 14:47:43.649][25][trace][http2] [source/common/http/http2/nghttp2.cc:25] nghttp2: send: reset nghttp2_active_outbound_item
[2020-05-18 14:47:43.649][25][trace][http2] [source/common/http/http2/nghttp2.cc:25] nghttp2: send: aob->item = 0x779db80
[2020-05-18 14:47:43.649][25][debug][connection] [source/common/network/connection_impl.cc:109] [C1] closing data_to_write=17 type=2
[2020-05-18 14:47:43.649][25][debug][connection] [source/common/network/connection_impl_base.cc:30] [C1] setting delayed close timer with timeout 1000 ms
[2020-05-18 14:47:43.649][25][trace][connection] [source/common/network/connection_impl.cc:490] [C1] socket event: 2
[2020-05-18 14:47:43.649][25][trace][connection] [source/common/network/connection_impl.cc:578] [C1] write ready
[2020-05-18 14:47:43.649][25][trace][connection] [source/common/network/raw_buffer_socket.cc:68] [C1] write returns: 17
[2020-05-18 14:47:43.649][25][debug][connection] [source/common/network/connection_impl.cc:619] [C1] write flush complete
[2020-05-18 14:47:43.649][12][trace][connection] [source/common/network/connection_impl.cc:490] [C0] socket event: 3
[2020-05-18 14:47:43.649][12][trace][connection] [source/common/network/connection_impl.cc:578] [C0] write ready
[2020-05-18 14:47:43.649][12][trace][connection] [source/common/network/connection_impl.cc:528] [C0] read ready
[2020-05-18 14:47:43.649][12][trace][connection] [source/common/network/raw_buffer_socket.cc:25] [C0] read returns: 17
[2020-05-18 14:47:43.649][12][trace][connection] [source/common/network/raw_buffer_socket.cc:39] [C0] read error: Resource temporarily unavailable
[2020-05-18 14:47:43.649][12][trace][http2] [source/common/http/http2/codec_impl.cc:488] [C0] dispatching 17 bytes
[2020-05-18 14:47:43.649][12][trace][http2] [source/common/http/http2/nghttp2.cc:25] nghttp2: recv: connection recv_window_size=0, local_window=268435456
[2020-05-18 14:47:43.649][12][trace][http2] [source/common/http/http2/nghttp2.cc:25] nghttp2: stream: adjusting kept idle streams num_idle_streams=0, max=100
[2020-05-18 14:47:43.649][12][trace][http2] [source/common/http/http2/codec_impl.cc:508] [C0] dispatched 17 bytes
[2020-05-18 14:47:43.649][12][trace][http2] [source/common/http/http2/nghttp2.cc:25] nghttp2: stream: adjusting kept idle streams num_idle_streams=0, max=100
[2020-05-18 14:47:44.651][25][debug][connection] [source/common/network/connection_impl_base.cc:44] [C1] triggered delayed close
[2020-05-18 14:47:44.651][25][debug][connection] [source/common/network/connection_impl.cc:200] [C1] closing socket: 1
[2020-05-18 14:47:44.651][25][debug][conn_handler] [source/server/connection_handler_impl.cc:110] [C1] adding to cleanup list
[2020-05-18 14:47:44.651][12][trace][connection] [source/common/network/connection_impl.cc:490] [C0] socket event: 3
[2020-05-18 14:47:44.652][12][trace][connection] [source/common/network/connection_impl.cc:578] [C0] write ready
[2020-05-18 14:47:44.652][25][trace][main] [source/common/event/dispatcher_impl.cc:169] item added to deferred deletion list (size=1)
[2020-05-18 14:47:44.652][12][trace][connection] [source/common/network/connection_impl.cc:528] [C0] read ready
[2020-05-18 14:47:44.652][25][trace][main] [source/common/event/dispatcher_impl.cc:169] item added to deferred deletion list (size=2)
[2020-05-18 14:47:44.652][25][trace][main] [source/common/event/dispatcher_impl.cc:83] clearing deferred deletion list (size=2)
[2020-05-18 14:47:44.652][12][trace][connection] [source/common/network/raw_buffer_socket.cc:25] [C0] read returns: 0
[2020-05-18 14:47:44.652][12][debug][connection] [source/common/network/connection_impl.cc:555] [C0] remote close
[2020-05-18 14:47:44.652][12][debug][connection] [source/common/network/connection_impl.cc:200] [C0] closing socket: 0
[2020-05-18 14:47:44.652][12][debug][client] [source/common/http/codec_client.cc:92] [C0] disconnect. resetting 0 pending requests
[2020-05-18 14:47:44.652][12][info][testing] [test/integration/server.cc:217] stopping integration test server
[2020-05-18 14:47:44.652][19][info][main] [source/server/server.cc:681] shutting down server instance
[2020-05-18 14:47:44.652][19][info][main] [source/server/server.cc:628] main dispatch loop exited
[2020-05-18 14:47:44.652][25][trace][http2] [source/common/http/http2/nghttp2.cc:25] nghttp2: send: reset nghttp2_active_outbound_item
[2020-05-18 14:47:44.652][25][trace][http2] [source/common/http/http2/nghttp2.cc:25] nghttp2: send: aob->item = (nil)
[2020-05-18 14:47:44.652][25][debug][main] [source/server/worker_impl.cc:120] worker exited dispatch loop
[2020-05-18 14:47:44.652][25][debug][grpc] [source/common/grpc/google_async_client_impl.cc:38] Joining completionThread
[2020-05-18 14:47:44.652][26][debug][grpc] [source/common/grpc/google_async_client_impl.cc:71] completionThread exiting
[2020-05-18 14:47:44.652][25][debug][grpc] [source/common/grpc/google_async_client_impl.cc:40] Joined completionThread
[2020-05-18 14:47:44.652][25][debug][upstream] [source/common/upstream/cluster_manager_impl.cc:980] shutting down thread local cluster manager
[2020-05-18 14:47:44.653][19][debug][main] [source/server/server.cc:187] flushing stats
[2020-05-18 14:47:44.654][19][debug][init] [source/common/init/watcher_impl.cc:27] ClusterImplBase destroyed
[2020-05-18 14:47:44.654][19][debug][init] [source/common/init/watcher_impl.cc:27] init manager Cluster cluster_0 destroyed
[2020-05-18 14:47:44.654][19][debug][grpc] [source/common/grpc/google_async_client_impl.cc:38] Joining completionThread
[2020-05-18 14:47:44.654][23][debug][grpc] [source/common/grpc/google_async_client_impl.cc:71] completionThread exiting
[2020-05-18 14:47:44.654][19][debug][grpc] [source/common/grpc/google_async_client_impl.cc:40] Joined completionThread
[2020-05-18 14:47:44.655][19][debug][upstream] [source/common/upstream/cluster_manager_impl.cc:980] shutting down thread local cluster manager
[2020-05-18 14:47:44.656][19][info][main] [source/server/server.cc:674] exiting
[2020-05-18 14:47:44.656][19][debug][init] [source/common/init/watcher_impl.cc:27] RunHelper destroyed
[2020-05-18 14:47:44.656][19][debug][main] [source/server/server.cc:129] destroying listener manager
[2020-05-18 14:47:44.656][19][debug][init] [source/common/init/target_impl.cc:32] target LDS destroyed
[2020-05-18 14:47:44.656][19][debug][init] [source/common/init/watcher_impl.cc:27] Listener-local-init-watcher http destroyed
[2020-05-18 14:47:44.657][19][debug][init] [source/common/init/watcher_impl.cc:27] init manager Listener-local-init-manager http 829003187647530112 destroyed
[2020-05-18 14:47:44.657][19][debug][init] [source/common/init/target_impl.cc:32] target Listener-init-target http destroyed
[2020-05-18 14:47:44.657][19][debug][main] [source/server/server.cc:131] destroyed listener manager
[2020-05-18 14:47:44.657][19][debug][main] [source/common/access_log/access_log_manager_impl.cc:16] destroying access logger /dev/null
[2020-05-18 14:47:44.657][19][debug][main] [source/common/access_log/access_log_manager_impl.cc:19] destroyed access loggers
[2020-05-18 14:47:44.657][19][debug][init] [source/common/init/watcher_impl.cc:27] init manager RTDS destroyed
[2020-05-18 14:47:44.658][19][debug][init] [source/common/init/watcher_impl.cc:27] RDTS destroyed
[2020-05-18 14:47:44.658][19][debug][init] [source/common/init/watcher_impl.cc:27] init manager Server destroyed
[2020-05-18 14:47:44.660][12][trace][http2] [source/common/http/http2/nghttp2.cc:25] nghttp2: send: reset nghttp2_active_outbound_item
[2020-05-18 14:47:44.660][12][trace][http2] [source/common/http/http2/nghttp2.cc:25] nghttp2: send: aob->item = (nil)

@aunu53
Copy link
Copy Markdown
Owner Author

aunu53 commented May 18, 2020

bazel test test/integration:protocol_integration_test --test_sharding_strategy=disabled --test_filter="*DrainClose/IPv4_Http2Downstream_HttpUpstream" --test_output=all --test_arg="-l trace"

[AUNI] waitFordDisconnect()
[2020-05-18 14:53:25.798][25][trace][connection] [source/common/network/connection_impl.cc:490] [C1] socket event: 3
[2020-05-18 14:53:25.798][25][trace][connection] [source/common/network/connection_impl.cc:578] [C1] write ready
[2020-05-18 14:53:25.798][25][trace][connection] [source/common/network/connection_impl.cc:528] [C1] read ready
[2020-05-18 14:53:25.798][25][trace][connection] [source/common/network/raw_buffer_socket.cc:25] [C1] read returns: 9
[2020-05-18 14:53:25.798][25][trace][connection] [source/common/network/raw_buffer_socket.cc:39] [C1] read error: Resource temporarily unavailable
[2020-05-18 14:53:25.798][25][trace][http2] [source/common/http/http2/codec_impl.cc:488] [C1] dispatching 9 bytes
[2020-05-18 14:53:25.798][25][trace][http2] [source/common/http/http2/nghttp2.cc:25] nghttp2: recv: connection recv_window_size=0, local_window=268435456
[2020-05-18 14:53:25.798][25][trace][http2] [source/common/http/http2/nghttp2.cc:25] nghttp2: stream: adjusting kept idle streams num_idle_streams=0, max=100
[2020-05-18 14:53:25.798][25][trace][http2] [source/common/http/http2/nghttp2.cc:25] nghttp2: recv: [IB_READ_HEAD]
[2020-05-18 14:53:25.798][25][trace][http2] [source/common/http/http2/nghttp2.cc:25] nghttp2: recv: payloadlen=0, type=4, flags=0x01, stream_id=0
[2020-05-18 14:53:25.798][25][trace][http2] [source/common/http/http2/nghttp2.cc:25] nghttp2: recv: SETTINGS
[2020-05-18 14:53:25.798][25][trace][http2] [source/common/http/http2/codec_impl.cc:553] [C1] about to recv frame type=4, flags=1
[2020-05-18 14:53:25.798][25][trace][http2] [source/common/http/http2/codec_impl.cc:1280] [C1] track inbound frame type=4 flags=1 length=0 padding_length=0
[2020-05-18 14:53:25.798][25][trace][http2] [source/common/http/http2/nghttp2.cc:25] nghttp2: recv: [IB_READ_SETTINGS]
[2020-05-18 14:53:25.798][25][trace][http2] [source/common/http/http2/nghttp2.cc:25] nghttp2: recv: readlen=0, payloadleft=0
[2020-05-18 14:53:25.798][25][trace][http2] [source/common/http/http2/codec_impl.cc:568] [C1] recv frame type=4
[2020-05-18 14:53:25.798][25][trace][http2] [source/common/http/http2/codec_impl.cc:508] [C1] dispatched 9 bytes
[2020-05-18 14:53:25.798][25][trace][http2] [source/common/http/http2/nghttp2.cc:25] nghttp2: stream: adjusting kept idle streams num_idle_streams=0, max=100
[2020-05-18 14:53:26.783][19][trace][main] [source/server/drain_manager_impl.cc:44] drain tick #1
[2020-05-18 14:53:27.765][19][info][main] [source/server/drain_manager_impl.cc:68] shutting down parent after drain
[2020-05-18 14:53:27.782][19][trace][main] [source/server/drain_manager_impl.cc:44] drain tick #2
[2020-05-18 14:53:28.784][19][trace][main] [source/server/drain_manager_impl.cc:44] drain tick #3
[2020-05-18 14:53:29.782][19][trace][main] [source/server/drain_manager_impl.cc:44] drain tick #4
[2020-05-18 14:53:30.709][19][debug][main] [source/server/server.cc:187] flushing stats
[2020-05-18 14:53:30.783][19][trace][main] [source/server/drain_manager_impl.cc:44] drain tick #5
[2020-05-18 14:53:31.785][19][trace][main] [source/server/drain_manager_impl.cc:44] drain tick #6
[2020-05-18 14:53:32.783][19][trace][main] [source/server/drain_manager_impl.cc:44] drain tick #7
[2020-05-18 14:53:33.782][19][trace][main] [source/server/drain_manager_impl.cc:44] drain tick #8
[2020-05-18 14:53:34.784][19][trace][main] [source/server/drain_manager_impl.cc:44] drain tick #9
[2020-05-18 14:53:35.706][19][debug][main] [source/server/server.cc:187] flushing stats
[2020-05-18 14:53:40.708][19][debug][main] [source/server/server.cc:187] flushing stats
[2020-05-18 14:53:45.706][19][debug][main] [source/server/server.cc:187] flushing stats
[2020-05-18 14:53:50.707][19][debug][main] [source/server/server.cc:187] flushing stats
[2020-05-18 14:53:55.708][19][debug][main] [source/server/server.cc:187] flushing stats
[2020-05-18 14:54:00.706][19][debug][main] [source/server/server.cc:187] flushing stats
[2020-05-18 14:54:05.713][19][debug][main] [source/server/server.cc:187] flushing stats
[2020-05-18 14:54:10.715][19][debug][main] [source/server/server.cc:187] flushing stats

@aunu53 aunu53 mentioned this pull request May 18, 2020
Closed
@aunu53
Copy link
Copy Markdown
Owner Author

aunu53 commented May 18, 2020

Ah bless, TSAN picks up on IPv6.

[AUNI] waitFordEndStream()
==================
WARNING: ThreadSanitizer: data race (pid=12)
  Write of size 8 at 0x7b9000004000 by thread T8:
    #0 memcpy ??:? (protocol_integration_test+0x25b82cd)
    #1 std::basic_streambuf<char, std::char_traits<char> >::xsputn(char const*, long) ??:? (libstdc++.so.6+0x131c37)
    #2 Envoy::Logger::StderrSinkDelegate::log(absl::string_view) ??:? (protocol_integration_test+0x659f671)
    #3 Envoy::Logger::DelegatingLogSink::log(spdlog::details::log_msg const&) ??:? (protocol_integration_test+0x659fc79)
    #4 spdlog::logger::sink_it_(spdlog::details::log_msg const&) ??:? (protocol_integration_test+0x269ca2f)
    #5 void spdlog::logger::log<unsigned long, unsigned int>(spdlog::source_loc, spdlog::level::level_enum, fmt::v6::basic_string_view<char>, unsigned long const&, unsigned int const&) ??:? (protocol_integration_test+0x2c97afe)
    #6 Envoy::Network::ConnectionImpl::onFileEvent(unsigned int) ??:? (protocol_integration_test+0x4a42a78)
    #7 Envoy::Network::ConnectionImpl::ConnectionImpl(Envoy::Event::Dispatcher&, std::unique_ptr<Envoy::Network::ConnectionSocket, std::default_delete<Envoy::Network::ConnectionSocket> >&&, std::unique_ptr<Envoy::Network::TransportSocket, std::default_delete<Envoy::Network::TransportSocket> >&&, Envoy::StreamInfo::StreamInfo&, bool)::$_2::operator()(unsigned int) const connection_impl.cc:? (protocol_integration_test+0x4a48ea4)
    #8 std::_Function_handler<void (unsigned int), Envoy::Network::ConnectionImpl::ConnectionImpl(Envoy::Event::Dispatcher&, std::unique_ptr<Envoy::Network::ConnectionSocket, std::default_delete<Envoy::Network::ConnectionSocket> >&&, std::unique_ptr<Envoy::Network::TransportSocket, std::default_delete<Envoy::Network::TransportSocket> >&&, Envoy::StreamInfo::StreamInfo&, bool)::$_2>::_M_invoke(std::_Any_data const&, unsigned int&&) connection_impl.cc:? (protocol_integration_test+0x4a48c51)
    #9 std::function<void (unsigned int)>::operator()(unsigned int) const ??:? (protocol_integration_test+0x3c0fecd)
    #10 Envoy::Event::FileEventImpl::assignEvents(unsigned int, event_base*)::$_0::operator()(int, short, void*) const file_event_impl.cc:? (protocol_integration_test+0x4a3181c)
    #11 Envoy::Event::FileEventImpl::assignEvents(unsigned int, event_base*)::$_0::__invoke(int, short, void*) file_event_impl.cc:? (protocol_integration_test+0x4a31737)
    #12 event_persist_closure event.c:? (protocol_integration_test+0x65382e4)
    #13 event_process_active_single_queue event.c:? (protocol_integration_test+0x6536da3)
    #14 event_process_active event.c:? (protocol_integration_test+0x652c95d)
    #15 event_base_loop ??:? (protocol_integration_test+0x652a6ba)
    #16 Envoy::Event::LibeventScheduler::run(Envoy::Event::Dispatcher::RunType) ??:? (protocol_integration_test+0x4c7f3a5)
    #17 Envoy::Event::DispatcherImpl::run(Envoy::Event::Dispatcher::RunType) ??:? (protocol_integration_test+0x4a25941)
    #18 Envoy::Server::WorkerImpl::threadRoutine(Envoy::Server::GuardDog&) ??:? (protocol_integration_test+0x2d5f84e)
    #19 Envoy::Server::WorkerImpl::start(Envoy::Server::GuardDog&)::$_4::operator()() const worker_impl.cc:? (protocol_integration_test+0x2d620d4)
    #20 std::_Function_handler<void (), Envoy::Server::WorkerImpl::start(Envoy::Server::GuardDog&)::$_4>::_M_invoke(std::_Any_data const&) worker_impl.cc:? (protocol_integration_test+0x2d61e6a)
    #21 std::function<void ()>::operator()() const ??:? (protocol_integration_test+0x284ff2e)
    #22 Envoy::Thread::ThreadImplPosix::ThreadImplPosix(std::function<void ()>)::$_0::operator()(void*) const thread_impl.cc:? (protocol_integration_test+0x650f3b8)
    #23 Envoy::Thread::ThreadImplPosix::ThreadImplPosix(std::function<void ()>)::$_0::__invoke(void*) thread_impl.cc:? (protocol_integration_test+0x650f348)

  Previous write of size 8 at 0x7b9000004000 by main thread:
    #0 memcpy ??:? (protocol_integration_test+0x25b82cd)
    #1 std::basic_streambuf<char, std::char_traits<char> >::xsputn(char const*, long) ??:? (libstdc++.so.6+0x131c37)
    #2 void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) ??:? (protocol_integration_test+0x66c20b6)
    #3 void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) ??:? (protocol_integration_test+0x66a88f1)
    #4 testing::Test::Run() ??:? (protocol_integration_test+0x6687d2b)
    #5 testing::TestInfo::Run() ??:? (protocol_integration_test+0x6688e59)
    #6 testing::TestSuite::Run() ??:? (protocol_integration_test+0x6689930)
    #7 testing::internal::UnitTestImpl::RunAllTests() ??:? (protocol_integration_test+0x669de17)
    #8 bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) ??:? (protocol_integration_test+0x66c6e36)
    #9 bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) ??:? (protocol_integration_test+0x66ac997)
    #10 testing::UnitTest::Run() ??:? (protocol_integration_test+0x669d63e)
    #11 RUN_ALL_TESTS() ??:? (protocol_integration_test+0x49824c7)
    #12 Envoy::TestRunner::RunTests(int, char**) ??:? (protocol_integration_test+0x4981477)
    #13 main ??:? (protocol_integration_test+0x497e4f3)

  Location is heap block of size 8192 at 0x7b9000004000 allocated by main thread:
    #0 malloc ??:? (protocol_integration_test+0x25acfb7)
    #1 operator new(unsigned long) ??:? (libstdc++.so.6+0xa5f74)
    #2 Envoy::MessageUtil::loadFromFile(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, google::protobuf::Message&, Envoy::ProtobufMessage::ValidationVisitor&, Envoy::Api::Api&, bool) ??:? (protocol_integration_test+0x51ede8c)
    #3 Envoy::TestUtility::loadFromFile(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, google::protobuf::Message&, Envoy::Api::Api&, bool) ??:? (protocol_integration_test+0x345746d)
    #4 Envoy::ConfigHelper::ConfigHelper(Envoy::Network::Address::IpVersion, Envoy::Api::Api&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) ??:? (protocol_integration_test+0x344a6b6)
    #5 Envoy::BaseIntegrationTest::BaseIntegrationTest(std::function<std::shared_ptr<Envoy::Network::Address::Instance const> (int)> const&, Envoy::Network::Address::IpVersion, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) ??:? (protocol_integration_test+0x296f1e9)
    #6 Envoy::HttpIntegrationTest::HttpIntegrationTest(Envoy::Http::CodecClient::Type, std::function<std::shared_ptr<Envoy::Network::Address::Instance const> (int)> const&, Envoy::Network::Address::IpVersion, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) ??:? (protocol_integration_test+0x288c1cf)
    #7 Envoy::HttpIntegrationTest::HttpIntegrationTest(Envoy::Http::CodecClient::Type, Envoy::Network::Address::IpVersion, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) ??:? (protocol_integration_test+0x288bfb5)
    #8 Envoy::HttpProtocolIntegrationTest::HttpProtocolIntegrationTest() ??:? (protocol_integration_test+0x26a79c6)
    #9 Envoy::ProtocolIntegrationTest_DrainClose_Test::ProtocolIntegrationTest_DrainClose_Test() ??:? (protocol_integration_test+0x274baac)
    #10 testing::internal::ParameterizedTestFactory<Envoy::ProtocolIntegrationTest_DrainClose_Test>::CreateTest() ??:? (protocol_integration_test+0x274ba10)
    #11 testing::Test* testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::TestFactoryBase, testing::Test*>(testing::internal::TestFactoryBase*, testing::Test* (testing::internal::TestFactoryBase::*)(), char const*) ??:? (protocol_integration_test+0x66c2696)
    #12 testing::Test* testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::TestFactoryBase, testing::Test*>(testing::internal::TestFactoryBase*, testing::Test* (testing::internal::TestFactoryBase::*)(), char const*) ??:? (protocol_integration_test+0x66a92b7)
    #13 testing::TestInfo::Run() ??:? (protocol_integration_test+0x6688df3)
    #14 testing::TestSuite::Run() ??:? (protocol_integration_test+0x6689930)
    #15 testing::internal::UnitTestImpl::RunAllTests() ??:? (protocol_integration_test+0x669de17)
    #16 bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) ??:? (protocol_integration_test+0x66c6e36)
    #17 bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) ??:? (protocol_integration_test+0x66ac997)
    #18 testing::UnitTest::Run() ??:? (protocol_integration_test+0x669d63e)
    #19 RUN_ALL_TESTS() ??:? (protocol_integration_test+0x49824c7)
    #20 Envoy::TestRunner::RunTests(int, char**) ??:? (protocol_integration_test+0x4981477)
    #21 main ??:? (protocol_integration_test+0x497e4f3)

  Thread T8 (tid=26, running) created by thread T2 at:
    #0 pthread_create ??:? (protocol_integration_test+0x25ae4a5)
    #1 Envoy::Thread::ThreadImplPosix::ThreadImplPosix(std::function<void ()>) ??:? (protocol_integration_test+0x650ea84)
    #2 std::_MakeUniq<Envoy::Thread::ThreadImplPosix>::__single_object std::make_unique<Envoy::Thread::ThreadImplPosix, std::function<void ()>&>(std::function<void ()>&) ??:? (protocol_integration_test+0x650f883)
    #3 Envoy::Thread::ThreadFactoryImplPosix::createThread(std::function<void ()>) ??:? (protocol_integration_test+0x650efdb)
    #4 Envoy::Server::WorkerImpl::start(Envoy::Server::GuardDog&) ??:? (protocol_integration_test+0x2d5edf2)
    #5 Envoy::Server::ListenerManagerImpl::startWorkers(Envoy::Server::GuardDog&) ??:? (protocol_integration_test+0x2c2b589)
    #6 Envoy::Server::InstanceImpl::startWorkers() ??:? (protocol_integration_test+0x2b6ffb1)
    #7 Envoy::Server::InstanceImpl::run()::$_15::operator()() const server.cc:? (protocol_integration_test+0x2b78fa4)
    #8 std::_Function_handler<void (), Envoy::Server::InstanceImpl::run()::$_15>::_M_invoke(std::_Any_data const&) server.cc:? (protocol_integration_test+0x2b78d1a)
    #9 std::function<void ()>::operator()() const ??:? (protocol_integration_test+0x284ff2e)
    #10 Envoy::Server::RunHelper::RunHelper(Envoy::Server::Instance&, Envoy::Server::Options const&, Envoy::Event::Dispatcher&, Envoy::Upstream::ClusterManager&, Envoy::AccessLog::AccessLogManager&, Envoy::Init::Manager&, Envoy::Server::OverloadManager&, std::function<void ()>)::$_9::operator()() const server.cc:? (protocol_integration_test+0x2b7694f)
    #11 std::_Function_handler<void (), Envoy::Server::RunHelper::RunHelper(Envoy::Server::Instance&, Envoy::Server::Options const&, Envoy::Event::Dispatcher&, Envoy::Upstream::ClusterManager&, Envoy::AccessLog::AccessLogManager&, Envoy::Init::Manager&, Envoy::Server::OverloadManager&, std::function<void ()>)::$_9>::_M_invoke(std::_Any_data const&) server.cc:? (protocol_integration_test+0x2b765ba)
    #12 std::function<void ()>::operator()() const ??:? (protocol_integration_test+0x284ff2e)
    #13 Envoy::Init::WatcherHandleImpl::ready() const ??:? (protocol_integration_test+0x4ecb525)
    #14 Envoy::Init::ManagerImpl::ready() ??:? (protocol_integration_test+0x4ec1349)
    #15 Envoy::Init::ManagerImpl::onTargetReady() ??:? (protocol_integration_test+0x4ec1599)
    #16 Envoy::Init::ManagerImpl::ManagerImpl(absl::string_view)::$_0::operator()() const manager_impl.cc:? (protocol_integration_test+0x4ec1df8)
    #17 std::_Function_handler<void (), Envoy::Init::ManagerImpl::ManagerImpl(absl::string_view)::$_0>::_M_invoke(std::_Any_data const&) manager_impl.cc:? (protocol_integration_test+0x4ec1b8a)
    #18 std::function<void ()>::operator()() const ??:? (protocol_integration_test+0x284ff2e)
    #19 Envoy::Init::WatcherHandleImpl::ready() const ??:? (protocol_integration_test+0x4ecb525)
    #20 Envoy::Init::TargetImpl::ready() ??:? (protocol_integration_test+0x4ec4a27)
    #21 Envoy::Server::LdsApiImpl::onConfigUpdate(google::protobuf::RepeatedPtrField<envoy::service::discovery::v3::Resource> const&, google::protobuf::RepeatedPtrField<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) ??:? (protocol_integration_test+0x2d47f94)
    #22 Envoy::Server::LdsApiImpl::onConfigUpdate(google::protobuf::RepeatedPtrField<google::protobuf::Any> const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) ??:? (protocol_integration_test+0x2d48920)
    #23 non-virtual thunk to Envoy::Server::LdsApiImpl::onConfigUpdate(google::protobuf::RepeatedPtrField<google::protobuf::Any> const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) ??:? (protocol_integration_test+0x2d489fc)
    #24 Envoy::Config::FilesystemSubscriptionImpl::refresh() ??:? (protocol_integration_test+0x4679afb)
    #25 Envoy::Config::FilesystemSubscriptionImpl::start(std::set<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&) ??:? (protocol_integration_test+0x4679842)
    #26 Envoy::Server::LdsApiImpl::LdsApiImpl(envoy::config::core::v3::ConfigSource const&, Envoy::Upstream::ClusterManager&, Envoy::Init::Manager&, Envoy::Stats::Scope&, Envoy::Server::ListenerManager&, Envoy::ProtobufMessage::ValidationVisitor&)::$_0::operator()() const lds_api.cc:? (protocol_integration_test+0x2d49571)
    #27 std::_Function_handler<void (), Envoy::Server::LdsApiImpl::LdsApiImpl(envoy::config::core::v3::ConfigSource const&, Envoy::Upstream::ClusterManager&, Envoy::Init::Manager&, Envoy::Stats::Scope&, Envoy::Server::ListenerManager&, Envoy::ProtobufMessage::ValidationVisitor&)::$_0>::_M_invoke(std::_Any_data const&) lds_api.cc:? (protocol_integration_test+0x2d492ba)
    #28 std::function<void ()>::operator()() const ??:? (protocol_integration_test+0x284ff2e)
    #29 Envoy::Init::TargetImpl::TargetImpl(absl::string_view, std::function<void ()>)::$_0::operator()(std::unique_ptr<Envoy::Init::WatcherHandle, std::default_delete<Envoy::Init::WatcherHandle> >) const target_impl.cc:? (protocol_integration_test+0x4ec634e)
    #30 std::_Function_handler<void (std::unique_ptr<Envoy::Init::WatcherHandle, std::default_delete<Envoy::Init::WatcherHandle> >), Envoy::Init::TargetImpl::TargetImpl(absl::string_view, std::function<void ()>)::$_0>::_M_invoke(std::_Any_data const&, std::unique_ptr<Envoy::Init::WatcherHandle, std::default_delete<Envoy::Init::WatcherHandle> >&&) target_impl.cc:? (protocol_integration_test+0x4ec5fe3)
    #31 std::function<void (std::unique_ptr<Envoy::Init::WatcherHandle, std::default_delete<Envoy::Init::WatcherHandle> >)>::operator()(std::unique_ptr<Envoy::Init::WatcherHandle, std::default_delete<Envoy::Init::WatcherHandle> >) const ??:? (protocol_integration_test+0x4ec797f)
    #32 Envoy::Init::TargetHandleImpl::initialize(Envoy::Init::Watcher const&) const ??:? (protocol_integration_test+0x4ec4180)
    #33 Envoy::Init::ManagerImpl::initialize(Envoy::Init::Watcher const&) ??:? (protocol_integration_test+0x4ec11fd)
    #34 Envoy::Server::RunHelper::RunHelper(Envoy::Server::Instance&, Envoy::Server::Options const&, Envoy::Event::Dispatcher&, Envoy::Upstream::ClusterManager&, Envoy::AccessLog::AccessLogManager&, Envoy::Init::Manager&, Envoy::Server::OverloadManager&, std::function<void ()>)::$_14::operator()() const server.cc:? (protocol_integration_test+0x2b7882a)
    #35 std::_Function_handler<void (), Envoy::Server::RunHelper::RunHelper(Envoy::Server::Instance&, Envoy::Server::Options const&, Envoy::Event::Dispatcher&, Envoy::Upstream::ClusterManager&, Envoy::AccessLog::AccessLogManager&, Envoy::Init::Manager&, Envoy::Server::OverloadManager&, std::function<void ()>)::$_14>::_M_invoke(std::_Any_data const&) server.cc:? (protocol_integration_test+0x2b7826a)
    #36 std::function<void ()>::operator()() const ??:? (protocol_integration_test+0x284ff2e)
    #37 Envoy::Upstream::ClusterManagerInitHelper::setInitializedCb(std::function<void ()>) ??:? (protocol_integration_test+0x2e11ef4)
    #38 Envoy::Upstream::ClusterManagerImpl::setInitializedCb(std::function<void ()>) ??:? (protocol_integration_test+0x2e3de04)
    #39 Envoy::Server::RunHelper::RunHelper(Envoy::Server::Instance&, Envoy::Server::Options const&, Envoy::Event::Dispatcher&, Envoy::Upstream::ClusterManager&, Envoy::AccessLog::AccessLogManager&, Envoy::Init::Manager&, Envoy::Server::OverloadManager&, std::function<void ()>) ??:? (protocol_integration_test+0x2b70b80)
    #40 Envoy::Server::InstanceImpl::run() ??:? (protocol_integration_test+0x2b71380)
    #41 Envoy::IntegrationTestServerImpl::createAndRunEnvoyServer(Envoy::OptionsImpl&, Envoy::Event::TimeSystem&, std::shared_ptr<Envoy::Network::Address::Instance const>, Envoy::ListenerHooks&, Envoy::Thread::BasicLockable&, Envoy::Server::ComponentFactory&, std::unique_ptr<Envoy::Runtime::RandomGenerator, std::default_delete<Envoy::Runtime::RandomGenerator> >&&, absl::optional<std::reference_wrapper<Envoy::ProcessObject> >) ??:? (protocol_integration_test+0x29fe14e)
    #42 Envoy::IntegrationTestServer::threadRoutine(Envoy::Network::Address::IpVersion, bool, absl::optional<std::reference_wrapper<Envoy::ProcessObject> >, Envoy::Server::FieldValidationConfig, unsigned int, std::chrono::duration<long, std::ratio<1l, 1l> >) ??:? (protocol_integration_test+0x29fdb8b)
    #43 Envoy::IntegrationTestServer::start(Envoy::Network::Address::IpVersion, std::function<void ()>, bool, bool, absl::optional<std::reference_wrapper<Envoy::ProcessObject> >, Envoy::Server::FieldValidationConfig, unsigned int, std::chrono::duration<long, std::ratio<1l, 1l> >)::$_0::operator()() const server.cc:? (protocol_integration_test+0x29ffa2a)
    #44 std::_Function_handler<void (), Envoy::IntegrationTestServer::start(Envoy::Network::Address::IpVersion, std::function<void ()>, bool, bool, absl::optional<std::reference_wrapper<Envoy::ProcessObject> >, Envoy::Server::FieldValidationConfig, unsigned int, std::chrono::duration<long, std::ratio<1l, 1l> >)::$_0>::_M_invoke(std::_Any_data const&) server.cc:? (protocol_integration_test+0x29ff60a)
    #45 std::function<void ()>::operator()() const ??:? (protocol_integration_test+0x284ff2e)
    #46 Envoy::Thread::ThreadImplPosix::ThreadImplPosix(std::function<void ()>)::$_0::operator()(void*) const thread_impl.cc:? (protocol_integration_test+0x650f3b8)
    #47 Envoy::Thread::ThreadImplPosix::ThreadImplPosix(std::function<void ()>)::$_0::__invoke(void*) thread_impl.cc:? (protocol_integration_test+0x650f348)

SUMMARY: ThreadSanitizer: data race ??:? in __interceptor_memcpy
==================

@aunu53 aunu53 closed this May 23, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@aunu53