Skip to content

[CallAttemptTracer] heap-use-after-free when testing transparent retries #38728

Closed
Closed
[CallAttemptTracer] heap-use-after-free when testing transparent retries#38728
Assignees
yashykt
Labels
kind/buglang/corepriority/P2
@yashykt

Description

@yashykt
yashykt
opened on Feb 12, 2025

Noticed this on #38437 when adding a test for transparent retries.

The following heap-use-after-free is reported -

=================================================================
==3013632==ERROR: AddressSanitizer: heap-use-after-free on address 0x50d000006df0 at pc 0x7f5ce94ee68f bp 0x7f5cc9ffe8e0 sp 0x7f5cc9ffe8d8
READ of size 8 at 0x50d000006df0 thread T281
    #0 0x7f5ce94ee68e in (anonymous namespace)::TcpTracerIfSampled(grpc_chttp2_stream*) /proc/self/cwd/src/core/ext/transport/chttp2/transport/chttp2_transport.cc:250:63
    #1 0x7f5ce94da85d in perform_stream_op_locked(void*, absl::lts_20240722::Status) /proc/self/cwd/src/core/ext/transport/chttp2/transport/chttp2_transport.cc:1670:19
    #2 0x7f5ce2c51cd0 in grpc_combiner_continue_exec_ctx() /proc/self/cwd/src/core/lib/iomgr/combiner.cc:216:5
    #3 0x7f5ce2c5ca76 in grpc_core::ExecCtx::Flush() /proc/self/cwd/src/core/lib/iomgr/exec_ctx.cc:77:17
    #4 0x7f5cef337b25 in grpc_core::ExecCtx::~ExecCtx() /proc/self/cwd/./src/core/lib/iomgr/exec_ctx.h:137:5
    #5 0x7f5ce2c8cb3c in grpc_core::WorkSerializer::WorkSerializerImpl::Run() /proc/self/cwd/src/core/util/work_serializer.cc:221:1
    #6 0x7f5ce3e45da0 in grpc_event_engine::experimental::WorkStealingThreadPool::ThreadState::Step() /proc/self/cwd/src/core/lib/event_engine/thread_pool/work_stealing_thread_pool.cc:524:14
    #7 0x7f5ce3e44b59 in grpc_event_engine::experimental::WorkStealingThreadPool::ThreadState::ThreadBody() /proc/self/cwd/src/core/lib/event_engine/thread_pool/work_stealing_thread_pool.cc:487:10
    #8 0x7f5ce3e475e0 in grpc_event_engine::experimental::WorkStealingThreadPool::WorkStealingThreadPoolImpl::StartThread()::$_0::operator()(void*) const /proc/self/cwd/src/core/lib/event_engine/thread_pool/work_stealing_thread_pool.cc:257:17
    #9 0x7f5ce3e47558 in grpc_event_engine::experimental::WorkStealingThreadPool::WorkStealingThreadPoolImpl::StartThread()::$_0::__invoke(void*) /proc/self/cwd/src/core/lib/event_engine/thread_pool/work_stealing_thread_pool.cc:255:7
    #10 0x7f5ce21122be in grpc_core::(anonymous namespace)::ThreadInternalsPosix::ThreadInternalsPosix(char const*, void (*)(void*), void*, bool*, grpc_core::Thread::Options const&)::'lambda'(void*)::operator()(void*) const /proc/self/cwd/src/core/util/posix/thd.cc:145:11
    #11 0x7f5ce2111c27 in grpc_core::(anonymous namespace)::ThreadInternalsPosix::ThreadInternalsPosix(char const*, void (*)(void*), void*, bool*, grpc_core::Thread::Options const&)::'lambda'(void*)::__invoke(void*) /proc/self/cwd/src/core/util/posix/thd.cc:115:9
    #12 0x7f5ce0d646c1 in start_thread nptl/pthread_create.c:447:8
    #13 0x7f5ce0ddf127 in clone3 misc/../sysdeps/unix/sysv/linux/x86_64/clone3.S:78

0x50d000006df0 is located 0 bytes inside of 144-byte region [0x50d000006df0,0x50d000006e80)
freed by thread T281 here:
    #0 0x56484f6c7aca in __interceptor_free (/usr/local/google/home/yashkt/.cache/bazel/_bazel_yashkt/3933dcff60587ac8c34d87542411a8b2/execroot/com_github_grpc_grpc/bazel-out/k8-dbg/bin/test/cpp/ext/otel/otel_tracing_test+0xc7aca) (BuildId: 5b1372e32b37211dd5de56fa1466f4c9e5c2df8f)
    #1 0x7f5cef8a5811 in grpc::internal::OpenTelemetryPluginImpl::ClientCallTracer::CallAttemptTracer::~CallAttemptTracer() /proc/self/cwd/./src/cpp/ext/otel/otel_client_call_tracer.h:58:35
    #2 0x7f5cef89cc95 in grpc::internal::OpenTelemetryPluginImpl::ClientCallTracer::CallAttemptTracer::RecordEnd(gpr_timespec const&) /proc/self/cwd/src/cpp/ext/otel/otel_client_call_tracer.cc:278:5
    #3 0x7f5ce815237a in grpc_core::ClientChannelFilter::LoadBalancedCall::RecordLatency() /proc/self/cwd/src/core/client_channel/client_channel_filter.cc:2438:28
    #4 0x7f5ce81571d9 in grpc_core::ClientChannelFilter::FilterBasedLoadBalancedCall::Orphan() /proc/self/cwd/src/core/client_channel/client_channel_filter.cc:2652:3
    #5 0x7f5ce8179158 in void grpc_core::OrphanableDelete::operator()<grpc_core::ClientChannelFilter::FilterBasedLoadBalancedCall>(grpc_core::ClientChannelFilter::FilterBasedLoadBalancedCall*) /proc/self/cwd/./src/core/util/orphanable.h:60:8
    #6 0x7f5ce8178d2b in std::unique_ptr<grpc_core::ClientChannelFilter::FilterBasedLoadBalancedCall, grpc_core::OrphanableDelete>::~unique_ptr() /usr/lib/gcc/x86_64-linux-gnu/14/../../../../include/c++/14/bits/unique_ptr.h:398:4
    #7 0x7f5ce82084e7 in grpc_core::RetryFilter::LegacyCallData::CallAttempt::~CallAttempt() /proc/self/cwd/src/core/client_channel/retry_filter_legacy_call_data.cc:167:1
    #8 0x7f5ce8242e1a in void grpc_core::UnrefDelete::operator()<grpc_core::RetryFilter::LegacyCallData::CallAttempt const>(grpc_core::RetryFilter::LegacyCallData::CallAttempt const*) const /proc/self/cwd/./src/core/util/ref_counted.h:223:5
    #9 0x7f5ce8231655 in grpc_core::RefCounted<grpc_core::RetryFilter::LegacyCallData::CallAttempt, grpc_core::PolymorphicRefCount, grpc_core::UnrefDelete>::Unref(grpc_core::DebugLocation const&, char const*) const /proc/self/cwd/./src/core/util/ref_counted.h:371:7
    #10 0x7f5ce82197de in grpc_core::RetryFilter::LegacyCallData::CallAttempt::BatchData::~BatchData() /proc/self/cwd/src/core/client_channel/retry_filter_legacy_call_data.cc:727:17
    #11 0x7f5ce8242e88 in void grpc_core::UnrefCallDtor::operator()<grpc_core::RetryFilter::LegacyCallData::CallAttempt::BatchData const>(grpc_core::RetryFilter::LegacyCallData::CallAttempt::BatchData const*) const /proc/self/cwd/./src/core/util/ref_counted.h:243:9
    #12 0x7f5ce8242e65 in grpc_core::RefCounted<grpc_core::RetryFilter::LegacyCallData::CallAttempt::BatchData, grpc_core::PolymorphicRefCount, grpc_core::UnrefCallDtor>::Unref() const /proc/self/cwd/./src/core/util/ref_counted.h:366:7
    #13 0x7f5ce822f9e5 in grpc_core::RefCountedPtr<grpc_core::RetryFilter::LegacyCallData::CallAttempt::BatchData>::~RefCountedPtr() /proc/self/cwd/./src/core/util/ref_counted_ptr.h:111:36
    #14 0x7f5ce8225c59 in grpc_core::RetryFilter::LegacyCallData::CallAttempt::BatchData::OnCompleteForCancelOp(void*, absl::lts_20240722::Status) /proc/self/cwd/src/core/client_channel/retry_filter_legacy_call_data.cc:1312:1
    #15 0x7f5ce2c5d91d in exec_ctx_run(grpc_closure*) /proc/self/cwd/src/core/lib/iomgr/exec_ctx.cc:43:3
    #16 0x7f5ce2c5ca59 in grpc_core::ExecCtx::Flush() /proc/self/cwd/src/core/lib/iomgr/exec_ctx.cc:74:9
    #17 0x7f5cef337b25 in grpc_core::ExecCtx::~ExecCtx() /proc/self/cwd/./src/core/lib/iomgr/exec_ctx.h:137:5
    #18 0x7f5ce2c8cb3c in grpc_core::WorkSerializer::WorkSerializerImpl::Run() /proc/self/cwd/src/core/util/work_serializer.cc:221:1
    #19 0x7f5ce3e45da0 in grpc_event_engine::experimental::WorkStealingThreadPool::ThreadState::Step() /proc/self/cwd/src/core/lib/event_engine/thread_pool/work_stealing_thread_pool.cc:524:14
    #20 0x7f5ce3e44b59 in grpc_event_engine::experimental::WorkStealingThreadPool::ThreadState::ThreadBody() /proc/self/cwd/src/core/lib/event_engine/thread_pool/work_stealing_thread_pool.cc:487:10
    #21 0x7f5ce3e475e0 in grpc_event_engine::experimental::WorkStealingThreadPool::WorkStealingThreadPoolImpl::StartThread()::$_0::operator()(void*) const /proc/self/cwd/src/core/lib/event_engine/thread_pool/work_stealing_thread_pool.cc:257:17
    #22 0x7f5ce3e47558 in grpc_event_engine::experimental::WorkStealingThreadPool::WorkStealingThreadPoolImpl::StartThread()::$_0::__invoke(void*) /proc/self/cwd/src/core/lib/event_engine/thread_pool/work_stealing_thread_pool.cc:255:7
    #23 0x7f5ce21122be in grpc_core::(anonymous namespace)::ThreadInternalsPosix::ThreadInternalsPosix(char const*, void (*)(void*), void*, bool*, grpc_core::Thread::Options const&)::'lambda'(void*)::operator()(void*) const /proc/self/cwd/src/core/util/posix/thd.cc:145:11
    #24 0x7f5ce2111c27 in grpc_core::(anonymous namespace)::ThreadInternalsPosix::ThreadInternalsPosix(char const*, void (*)(void*), void*, bool*, grpc_core::Thread::Options const&)::'lambda'(void*)::__invoke(void*) /proc/self/cwd/src/core/util/posix/thd.cc:115:9
    #25 0x7f5ce0d646c1 in start_thread nptl/pthread_create.c:447:8

previously allocated by thread T282 here:
    #0 0x56484f6c7d72 in __interceptor_malloc (/usr/local/google/home/yashkt/.cache/bazel/_bazel_yashkt/3933dcff60587ac8c34d87542411a8b2/execroot/com_github_grpc_grpc/bazel-out/k8-dbg/bin/test/cpp/ext/otel/otel_tracing_test+0xc7d72) (BuildId: 5b1372e32b37211dd5de56fa1466f4c9e5c2df8f)
    #1 0x7f5ce0ab37eb in operator new(unsigned long) (/lib/x86_64-linux-gnu/libstdc++.so.6+0xb37eb) (BuildId: 69568e25e14544c953a9f79da5e158d0f644ad17)
    #2 0x7f5ce8151878 in grpc_core::(anonymous namespace)::CreateCallAttemptTracer(grpc_core::Arena*, bool) /proc/self/cwd/src/core/client_channel/client_channel_filter.cc:2386:31
    #3 0x7f5ce8150f43 in grpc_core::ClientChannelFilter::LoadBalancedCall::LoadBalancedCall(grpc_core::ClientChannelFilter*, grpc_core::Arena*, absl::lts_20240722::AnyInvocable<void ()>, bool) /proc/self/cwd/src/core/client_channel/client_channel_filter.cc:2401:3
    #4 0x7f5ce81563fa in grpc_core::ClientChannelFilter::FilterBasedLoadBalancedCall::FilterBasedLoadBalancedCall(grpc_core::ClientChannelFilter*, grpc_call_element_args const&, grpc_polling_entity*, grpc_closure*, absl::lts_20240722::AnyInvocable<void ()>, bool) /proc/self/cwd/src/core/client_channel/client_channel_filter.cc:2625:7
    #5 0x7f5ce816d875 in grpc_core::ClientChannelFilter::FilterBasedLoadBalancedCall* grpc_core::Arena::New<grpc_core::ClientChannelFilter::FilterBasedLoadBalancedCall, grpc_core::ClientChannelFilter*, grpc_call_element_args const&, grpc_polling_entity*&, grpc_closure*&, absl::lts_20240722::AnyInvocable<void ()>, bool&>(grpc_core::ClientChannelFilter*&&, grpc_call_element_args const&, grpc_polling_entity*&, grpc_closure*&, absl::lts_20240722::AnyInvocable<void ()>&&, bool&) /proc/self/cwd/./src/core/lib/resource_quota/arena.h:181:13
    #6 0x7f5ce8138d02 in grpc_core::ClientChannelFilter::CreateLoadBalancedCall(grpc_call_element_args const&, grpc_polling_entity*, grpc_closure*, absl::lts_20240722::AnyInvocable<void ()>, bool) /proc/self/cwd/src/core/client_channel/client_channel_filter.cc:1142:19
    #7 0x7f5ce8207e1a in grpc_core::RetryFilter::LegacyCallData::CreateLoadBalancedCall(absl::lts_20240722::AnyInvocable<void ()>, bool) /proc/self/cwd/src/core/client_channel/retry_filter_legacy_call_data.cc:1633:36
    #8 0x7f5ce8206661 in grpc_core::RetryFilter::LegacyCallData::CallAttempt::CallAttempt(grpc_core::RetryFilter::LegacyCallData*, bool) /proc/self/cwd/src/core/client_channel/retry_filter_legacy_call_data.cc:129:21
    #9 0x7f5ce82341d9 in grpc_core::RefCountedPtr<grpc_core::RetryFilter::LegacyCallData::CallAttempt> grpc_core::MakeRefCounted<grpc_core::RetryFilter::LegacyCallData::CallAttempt, grpc_core::RetryFilter::LegacyCallData*, bool&>(grpc_core::RetryFilter::LegacyCallData*&&, bool&) /proc/self/cwd/./src/core/util/ref_counted_ptr.h:369:31
    #10 0x7f5ce822c185 in grpc_core::RetryFilter::LegacyCallData::CreateCallAttempt(bool) /proc/self/cwd/src/core/client_channel/retry_filter_legacy_call_data.cc:1642:19
    #11 0x7f5ce822ca8e in grpc_core::RetryFilter::LegacyCallData::StartTransparentRetry(void*, absl::lts_20240722::Status) /proc/self/cwd/src/core/client_channel/retry_filter_legacy_call_data.cc:1935:12
    #12 0x7f5ce2c5d91d in exec_ctx_run(grpc_closure*) /proc/self/cwd/src/core/lib/iomgr/exec_ctx.cc:43:3
    #13 0x7f5ce2c5ca59 in grpc_core::ExecCtx::Flush() /proc/self/cwd/src/core/lib/iomgr/exec_ctx.cc:74:9
    #14 0x7f5cef337b25 in grpc_core::ExecCtx::~ExecCtx() /proc/self/cwd/./src/core/lib/iomgr/exec_ctx.h:137:5
    #15 0x7f5ce4595ee1 in grpc_event_engine::experimental::(anonymous namespace)::EventEngineEndpointWrapper::FinishPendingRead(absl::lts_20240722::Status) /proc/self/cwd/src/core/lib/iomgr/event_engine_shims/endpoint.cc:137:5
    #16 0x7f5ce4596df4 in grpc_event_engine::experimental::(anonymous namespace)::EventEngineEndpointWrapper::Read(grpc_closure*, grpc_slice_buffer*, grpc_event_engine::experimental::EventEngine::Endpoint::ReadArgs const*)::'lambda'(absl::lts_20240722::Status)::operator()(absl::lts_20240722::Status) const /proc/self/cwd/src/core/lib/iomgr/event_engine_shims/endpoint.cc:110:39
    #17 0x7f5ce4596bde in void std::__invoke_impl<void, grpc_event_engine::experimental::(anonymous namespace)::EventEngineEndpointWrapper::Read(grpc_closure*, grpc_slice_buffer*, grpc_event_engine::experimental::EventEngine::Endpoint::ReadArgs const*)::'lambda'(absl::lts_20240722::Status)&, absl::lts_20240722::Status>(std::__invoke_other, grpc_event_engine::experimental::(anonymous namespace)::EventEngineEndpointWrapper::Read(grpc_closure*, grpc_slice_buffer*, grpc_event_engine::experimental::EventEngine::Endpoint::ReadArgs const*)::'lambda'(absl::lts_20240722::Status)&, absl::lts_20240722::Status&&) /usr/lib/gcc/x86_64-linux-gnu/14/../../../../include/c++/14/bits/invoke.h:61:14
    #18 0x7f5ce4596adc in std::__invoke_result<grpc_event_engine::experimental::(anonymous namespace)::EventEngineEndpointWrapper::Read(grpc_closure*, grpc_slice_buffer*, grpc_event_engine::experimental::EventEngine::Endpoint::ReadArgs const*)::'lambda'(absl::lts_20240722::Status)&, absl::lts_20240722::Status>::type std::__invoke<grpc_event_engine::experimental::(anonymous namespace)::EventEngineEndpointWrapper::Read(grpc_closure*, grpc_slice_buffer*, grpc_event_engine::experimental::EventEngine::Endpoint::ReadArgs const*)::'lambda'(absl::lts_20240722::Status)&, absl::lts_20240722::Status>(grpc_event_engine::experimental::(anonymous namespace)::EventEngineEndpointWrapper::Read(grpc_closure*, grpc_slice_buffer*, grpc_event_engine::experimental::EventEngine::Endpoint::ReadArgs const*)::'lambda'(absl::lts_20240722::Status)&, absl::lts_20240722::Status&&) /usr/lib/gcc/x86_64-linux-gnu/14/../../../../include/c++/14/bits/invoke.h:96:14
    #19 0x7f5ce4596aac in std::invoke_result<grpc_event_engine::experimental::(anonymous namespace)::EventEngineEndpointWrapper::Read(grpc_closure*, grpc_slice_buffer*, grpc_event_engine::experimental::EventEngine::Endpoint::ReadArgs const*)::'lambda'(absl::lts_20240722::Status)&, absl::lts_20240722::Status>::type std::invoke<grpc_event_engine::experimental::(anonymous namespace)::EventEngineEndpointWrapper::Read(grpc_closure*, grpc_slice_buffer*, grpc_event_engine::experimental::EventEngine::Endpoint::ReadArgs const*)::'lambda'(absl::lts_20240722::Status)&, absl::lts_20240722::Status>(grpc_event_engine::experimental::(anonymous namespace)::EventEngineEndpointWrapper::Read(grpc_closure*, grpc_slice_buffer*, grpc_event_engine::experimental::EventEngine::Endpoint::ReadArgs const*)::'lambda'(absl::lts_20240722::Status)&, absl::lts_20240722::Status&&) /usr/lib/gcc/x86_64-linux-gnu/14/../../../../include/c++/14/functional:120:14
    #20 0x7f5ce4596a6c in void absl::lts_20240722::internal_any_invocable::InvokeR<void, grpc_event_engine::experimental::(anonymous namespace)::EventEngineEndpointWrapper::Read(grpc_closure*, grpc_slice_buffer*, grpc_event_engine::experimental::EventEngine::Endpoint::ReadArgs const*)::'lambda'(absl::lts_20240722::Status)&, absl::lts_20240722::Status, void>(grpc_event_engine::experimental::(anonymous namespace)::EventEngineEndpointWrapper::Read(grpc_closure*, grpc_slice_buffer*, grpc_event_engine::experimental::EventEngine::Endpoint::ReadArgs const*)::'lambda'(absl::lts_20240722::Status)&, absl::lts_20240722::Status&&) /proc/self/cwd/external/com_google_absl/absl/functional/internal/any_invocable.h:132:3
    #21 0x7f5ce45969d9 in void absl::lts_20240722::internal_any_invocable::LocalInvoker<false, void, grpc_event_engine::experimental::(anonymous namespace)::EventEngineEndpointWrapper::Read(grpc_closure*, grpc_slice_buffer*, grpc_event_engine::experimental::EventEngine::Endpoint::ReadArgs const*)::'lambda'(absl::lts_20240722::Status)&, absl::lts_20240722::Status>(absl::lts_20240722::internal_any_invocable::TypeErasedState*, absl::lts_20240722::internal_any_invocable::ForwardedParameter<absl::lts_20240722::Status>::type) /proc/self/cwd/external/com_google_absl/absl/functional/internal/any_invocable.h:310:10
    #22 0x7f5cf0b08b53 in absl::lts_20240722::internal_any_invocable::Impl<void (absl::lts_20240722::Status)>::operator()(absl::lts_20240722::Status) /proc/self/cwd/external/com_google_absl/absl/functional/internal/any_invocable.h:868:1
    #23 0x7f5ce40730de in grpc_event_engine::experimental::PosixEndpointImpl::HandleRead(absl::lts_20240722::Status) /proc/self/cwd/src/core/lib/event_engine/posix_engine/posix_endpoint.cc:592:3
    #24 0x7f5ce408a104 in grpc_event_engine::experimental::PosixEndpointImpl::PosixEndpointImpl(grpc_event_engine::experimental::EventHandle*, grpc_event_engine::experimental::PosixEngineClosure*, std::shared_ptr<grpc_event_engine::experimental::EventEngine>, grpc_event_engine::experimental::MemoryAllocator&&, grpc_event_engine::experimental::PosixTcpOptions const&)::$_0::operator()(absl::lts_20240722::Status) const /proc/self/cwd/src/core/lib/event_engine/posix_engine/posix_endpoint.cc:1341:37
    #25 0x7f5ce4089eee in void std::__invoke_impl<void, grpc_event_engine::experimental::PosixEndpointImpl::PosixEndpointImpl(grpc_event_engine::experimental::EventHandle*, grpc_event_engine::experimental::PosixEngineClosure*, std::shared_ptr<grpc_event_engine::experimental::EventEngine>, grpc_event_engine::experimental::MemoryAllocator&&, grpc_event_engine::experimental::PosixTcpOptions const&)::$_0&, absl::lts_20240722::Status>(std::__invoke_other, grpc_event_engine::experimental::PosixEndpointImpl::PosixEndpointImpl(grpc_event_engine::experimental::EventHandle*, grpc_event_engine::experimental::PosixEngineClosure*, std::shared_ptr<grpc_event_engine::experimental::EventEngine>, grpc_event_engine::experimental::MemoryAllocator&&, grpc_event_engine::experimental::PosixTcpOptions const&)::$_0&, absl::lts_20240722::Status&&) /usr/lib/gcc/x86_64-linux-gnu/14/../../../../include/c++/14/bits/invoke.h:61:14
    #26 0x7f5ce4089dec in std::__invoke_result<grpc_event_engine::experimental::PosixEndpointImpl::PosixEndpointImpl(grpc_event_engine::experimental::EventHandle*, grpc_event_engine::experimental::PosixEngineClosure*, std::shared_ptr<grpc_event_engine::experimental::EventEngine>, grpc_event_engine::experimental::MemoryAllocator&&, grpc_event_engine::experimental::PosixTcpOptions const&)::$_0&, absl::lts_20240722::Status>::type std::__invoke<grpc_event_engine::experimental::PosixEndpointImpl::PosixEndpointImpl(grpc_event_engine::experimental::EventHandle*, grpc_event_engine::experimental::PosixEngineClosure*, std::shared_ptr<grpc_event_engine::experimental::EventEngine>, grpc_event_engine::experimental::MemoryAllocator&&, grpc_event_engine::experimental::PosixTcpOptions const&)::$_0&, absl::lts_20240722::Status>(grpc_event_engine::experimental::PosixEndpointImpl::PosixEndpointImpl(grpc_event_engine::experimental::EventHandle*, grpc_event_engine::experimental::PosixEngineClosure*, std::shared_ptr<grpc_event_engine::experimental::EventEngine>, grpc_event_engine::experimental::MemoryAllocator&&, grpc_event_engine::experimental::PosixTcpOptions const&)::$_0&, absl::lts_20240722::Status&&) /usr/lib/gcc/x86_64-linux-gnu/14/../../../../include/c++/14/bits/invoke.h:96:14
    #27 0x7f5ce4089dbc in std::invoke_result<grpc_event_engine::experimental::PosixEndpointImpl::PosixEndpointImpl(grpc_event_engine::experimental::EventHandle*, grpc_event_engine::experimental::PosixEngineClosure*, std::shared_ptr<grpc_event_engine::experimental::EventEngine>, grpc_event_engine::experimental::MemoryAllocator&&, grpc_event_engine::experimental::PosixTcpOptions const&)::$_0&, absl::lts_20240722::Status>::type std::invoke<grpc_event_engine::experimental::PosixEndpointImpl::PosixEndpointImpl(grpc_event_engine::experimental::EventHandle*, grpc_event_engine::experimental::PosixEngineClosure*, std::shared_ptr<grpc_event_engine::experimental::EventEngine>, grpc_event_engine::experimental::MemoryAllocator&&, grpc_event_engine::experimental::PosixTcpOptions const&)::$_0&, absl::lts_20240722::Status>(grpc_event_engine::experimental::PosixEndpointImpl::PosixEndpointImpl(grpc_event_engine::experimental::EventHandle*, grpc_event_engine::experimental::PosixEngineClosure*, std::shared_ptr<grpc_event_engine::experimental::EventEngine>, grpc_event_engine::experimental::MemoryAllocator&&, grpc_event_engine::experimental::PosixTcpOptions const&)::$_0&, absl::lts_20240722::Status&&) /usr/lib/gcc/x86_64-linux-gnu/14/../../../../include/c++/14/functional:120:14
    #28 0x7f5ce4089d7c in void absl::lts_20240722::internal_any_invocable::InvokeR<void, grpc_event_engine::experimental::PosixEndpointImpl::PosixEndpointImpl(grpc_event_engine::experimental::EventHandle*, grpc_event_engine::experimental::PosixEngineClosure*, std::shared_ptr<grpc_event_engine::experimental::EventEngine>, grpc_event_engine::experimental::MemoryAllocator&&, grpc_event_engine::experimental::PosixTcpOptions const&)::$_0&, absl::lts_20240722::Status, void>(grpc_event_engine::experimental::PosixEndpointImpl::PosixEndpointImpl(grpc_event_engine::experimental::EventHandle*, grpc_event_engine::experimental::PosixEngineClosure*, std::shared_ptr<grpc_event_engine::experimental::EventEngine>, grpc_event_engine::experimental::MemoryAllocator&&, grpc_event_engine::experimental::PosixTcpOptions const&)::$_0&, absl::lts_20240722::Status&&) /proc/self/cwd/external/com_google_absl/absl/functional/internal/any_invocable.h:132:3
    #29 0x7f5ce4089ce9 in void absl::lts_20240722::internal_any_invocable::LocalInvoker<false, void, grpc_event_engine::experimental::PosixEndpointImpl::PosixEndpointImpl(grpc_event_engine::experimental::EventHandle*, grpc_event_engine::experimental::PosixEngineClosure*, std::shared_ptr<grpc_event_engine::experimental::EventEngine>, grpc_event_engine::experimental::MemoryAllocator&&, grpc_event_engine::experimental::PosixTcpOptions const&)::$_0&, absl::lts_20240722::Status>(absl::lts_20240722::internal_any_invocable::TypeErasedState*, absl::lts_20240722::internal_any_invocable::ForwardedParameter<absl::lts_20240722::Status>::type) /proc/self/cwd/external/com_google_absl/absl/functional/internal/any_invocable.h:310:10

The sequence of events is -

  1. The client application creates a new RPC.
  2. A ClientChannelFilter::LoadBalancedCall is created for the first attempt.
  3. A CallAttemptTracer is created for this first attempt and a pointer to this is saved on the arena context.
  4. The first attempt fails without any bytes having been sent on the wire, making this RPC eligible for transparent retries.
  5. A new ClientChannelFilter::LoadBalancedCall created for this new attempt.
  6. A new CallAttemptTracer is created for this second (transparent) attempt and a pointer is saved on the arena context.
  7. The previous call attempt starts cleaning up, and attempts to end the call attempt tracer. Unfortunately, this is done by reading the pointer from the arena context, resulting in the deletion of the call attempt tracer of the second attempt, and eventually resulting in a heap-use-after-free when the new call attempt tracer is accessed.

Metadata

Metadata

Assignees

Labels

kind/buglang/corepriority/P2

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions