Segmentation faults in Lua filter (LuaThreadLocal) #10241
Description
Title: Segmentation faults in Lua filter (LuaThreadLocal)
Description:
Envoy occasionally crashes, which is unexpected.
Repro steps:
Unsure how to reproduce at the moment, it seems spurious (<5 times per day in a QA fleet since collecting core dumps).
It has been observed in:
1.12.2+ hot restart: send used stats only during hot restart #9121 + http: Fix ASSERT failure and infinite loop when attempting to unset readDisable state on a closed connection. #9509 + patches from1.12.31.13.1
We haven't collected core dumps from other versions.
Config:
We have various Lua filters configured.
Call Stack:
From the 1.12.2 build described above:
#0 Envoy::ThreadLocal::Slot::getTyped<Envoy::Extensions::Filters::Common::Lua::ThreadLocalState::LuaThreadLocal> (this=0xe) at bazel-out/k8-opt/bin/external/envoy/include/envoy/thread_local/_virtual_includes/thread_local_interface/envoy/thread_local/thread_local.h:51
#1 0x000055e16e7980ec in Envoy::Extensions::Filters::Common::Lua::ThreadLocalState::registerType<Envoy::Extensions::Filters::Common::Lua::BufferWrapper>()::{lambda()#1}::operator()() const (__closure=<optimized out>) at bazel-out/k8-opt/bin/external/envoy/source/extensions/filters/common/lua/_virtual_includes/lua_lib/extensions/filters/common/lua/lua.h:390
#2 std::_Function_handler<void (), Envoy::Extensions::Filters::Common::Lua::ThreadLocalState::registerType<Envoy::Extensions::Filters::Common::Lua::BufferWrapper>()::{lambda()#1}>::_M_invoke(std::_Any_data const&) (__functor=...) at /usr/include/c++/7/bits/std_function.h:316
#3 0x000055e16ec9b053 in std::function<void ()>::operator()() const (this=0x7f7fb763b850) at /usr/include/c++/7/bits/std_function.h:706
#4 Envoy::Event::DispatcherImpl::runPostCallbacks (this=this@entry=0x55e17203cdc0) at external/envoy/source/common/event/dispatcher_impl.cc:222
#5 0x000055e16ec9b14c in Envoy::Event::DispatcherImpl::run (this=0x55e17203cdc0, type=Envoy::Event::Dispatcher::RunType::Block) at external/envoy/source/common/event/dispatcher_impl.cc:193
#6 0x000055e16ec951fa in Envoy::Server::WorkerImpl::threadRoutine (this=0x55e172222850, guard_dog=...) at external/envoy/source/server/worker_impl.cc:110
#7 0x000055e16eff01d5 in std::function<void ()>::operator()() const (this=<optimized out>) at /usr/include/c++/7/bits/std_function.h:706
#8 Envoy::Thread::ThreadImplPosix::<lambda(void*)>::operator() (__closure=0x0, arg=<optimized out>) at external/envoy/source/common/common/posix/thread_impl.cc:33
#9 Envoy::Thread::ThreadImplPosix::<lambda(void*)>::_FUN(void *) () at external/envoy/source/common/common/posix/thread_impl.cc:35
#10 0x00007f7fbba2a6ba in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0
#11 0x00007f7fbb76041d in clone () from /lib/x86_64-linux-gnu/libc.so.6