Lua filter causes a crash when the script uses streamInfo():downstreamSslConnection() #14091
Description
Envoy crashes with segmentation fault, when Lua script contains streamInfo():downstreamSslConnection().
The crash happens after issuing many https requests.
The bug exists in v1.16.0 as well as in the latest master version.
I have a permission from envoy-security@googlegroups.com for fixing this publicly.
Repro steps:
Run envoy with attached config and make 1000 https requests.
Config:
Click to expand!
static_resources:
listeners:
- name: listener_0
address:
socket_address:
address: 0.0.0.0
port_value: 10000
filter_chains:
- filters:
- name: envoy.filters.network.http_connection_manager
typed_config:
"@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
stat_prefix: ingress_http
http_filters:
- name: envoy.filters.http.lua
typed_config:
"@type": type.googleapis.com/envoy.extensions.filters.http.lua.v3.Lua
inline_code: |
function envoy_on_request(handle)
if handle:connection():ssl() ~= nil then
if handle:streamInfo() ~= nil then
if handle:streamInfo():downstreamSslConnection() ~= nil then
handle:logInfo("lua - envoy_on_request - SSL; streamInfo == present; downstreamSslConnection == present")
else
handle:logInfo("lua - envoy_on_request - SSL; streamInfo == present; downstreamSslConnection == nil")
end
else
handle:logInfo("lua - envoy_on_request - SSL; streamInfo == nil")
end
else
handle:logInfo("lua - envoy_on_request - RAW BUFFER")
end
end
- name: envoy.filters.http.router
route_config:
name: local_route
virtual_hosts:
- name: local_service
domains: ["*"]
routes:
- match:
prefix: "/"
direct_response:
status: 200
body:
inline_string: "Envoy direct response"
transport_socket:
name: envoy.transport_sockets.tls
typed_config:
"@type": type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.DownstreamTlsContext
common_tls_context:
tls_certificates:
certificate_chain:
filename: ./certs/rsa-2048-signed-rsa-server/cert.crt
private_key:
filename: ./certs/rsa-2048-signed-rsa-server/key.prv
Call Stack:
Click to expand!
Thread 11 "wrk:worker_2" received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7ffff280f700 (LWP 71)]
0x00005555567d8ef0 in lua_rawgeti ()
#0 0x00005555567d8ef0 in lua_rawgeti ()
#1 0x000055555682905c in luaL_unref ()
#2 0x00005555567b8790 in Envoy::Extensions::Filters::Common::Lua::LuaRef<Envoy::Extensions::Filters::Common::Lua::SslConnectionWrapper>::unref (
this=<optimized out>)
at bazel-out/k8-opt/bin/source/extensions/filters/common/lua/_virtual_includes/lua_lib/extensions/filters/common/lua/lua.h:277
#3 Envoy::Extensions::Filters::Common::Lua::LuaRef<Envoy::Extensions::Filters::Common::Lua::SslConnectionWrapper>::~LuaRef (this=<optimized out>)
at bazel-out/k8-opt/bin/source/extensions/filters/common/lua/_virtual_includes/lua_lib/extensions/filters/common/lua/lua.h:239
#4 Envoy::Extensions::Filters::Common::Lua::LuaDeathRef<Envoy::Extensions::Filters::Common::Lua::SslConnectionWrapper>::~LuaDeathRef (this=<optimized out>)
at bazel-out/k8-opt/bin/source/extensions/filters/common/lua/_virtual_includes/lua_lib/extensions/filters/common/lua/lua.h:294
#5 Envoy::Extensions::HttpFilters::Lua::StreamInfoWrapper::~StreamInfoWrapper
(this=0x7ffff7e23c38)
at bazel-out/k8-opt/bin/source/extensions/filters/http/lua/_virtual_includes/wrappers_lib/extensions/filters/http/lua/wrappers.h:181
#6 0x00005555567b4b27 in Envoy::Extensions::Filters::Common::Lua::BaseLuaObject<Envoy::Extensions::HttpFilters::Lua::StreamInfoWrapper>::registerType(lua_State*)::{lambda(lua_State*)#1}::operator()(lua_State*) const (
this=<optimized out>, state=<optimized out>)
at bazel-out/k8-opt/bin/source/extensions/filters/common/lua/_virtual_includes/lua_lib/extensions/filters/common/lua/lua.h:146
#7 0x00005555567c3b7a in lj_BC_FUNCC ()
#8 0x00005555567c62ec in gc_finalize ()
#9 0x00005555567c6a58 in gc_onestep ()
#10 0x00005555567c6628 in lj_gc_step ()
#11 0x00005555567d8929 in lua_newuserdata ()
#12 0x00005555567abd90 in Envoy::Extensions::Filters::Common::Lua::allocateLuaUserData<Envoy::Extensions::HttpFilters::Lua::StreamHandleWrapper> (state=
0x7ffff7e23ce8)
at bazel-out/k8-opt/bin/source/extensions/filters/common/lua/_virtual_includes/lua_lib/extensions/filters/common/lua/lua.h:82
#13 Envoy::Extensions::Filters::Common::Lua::BaseLuaObject<Envoy::Extensions::HttpFilters::Lua::StreamHandleWrapper>::create<Envoy::Extensions::Filters::Common::Lua::Coroutine&, Envoy::Http::HeaderMap&, bool&, Envoy::Extensions::HttpFilters::Lua::Filter&, Envoy::Extensions::HttpFilters::Lua::FilterCallbacks&> (
state=0x7ffff7e23ce8, args=..., args=..., args=..., args=..., args=...)
at bazel-out/k8-opt/bin/source/extensions/filters/common/lua/_virtual_includes/lua_lib/extensions/filters/common/lua/lua.h:121
#14 0x00005555567abc44 in Envoy::Extensions::HttpFilters::Lua::Filter::doHeaders (this=0x2d78bf1a3778, handle=..., coroutine=..., callbacks=...,
function_ref=1, setup=<optimized out>, headers=..., end_stream=true)
at source/extensions/filters/http/lua/lua_filter.cc:684
#15 0x00005555567ad6c5 in Envoy::Extensions::HttpFilters::Lua::Filter::decodeHeaders (this=0x2d78bf1a3778, headers=..., end_stream=true)
at bazel-out/k8-opt/bin/source/extensions/filters/http/lua/_virtual_includes/lua_filter_lib/extensions/filters/http/lua/lua_filter.h:442
#16 0x000055555749a02f in Envoy::Http::ActiveStreamDecoderFilter::decodeHeaders (this=0x2d78bfa12900, headers=..., end_stream=true)
at bazel-out/k8-opt/bin/source/common/http/_virtual_includes/filter_manager_lib/common/http/filter_manager.h:199
#17 Envoy::Http::FilterManager::decodeHeaders (this=<optimized out>,
filter=<optimized out>, headers=..., end_stream=<optimized out>)
at source/common/http/filter_manager.cc:452
#18 0x000055555748dc93 in Envoy::Http::FilterManager::decodeHeaders (
this=0x2d78bf217278, headers=..., end_stream=<optimized out>)
at bazel-out/k8-opt/bin/source/common/http/_virtual_includes/filter_manager_lib/common/http/filter_manager.h:582
#19 Envoy::Http::ConnectionManagerImpl::ActiveStream::decodeHeaders (
this=0x2d78bf217200, headers=..., end_stream=<optimized out>)
at source/common/http/conn_manager_impl.cc:1024
#20 0x00005555574ac6f6 in Envoy::Http::Legacy::Http1::ServerConnectionImpl::onMessageComplete (this=0x2d78bf15e680)
at source/common/http/http1/codec_impl_legacy.cc:970
#21 0x00005555574aa9e5 in Envoy::Http::Legacy::Http1::ConnectionImpl::onMessageCompleteBase (this=0x2d78bf15e690)
at source/common/http/http1/codec_impl_legacy.cc:764
#22 0x00005555574a78dd in Envoy::Http::Legacy::Http1::ConnectionImpl::$_8::operator() (this=<optimized out>, parser=<optimized out>)
at source/common/http/http1/codec_impl_legacy.cc:434
#23 Envoy::Http::Legacy::Http1::ConnectionImpl::$_8::__invoke (
parser=<optimized out>)
at source/common/http/http1/codec_impl_legacy.cc:433
#24 0x0000555557631beb in http_parser_execute (parser=0x2d78bf15e6d8,
settings=<optimized out>, data=<optimized out>, len=<optimized out>)
at external/com_github_nodejs_http_parser/http_parser.c:2167
#25 0x00005555574a91fa in Envoy::Http::Legacy::Http1::ConnectionImpl::dispatchSlice (this=0x2d78bf15e690, slice=0x0, len=140737261838648)
at source/common/http/http1/codec_impl_legacy.cc:572
#26 0x00005555574a8d9f in Envoy::Http::Legacy::Http1::ConnectionImpl::innerDispatch (this=0x2d78bf15e690, data=...)
at source/common/http/http1/codec_impl_legacy.cc:548
#27 0x00005555574b1552 in Envoy::Http::Legacy::Http1::ConnectionImpl::dispatch(Envoy::Buffer::Instance&)::$_14::operator()(Envoy::Buffer::Instance&) const (
this=<optimized out>, data=...)
at source/common/http/http1/codec_impl_legacy.cc:531
#28 _ZNSt3__18__invokeIRZN5Envoy4Http6Legacy5Http114ConnectionImpl8dispatchERNS1_6Buffer8InstanceEE4$_14JS8_EEEDTclclsr3std3__1E7forwardIT_Efp_Espclsr3std3__1E7forwardIT0_Efp0_EEEOSB_DpOSC_ (__f=..., __args=...)
at /opt/llvm/bin/../include/c++/v1/type_traits:3539
#29 std::__1::__invoke_void_return_wrapper<absl::Status>::__call<Envoy::Http::Legacy::Http1::ConnectionImpl::dispatch(Envoy::Buffer::Instance&)::$_14(Envoy::Buffer::Instance&)&>(Envoy::Http::Legacy::Http1::ConnectionImpl::dispatch(Envoy::Buffer::Instance&)::$_14(Envoy::Buffer::Instance&)&) (__args=..., __args=...)
at /opt/llvm/bin/../include/c++/v1/__functional_base:317
#30 std::__1::__function::__alloc_func<Envoy::Http::Legacy::Http1::ConnectionImpl::dispatch(Envoy::Buffer::Instance&)::$_14(std::__1::allocator<std::__1::allocator>, absl::Status (Envoy::Buffer::Instance&))>::operator()(Envoy::Buffer::Instance&) (this=<optimized out>, __arg=...)
at /opt/llvm/bin/../include/c++/v1/functional:1540
#31 std::__1::__function::__func<Envoy::Http::Legacy::Http1::ConnectionImpl::dispatch(Envoy::Buffer::Instance&)::$_14(std::__1::allocator<std::__1::allocator>, absl::Status (Envoy::Buffer::Instance&))>::operator()(Envoy::Buffer::Instance&) (this=<optimized out>, __arg=...)
at /opt/llvm/bin/../include/c++/v1/functional:1714
#32 0x0000555557603847 in std::__1::__function::__value_func<absl::Status (Envoy::Buffer::Instance&)>::operator()(Envoy::Buffer::Instance&) const (
this=<optimized out>, __args=...)
at /opt/llvm/bin/../include/c++/v1/functional:1867
#33 std::__1::function<absl::Status (Envoy::Buffer::Instance&)>::operator()(Envoy::Buffer::Instance&) const (this=<optimized out>, __arg=...)
at /opt/llvm/bin/../include/c++/v1/functional:2473
#34 Envoy::Http::Utility::exceptionToStatus(std::__1::function<absl::Status (Envoy::Buffer::Instance&)>, Envoy::Buffer::Instance&) (dispatch=..., data=...)
at source/common/http/utility.cc:43
#35 0x00005555574a8a51 in virtual thunk to Envoy::Http::Legacy::Http1::ConnectionImpl::dispatch(Envoy::Buffer::Instance&) ()
at source/common/http/http1/codec_impl_legacy.cc:530
#36 0x0000555557489e2c in Envoy::Http::ConnectionManagerImpl::onData (
this=0x2d78bfb483c0, data=...)
at source/common/http/conn_manager_impl.cc:282
#37 0x0000555557202873 in Envoy::Network::FilterManagerImpl::onContinueReading
(this=0x2d78bf163bf8, filter=<optimized out>, buffer_source=...)
at source/common/network/filter_manager_impl.cc:66
#38 0x00005555571fe845 in Envoy::Network::ConnectionImpl::onRead (
this=0x2d78bf163b80, read_buffer_size=256)
at source/common/network/connection_impl.cc:292
#39 Envoy::Network::ConnectionImpl::onReadReady (this=0x2d78bf163b80)
at source/common/network/connection_impl.cc:574
#40 0x00005555571fc815 in Envoy::Network::ConnectionImpl::onFileEvent (
this=0x2d78bf163b80, events=<optimized out>)
at source/common/network/connection_impl.cc:534
#41 0x00005555571f2366 in std::__1::__function::__value_func<void (unsigned int)>::operator()(unsigned int&&) const (this=0x2d78bf127330,
__args=@0x7ffff27fb0cc: 3)
at /opt/llvm/bin/../include/c++/v1/functional:1867
#42 std::__1::function<void (unsigned int)>::operator()(unsigned int) const (
this=0x2d78bf127330, __arg=3)
at /opt/llvm/bin/../include/c++/v1/functional:2473
#43 Envoy::Event::FileEventImpl::mergeInjectedEventsAndRunCb (
this=0x2d78bf1272b0, events=3)
at source/common/event/file_event_impl.cc:133
#44 Envoy::Event::FileEventImpl::assignEvents(unsigned int, event_base*)::$_1::operator()(int, short, void*) const (this=<optimized out>,
what=<optimized out>, arg=0x2d78bf1272b0)
at source/common/event/file_event_impl.cc:106
#45 Envoy::Event::FileEventImpl::assignEvents(unsigned int, event_base*)::$_1::__invoke(int, short, void*) (what=<optimized out>, arg=0x2d78bf1272b0)
at source/common/event/file_event_impl.cc:90
#46 0x000055555762afe8 in event_process_active_single_queue ()
#47 0x00005555576299be in event_base_loop ()
#48 0x00005555571e3838 in Envoy::Server::WorkerImpl::threadRoutine (
this=0x2d78bfbcd1d0, guard_dog=...) at source/server/worker_impl.cc:132
#49 0x00005555577d88c3 in std::__1::__function::__value_func<void ()>::operator()() const (this=<optimized out>)
at /opt/llvm/bin/../include/c++/v1/functional:1867
#50 std::__1::function<void ()>::operator()() const (this=<optimized out>)
at /opt/llvm/bin/../include/c++/v1/functional:2473
#51 Envoy::Thread::ThreadImplPosix::ThreadImplPosix(std::__1::function<void ()>, std::__1::optional<Envoy::Thread::Options> const&)::{lambda(void*)#1}::operator()(void*) const (this=<optimized out>, arg=<optimized out>)
at source/common/common/posix/thread_impl.cc:49
#52 Envoy::Thread::ThreadImplPosix::ThreadImplPosix(std::__1::function<void ()>, std::__1::optional<Envoy::Thread::Options> const&)::{lambda(void*)#1}::__invoke(void*) (arg=<optimized out>)
at source/common/common/posix/thread_impl.cc:48
#53 0x00007ffff74116db in start_thread (arg=0x7ffff280f700)
at pthread_create.c:463
#54 0x00007ffff713a71f in clone ()
at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
A debugging session is active.