LDS clobbers statically configured listeners (and crashes) #2234
Description
Given a V2 bootstrap configuration with a Listener defined in static_resources and one or more listeners configured via dynamic_resources, the former is removed as soon as a successful DynamicResponse for LDS is received. Envoy then asserts, perhaps because the removed listener was never started.
The correct behavior should be to leave the statically defined Listener in place a la CDS/Cluster.
Repro steps:
See attached tar ball for config file. It has a healthcheck listener configured in bootstrap.conf, and a second listener in lds.conf. The log file shows the lds listener being added, the healthcheck listener being removed, and then the subsequent crash with decoded stack trace.
Config:
See attached file.
Logs:
See attached file.
Call Stack:
[2017-12-19 13:27:16.058][23811][debug][config] source/server/listener_manager_impl.cc:454] begin remove listener: name=healthcheck
[2017-12-19 13:27:16.058][23811][debug][config] source/server/listener_manager_impl.cc:162] draining listener: name=healthcheck, hash=859362694391224026, address=0.0.0.0:8888
[2017-12-19 13:27:16.058][23811][critical][assert] source/server/worker_impl.cc:96] assert failure: thread_
[2017-12-19 13:27:16.058][23811][critical][backtrace] bazel-out/k8-fastbuild/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:101] Caught Aborted, suspect faulting address 0x3e800005d03
[2017-12-19 13:27:16.058][23811][critical] Backtrace (most recent call first) from thread 0:
#1 ?? ??:0
#2 ?? ??:0
#3
#4 Envoy::Server::WorkerImpl::stopListener(Envoy::Server::Listener&) at ??:?
#5 Envoy::Server::ListenerManagerImpl::drainListener(std::unique_ptr<Envoy::Server::ListenerImpl, std::default_delete<Envoy::Server::ListenerImpl> >&&) at ??:?
#6 Envoy::Server::ListenerManagerImpl::removeListener(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) at ??:?
#7 Envoy::Server::LdsApi::onConfigUpdate(google::protobuf::RepeatedPtrField<envoy::api::v2::Listener> const&) at ??:?
#8 Envoy::Config::FilesystemSubscriptionImpl<envoy::api::v2::Listener>::refresh() at ??:?
#9 Envoy::Config::FilesystemSubscriptionImpl<envoy::api::v2::Listener>::start(std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&, Envoy::Config::SubscriptionCallbacks<envoy::api::v2::Listener>&) at ??:?
#10 Envoy::Server::LdsApi::initialize(std::function<void ()>) at ??:?
#11 Envoy::Server::InitManagerImpl::initializeTarget(Envoy::Init::Target&) at ??:?
#12 Envoy::Server::InitManagerImpl::initialize(std::function<void ()>) at ??:?
#13 Envoy::Server::RunHelper::RunHelper(Envoy::Event::Dispatcher&, Envoy::Upstream::ClusterManager&, Envoy::Server::HotRestart&, Envoy::AccessLog::AccessLogManager&, Envoy::Server::InitManagerImpl&, std::function<void ()>)::{lambda()#4}::operator()() const at server.cc:?
#14 std::_Function_handler<void (), Envoy::Server::RunHelper::RunHelper(Envoy::Event::Dispatcher&, Envoy::Upstream::ClusterManager&, Envoy::Server::HotRestart&, Envoy::AccessLog::AccessLogManager&, Envoy::Server::InitManagerImpl&, std::function<void ()>)::{lambda()#4}>::_M_invoke(std::_Any_data const&) at server.cc:?
#15 std::function<void ()>::operator()() const at ??:?
#16 Envoy::Upstream::ClusterManagerInitHelper::setInitializedCb(std::function<void ()>) at ??:?
#17 Envoy::Upstream::ClusterManagerImpl::setInitializedCb(std::function<void ()>) at ??:?
#18 Envoy::Server::RunHelper::RunHelper(Envoy::Event::Dispatcher&, Envoy::Upstream::ClusterManager&, Envoy::Server::HotRestart&, Envoy::AccessLog::AccessLogManager&, Envoy::Server::InitManagerImpl&, std::function<void ()>) at ??:?
#19 Envoy::Server::InstanceImpl::run() at ??:?
#20 Envoy::main_common(Envoy::OptionsImpl&) at ??:?
#21 main at ??:?
#22
#23 ?? ??:0
#24
#25 _start at ??:?
#26