Skip to content

debug assert when appending request headers #3919

@snowp

Description

@snowp
[2018-07-20 20:46:32.953][16][critical][assert] source/common/http/header_map_impl.cc:326] assert failure: *ref_lookup_response.entry_ == nullptr.
[2018-07-20 20:46:33.897][16][critical][backtrace] bazel-out/k8-dbg/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:125] Caught Segmentation fault, suspect faulting address 0x0
[2018-07-20 20:46:33.898][16][critical][backtrace] bazel-out/k8-dbg/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:94] Backtrace thr<16> obj</lib64/libc.so.6> (If unsymbolized, use tools/stack_decode.py):
[2018-07-20 20:46:33.899][16][critical][backtrace] bazel-out/k8-dbg/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:114] thr<16> #0 0x7fe2c24b6ab7 abort
[2018-07-20 20:46:33.899][16][critical][backtrace] bazel-out/k8-dbg/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:104] thr<16> obj</usr/local/bin/envoy>
[2018-07-20 20:46:33.899][16][critical][backtrace] bazel-out/k8-dbg/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:114] thr<16> #1 0xfa6fed Envoy::Http::HeaderMapImpl::insertByKey()
[2018-07-20 20:46:33.899][16][critical][backtrace] bazel-out/k8-dbg/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:114] thr<16> #2 0xfa757d Envoy::Http::HeaderMapImpl::addReferenceKey()
[2018-07-20 20:46:33.899][16][critical][backtrace] bazel-out/k8-dbg/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:114] thr<16> #3 0xcd9bd5 Envoy::Router::HeaderParser::evaluateHeaders()
[2018-07-20 20:46:33.899][16][critical][backtrace] bazel-out/k8-dbg/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:114] thr<16> #4 0xcba09a Envoy::Router::RouteEntryImplBase::finalizeRequestHeaders()
[2018-07-20 20:46:33.899][16][critical][backtrace] bazel-out/k8-dbg/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:114] thr<16> #5 0xc140f3 Envoy::Router::Filter::decodeHeaders()
[2018-07-20 20:46:33.899][16][critical][backtrace] bazel-out/k8-dbg/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:114] thr<16> #6 0xc705e0 Envoy::Http::ConnectionManagerImpl::ActiveStreamDecoderFilter::decodeHeaders()
[2018-07-20 20:46:33.899][16][critical][backtrace] bazel-out/k8-dbg/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:114] thr<16> #7 0xc67527 Envoy::Http::ConnectionManagerImpl::ActiveStream::decodeHeaders()
[2018-07-20 20:46:33.899][16][critical][backtrace] bazel-out/k8-dbg/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:114] thr<16> #8 0xc66c7c Envoy::Http::ConnectionManagerImpl::ActiveStream::decodeHeaders()
[2018-07-20 20:46:33.899][16][critical][backtrace] bazel-out/k8-dbg/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:114] thr<16> #9 0xcf8de8 Envoy::Http::Http1::ServerConnectionImpl::onMessageComplete()
[2018-07-20 20:46:33.899][16][critical][backtrace] bazel-out/k8-dbg/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:114] thr<16> #10 0xcf77ec Envoy::Http::Http1::ConnectionImpl::onMessageCompleteBase()
[2018-07-20 20:46:33.899][16][critical][backtrace] bazel-out/k8-dbg/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:114] thr<16> #11 0xcf63c5 Envoy::Http::Http1::ConnectionImpl::{lambda()#7}::operator()()
[2018-07-20 20:46:33.899][16][critical][backtrace] bazel-out/k8-dbg/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:114] thr<16> #12 0xcf63ea Envoy::Http::Http1::ConnectionImpl::{lambda()#7}::_FUN()
[2018-07-20 20:46:33.899][16][critical][backtrace] bazel-out/k8-dbg/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:114] thr<16> #13 0xd033a6 http_parser_execute
[2018-07-20 20:46:33.899][16][critical][backtrace] bazel-out/k8-dbg/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:114] thr<16> #14 0xcf72c5 Envoy::Http::Http1::ConnectionImpl::dispatchSlice()
[2018-07-20 20:46:33.899][16][critical][backtrace] bazel-out/k8-dbg/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:114] thr<16> #15 0xcf7199 Envoy::Http::Http1::ConnectionImpl::dispatch()
[2018-07-20 20:46:33.899][16][critical][backtrace] bazel-out/k8-dbg/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:114] thr<16> #16 0xc63fdc Envoy::Http::ConnectionManagerImpl::onData()
[2018-07-20 20:46:33.899][16][critical][backtrace] bazel-out/k8-dbg/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:114] thr<16> #17 0x968ea0 Envoy::Network::FilterManagerImpl::onContinueReading()
[2018-07-20 20:46:33.899][16][critical][backtrace] bazel-out/k8-dbg/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:114] thr<16> #18 0x968fa5 Envoy::Network::FilterManagerImpl::onRead()
[2018-07-20 20:46:33.899][16][critical][backtrace] bazel-out/k8-dbg/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:114] thr<16> #19 0x95fc6d Envoy::Network::ConnectionImpl::onRead()
[2018-07-20 20:46:33.899][16][critical][backtrace] bazel-out/k8-dbg/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:114] thr<16> #20 0x9611b4 Envoy::Network::ConnectionImpl::onReadReady()
[2018-07-20 20:46:33.899][16][critical][backtrace] bazel-out/k8-dbg/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:114] thr<16> #21 0x960fa3 Envoy::Network::ConnectionImpl::onFileEvent()
[2018-07-20 20:46:33.899][16][critical][backtrace] bazel-out/k8-dbg/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:114] thr<16> #22 0x95df1c Envoy::Network::ConnectionImpl::ConnectionImpl()::{lambda()#3}::operator()()
[2018-07-20 20:46:33.899][16][critical][backtrace] bazel-out/k8-dbg/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:114] thr<16> #23 0x962c53 std::_Function_handler<>::_M_invoke()
[2018-07-20 20:46:33.899][16][critical][backtrace] bazel-out/k8-dbg/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:114] thr<16> #24 0x954e8b std::function<>::operator()()
[2018-07-20 20:46:33.899][16][critical][backtrace] bazel-out/k8-dbg/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:114] thr<16> #25 0x953e97 Envoy::Event::FileEventImpl::assignEvents()::{lambda()#1}::operator()()
[2018-07-20 20:46:33.899][16][critical][backtrace] bazel-out/k8-dbg/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:114] thr<16> #26 0x953f01 Envoy::Event::FileEventImpl::assignEvents()::{lambda()#1}::_FUN()
[2018-07-20 20:46:33.899][16][critical][backtrace] bazel-out/k8-dbg/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:114] thr<16> #27 0xf1c6b1 event_process_active_single_queue.isra.29
[2018-07-20 20:46:33.899][16][critical][backtrace] bazel-out/k8-dbg/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:114] thr<16> #28 0xf1ce0e event_base_loop
[2018-07-20 20:46:33.899][16][critical][backtrace] bazel-out/k8-dbg/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:114] thr<16> #29 0x94ea59 Envoy::Event::DispatcherImpl::run()
[2018-07-20 20:46:33.900][16][critical][backtrace] bazel-out/k8-dbg/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:114] thr<16> #30 0x93e87c Envoy::Server::WorkerImpl::threadRoutine()
[2018-07-20 20:46:33.900][16][critical][backtrace] bazel-out/k8-dbg/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:114] thr<16> #31 0x93e1bf Envoy::Server::WorkerImpl::start()::{lambda()#1}::operator()()
[2018-07-20 20:46:33.900][16][critical][backtrace] bazel-out/k8-dbg/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:114] thr<16> #32 0x93ef72 std::_Function_handler<>::_M_invoke()
[2018-07-20 20:46:33.900][16][critical][backtrace] bazel-out/k8-dbg/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:114] thr<16> #33 0x4f7615 std::function<>::operator()()
[2018-07-20 20:46:33.900][16][critical][backtrace] bazel-out/k8-dbg/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:114] thr<16> #34 0xf75387 Envoy::Thread::Thread::Thread()::{lambda()#1}::operator()()
[2018-07-20 20:46:33.900][16][critical][backtrace] bazel-out/k8-dbg/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:114] thr<16> #35 0xf753ac Envoy::Thread::Thread::Thread()::{lambda()#1}::_FUN()
[2018-07-20 20:46:33.900][16][critical][backtrace] bazel-out/k8-dbg/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:104] thr<16> obj</lib64/libpthread.so.0>
[2018-07-20 20:46:33.900][16][critical][backtrace] bazel-out/k8-dbg/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:114] thr<16> #36 0x7fe2c2b55e24 start_thread
[2018-07-20 20:46:33.900][16][critical][backtrace] bazel-out/k8-dbg/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:104] thr<16> obj</lib64/libc.so.6>
[2018-07-20 20:46:33.900][16][critical][backtrace] bazel-out/k8-dbg/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:114] thr<16> #37 0x7fe2c257dbac clone
[2018-07-20 20:46:33.900][16][critical][backtrace] bazel-out/k8-dbg/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:121] end backtrace thread 16

this went away after setting append: false on request_headers_to_add

From looking at the code it seems like the append flow ends up calling a method that asserts that there are no existing headers.

This is running on c92a3017017dfa31241dec13dc6a1479090318d0/1.8.0-dev/Modified/DEBUG

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions