Skip to content

REST ApiConfigSource without refresh_delay results in floating point exception #2338

Closed
Closed
REST ApiConfigSource without refresh_delay results in floating point exception#2338
Assignees
mattklein123
Labels
bug
Milestone
1.6.0
@JayH5

Description

@JayH5
JayH5
opened on Jan 10, 2018

Title: REST ApiConfigSource without refresh_delay results in floating point exception

Description:
With Envoy version 1.5.0, when configuring an LDS or CDS v2 REST discovery service in the bootstrap configuration, if the refresh_delay parameter is not specified, Envoy will crash and exit with a floating point exception.

I haven't checked if this is also the case with EDS or RDS.

The expected behaviour would be for Envoy to reject the configuration with an explanation of what went wrong.

Adding the refresh_delay field fixes the crash.

Repro steps:
Run the Docker image envoyproxy/envoy:v1.5.0 with the below config. It doesn't matter whether the xds_cluster is accessible or not.

Admin and Stats Output:
Envoy does not stay up long enough to collect this information.

Config:
This is almost exactly the same config as the dynamic example in the documentation but with the xDS API type set to REST instead of GRPC. I'm also using HTTP 1.1 instead of HTTP/2, but I don't think that should matter.

admin:
  access_log_path: /tmp/admin_access.log
  address:
    socket_address: { address: 127.0.0.1, port_value: 9901 }

dynamic_resources:
  lds_config:
    api_config_source:
      api_type: REST
      cluster_name: [xds_cluster]
  cds_config:
    api_config_source:
      api_type: REST
      cluster_name: [xds_cluster]

static_resources:
  clusters:
  - name: xds_cluster
    connect_timeout: 0.25s
    type: STATIC
    lb_policy: ROUND_ROBIN
    http_protocol_options: {}
    hosts: [{ socket_address: { address: 127.0.0.1, port_value: 5000 }}]

Logs:
Note that the config update fails below. Whether the update succeeds or fails, the error that follows still seems to occur.

[2018-01-10 12:29:35.691][1][info][main] source/server/server.cc:158] initializing epoch 0 (hot restart version=9.200.16384.127)
[2018-01-10 12:29:35.706][1][info][upstream] source/common/upstream/cluster_manager_impl.cc:127] cm init: initializing cds
[2018-01-10 12:29:35.707][1][warning][config] bazel-out/local-opt/bin/source/common/config/_virtual_includes/http_subscription_lib/common/config/http_subscription_impl.h:98] REST config update failed: fetch failure
[2018-01-10 12:29:35.707][1][info][upstream] source/common/upstream/cluster_manager_impl.cc:131] cm init: all clusters initialized
[2018-01-10 12:29:35.707][1][critical][backtrace] bazel-out/local-opt/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:101] Caught Floating point exception, suspect faulting address 0x684a52
[2018-01-10 12:29:35.707][1][critical][backtrace] bazel-out/local-opt/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:85] Backtrace obj<envoy> thr<0> (use tools/stack_decode.py):
[2018-01-10 12:29:35.707][1][critical][backtrace] bazel-out/local-opt/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:95] thr<0> #0 0x684a52
[2018-01-10 12:29:35.707][1][critical][backtrace] bazel-out/local-opt/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:95] thr<0> #1 0x694755
[2018-01-10 12:29:35.707][1][critical][backtrace] bazel-out/local-opt/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:95] thr<0> #2 0x696546
[2018-01-10 12:29:35.707][1][critical][backtrace] bazel-out/local-opt/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:95] thr<0> #3 0x72ab99
[2018-01-10 12:29:35.707][1][critical][backtrace] bazel-out/local-opt/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:95] thr<0> #4 0x69717b
[2018-01-10 12:29:35.707][1][critical][backtrace] bazel-out/local-opt/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:95] thr<0> #5 0x69be07
[2018-01-10 12:29:35.707][1][critical][backtrace] bazel-out/local-opt/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:95] thr<0> #6 0x69ca26
[2018-01-10 12:29:35.707][1][critical][backtrace] bazel-out/local-opt/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:95] thr<0> #7 0x694a82
[2018-01-10 12:29:35.707][1][critical][backtrace] bazel-out/local-opt/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:95] thr<0> #8 0x694c30
[2018-01-10 12:29:35.707][1][critical][backtrace] bazel-out/local-opt/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:95] thr<0> #9 0x695635
[2018-01-10 12:29:35.707][1][critical][backtrace] bazel-out/local-opt/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:95] thr<0> #10 0x684c3b
[2018-01-10 12:29:35.707][1][critical][backtrace] bazel-out/local-opt/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:95] thr<0> #11 0x55399f
[2018-01-10 12:29:35.707][1][critical][backtrace] bazel-out/local-opt/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:95] thr<0> #12 0x54fecc
[2018-01-10 12:29:35.707][1][critical][backtrace] bazel-out/local-opt/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:95] thr<0> #13 0x54853d
[2018-01-10 12:29:35.707][1][critical][backtrace] bazel-out/local-opt/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:95] thr<0> #14 0x54da16
[2018-01-10 12:29:35.707][1][critical][backtrace] bazel-out/local-opt/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:95] thr<0> #15 0x54e107
[2018-01-10 12:29:35.707][1][critical][backtrace] bazel-out/local-opt/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:95] thr<0> #16 0x67b967
[2018-01-10 12:29:35.707][1][critical][backtrace] bazel-out/local-opt/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:95] thr<0> #17 0x51b078
[2018-01-10 12:29:35.707][1][critical][backtrace] bazel-out/local-opt/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:95] thr<0> #18 0x51c35b
[2018-01-10 12:29:35.707][1][critical][backtrace] bazel-out/local-opt/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:95] thr<0> #19 0x451700
[2018-01-10 12:29:35.707][1][critical][backtrace] bazel-out/local-opt/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:95] thr<0> #20 0x41405f
[2018-01-10 12:29:35.707][1][critical][backtrace] bazel-out/local-opt/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:93] thr<0> obj</usr/glibc-compat/lib/libc.so.6>
[2018-01-10 12:29:35.707][1][critical][backtrace] bazel-out/local-opt/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:95] thr<0> #21 0x7f03337ca06f
[2018-01-10 12:29:35.707][1][critical][backtrace] bazel-out/local-opt/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:93] thr<0> obj<envoy>
[2018-01-10 12:29:35.707][1][critical][backtrace] bazel-out/local-opt/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:95] thr<0> #22 0x448658
[2018-01-10 12:29:35.707][1][critical][backtrace] bazel-out/local-opt/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:97] end backtrace thread 0

Call Stack:

  #1 Envoy::Http::RestApiFetcher::requestComplete() at rest_api_fetcher.cc:65
  #2 Envoy::Http::AsyncRequestImpl::onComplete() at async_client_impl.cc:195
  #3 Envoy::Http::AsyncStreamImpl::encodeData(Envoy::Buffer::Instance&, bool) at async_client_impl.cc:100
  #4 std::function<void (Envoy::Buffer::Instance&, bool)>::operator()(Envoy::Buffer::Instance&, bool) const at functional:2267
  #5  (inlined by) Envoy::Http::Utility::sendLocalReply(std::function<void (std::unique_ptr<Envoy::Http::HeaderMap, std::default_delete<Envoy::Http::HeaderMap> >&&, bool)>, std::function<void (Envoy::Buffer::Instance&, bool)>, bool const&, Envoy::Http::Code, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) at utility.cc:243
  #6 Envoy::Router::Filter::sendLocalReply(Envoy::Http::Code, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, bool) at router.cc:188
  #7 Envoy::Router::Filter::sendNoHealthyUpstreamResponse() at router.cc:315
  #8 Envoy::Router::Filter::decodeHeaders(Envoy::Http::HeaderMap&, bool) at router.cc:259
  #9 Envoy::Http::AsyncStreamImpl::sendHeaders(Envoy::Http::HeaderMap&, bool) at async_client_impl.cc:123
  #10 Envoy::Http::AsyncRequestImpl::initialize() at async_client_impl.cc:188
  #11 Envoy::Http::AsyncClientImpl::send(std::unique_ptr<Envoy::Http::Message, std::default_delete<Envoy::Http::Message> >&&, Envoy::Http::AsyncClient::Callbacks&, Envoy::Optional<std::chrono::duration<long, std::ratio<1l, 1000l> > > const&) at async_client_impl.cc:45 (discriminator 3)
  #12 Envoy::Http::RestApiFetcher::refresh() at rest_api_fetcher.cc:55
  #13 Envoy::Config::HttpSubscriptionImpl<envoy::api::v2::Cluster>::start(std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&, Envoy::Config::SubscriptionCallbacks<envoy::api::v2::Cluster>&) at http_subscription_impl.h:54
  #14 Envoy::Upstream::CdsApiImpl::initialize() at cds_api_impl.h:31
  #15 Envoy::Upstream::ClusterManagerInitHelper::maybeFinishInitialize() at cluster_manager_impl.cc:129
  #16 Envoy::Upstream::ClusterManagerInitHelper::onStaticLoadComplete() at cluster_manager_impl.cc:142
  #17  (inlined by) Envoy::Upstream::ClusterManagerImpl::ClusterManagerImpl(envoy::api::v2::Bootstrap const&, Envoy::Upstream::ClusterManagerFactory&, Envoy::Stats::Store&, Envoy::ThreadLocal::SlotAllocator&, Envoy::Runtime::Loader&, Envoy::Runtime::RandomGenerator&, Envoy::LocalInfo::LocalInfo const&, Envoy::AccessLog::AccessLogManager&, Envoy::Event::Dispatcher&) at cluster_manager_impl.cc:236
  #18 Envoy::Upstream::ProdClusterManagerFactory::clusterManagerFromProto(envoy::api::v2::Bootstrap const&, Envoy::Stats::Store&, Envoy::ThreadLocal::Instance&, Envoy::Runtime::Loader&, Envoy::Runtime::RandomGenerator&, Envoy::LocalInfo::LocalInfo const&, Envoy::AccessLog::AccessLogManager&) at cluster_manager_impl.cc:708
  #19 Envoy::Server::Configuration::MainImpl::initialize(envoy::api::v2::Bootstrap const&, Envoy::Server::Instance&, Envoy::Upstream::ClusterManagerFactory&) at configuration_impl.cc:40
  #20 Envoy::Server::InstanceImpl::initialize(Envoy::Server::Options&, std::shared_ptr<Envoy::Network::Address::Instance const>, Envoy::Server::ComponentFactory&) at server.cc:244 (discriminator 1)
  #21 Envoy::Server::InstanceImpl::InstanceImpl(Envoy::Server::Options&, std::shared_ptr<Envoy::Network::Address::Instance const>, Envoy::TestHooks&, Envoy::Server::HotRestart&, Envoy::Stats::StoreRoot&, Envoy::Thread::BasicLockable&, Envoy::Server::ComponentFactory&, Envoy::ThreadLocal::Instance&) at server.cc:69
  #22 Envoy::main_common(Envoy::OptionsImpl&) at main_common.cc:87
  #23 main at main.cc:47
  #24
  #25 ?? ??:0
  #26
  #27 _start at ??:?
  #28

Metadata

Metadata

Assignees

Labels

bug

Type

No type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions