Skip to content

Envoy segfaults when a dynamic listener fails to bind to a port and gRPC access logging is enabled #12364

@chriskuehl

Description

@chriskuehl

Envoy segfaults for me when a dynamic listener fails to bind to a port (e.g. because another process is already bound to that port) and gRPC access logging is enabled for that listener. This is happening since at least Envoy 07fa5e0 (Jan 15 2020) and continues today on master (f958d39).

I'm not sure if this is truly related to gRPC access logging, but I wasn't able to narrow down the cause further.

Repro steps:

The smallest reproduction I can find is using the envoy.yaml and listeners.yaml posted here:
https://gist.github.com/chriskuehl/a0672bd237216f5b339daa2980fc3f17

  1. Copy envoy.yaml and listeners.yaml from the gist.
  2. Start any process (not envoy) listening on port 25000. This will conflict with the dynamic listener from the config above and cause the segfault. For example you can run nc -l -p 25000.
  3. With the port still occupied, start Envoy with envoy --config-path envoy.yaml; the Envoy worker should segfault immediately upon start (see the gist for output logs).

Note that the listener needs to be defined as a dynamic resource for this to happen. If you comment out the "access_log" settings from the listener's config and try starting Envoy, you will see an error about the port being unavailable, but no segfault.

I tested this using the envoyproxy/envoy-alpine-debug-dev Docker image today with Envoy sha f958d39 and linux kernel 4.4.0-1094-aws.

Config:
https://gist.github.com/chriskuehl/a0672bd237216f5b339daa2980fc3f17

Logs:
https://gist.github.com/chriskuehl/a0672bd237216f5b339daa2980fc3f17

Call Stack:
Unfortunately I have not been able to get a backtrace to be printed.

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions