#3165 added additional file path sanity checking to avoid legitimate bad cases found during fuzz testing.

Unfortunately it breaks one valid case we have.

For background: Consul Connect need to configure Envoy's bootstrap to point it to the local Consul agent as an xDS endpoint. In order to enforce our security model where an agent is shared between different services/pods, the bootstrap therefore has to include a secret value that can be used to Authenticate the caller when passed in gRPC headers:

https://github.com/hashicorp/consul/blob/acfcc7daf4b795a9c531f4da275d5628a4b0017e/command/connect/envoy/bootstrap_tpl.go#L159-L166

...
    "ads_config": {
      "api_type": "GRPC",
      "grpc_services": {
        "initial_metadata": [
          {
            "key": "x-consul-token",
            "value": "{{ .Token }}"
          }
...

We then have a helper command consul connect envoy that generates to correct bootstrap based on the local environment and the central proxy configuration in Consul before execing the envoy binary directly. (It is possible to just output the config for use with container etc. but the exec form is a nice shorthand where both consul and envoy binaries are available e.g. in Dev).

This left us with an unpleasant choice about how to pass the bootstrap config:

1. Write it to a file and pass the path
2. Pass it in the command arg

1 is unpleasant because you are writing a secret to disk somewhere silently without direct user control. 2 is even worse because although the secret isn't on disk it's now visible to most users on the system that can access the /proc info for the command and see it's arguments.

The solution we came up with was to create a temp file, and immediately unlink (without closing) it so it's not accessible in the file system, disable CLOEXEC on it, write the bootstrap content to it, and then exec Envoy. We then pass Envoy the /dev/fd/<fd> path as the config file which allows it to open the file directly from the inherited open FD even though the file is not present on the filesystem. Code for this is in https://github.com/hashicorp/consul/blob/master/command/connect/envoy/exec_unix.go.

Of course this mechanism is now broken in 1.10 by the check for /dev.

It seems reasonable to me to relax the checking and allow /dev/fd/* for this purpose.

We'd also be open to exploring alternative secure ways to pass bootstrap config to Envoy.

Thanks!