Some examples of the use of SO_ORIGINAL_DST in other proxies for comparison

haproxy:
https://github.com/joyent/haproxy-1.4/blob/master/include/proto/stream_sock.h

squid: https://github.com/schmurfy/squid/blob/a33e4089e463d175014d57afed74d3897a7de02d/src/ip/IpIntercept.cc

redsocks:
https://github.com/darkk/redsocks/blob/master/base.c

@enricoschiattarella at a high level this looks good. I will have some comments but in the meantime can you sign the CLA since that will take a little bit.

SAMPLE USAGE / TESTING

(1) Start simple ncat server on ports 1111 (HTTP) and 2222 (TCP):

while true ; do ncat -l -p 1111 -c 'echo -e "HTTP/1.1 200 OK\n\n Hello from 1111"'; done
ncat -l 2222 -k -c echo "Hello from 2222"

(2) Program iptables to redirect all connections destined to ports 8081 and 8082 to port 12345:

sudo iptables -t nat -I OUTPUT 1 -p tcp --dport 8081 -j REDIRECT --to-port 12345
sudo iptables -t nat -I OUTPUT 1 -p tcp --dport 8082 -j REDIRECT --to-port 12345

(OUTPUT chain only applies to locally-initiated connections)

(3) Start Envoy with below config.

{
  "listeners": [
    {
      "port": 8081,
      "bind_to_port": false,
      "filters": [
        {
          "type": "read",
          "name": "http_connection_manager",
          "config": {
            "codec_type": "auto",
            "stat_prefix": "ingress_http",
            "route_config": {
              "virtual_hosts": [
                {
                  "name": "backend",
                  "domains": ["*"],
                  "routes": [
                    {
                      "timeout_ms": 0,
                      "prefix": "/",
                      "cluster": "service1"
                    }
                  ]
                }
              ]
            },
            "access_log": [
              {
                "path": "/tmp/access.envoy"
              }
            ],
            "filters": [
              {
                "type": "decoder",
                "name": "router",
                "config": {}
              }
            ]
          }
        }
      ]
    },
    {
      "port": 8082,
      "bind_to_port": false,
      "filters": [
        {
          "type": "read",
          "name": "tcp_proxy",
          "config": {
          "cluster": "service2",
          "stat_prefix": "tcp"
          }
        }
      ]
    },
    {
      "port": 12345,
      "use_original_dst": true,
      "filters": []
    }
  ],
  "admin": {
    "access_log_path": "/tmp/access.envoy",
    "port": 8001
  },
  "cluster_manager": {
    "clusters": [
      {
        "name": "service1",
        "connect_timeout_ms": 250,
        "type": "strict_dns",
        "lb_type": "round_robin",
        "hosts": [
          {
            "url": "tcp://localhost:1111"
          }
        ]
      },
      {
        "name": "service2",
        "connect_timeout_ms": 250,
        "type": "strict_dns",
        "lb_type": "round_robin",
        "hosts": [
          {
            "url": "tcp://localhost:2222"
          }
        ]
      }
    ]
  }
}

(4) Use curl to verify that connections established to ports 8081 and 8082 go through the proxy and finally make it to the actual service:

curl localhost:8081
Hello from 1111
curl localhost:8082
Hello from 2222

cat /tmp/envoy.access
[2017-01-06T17:40:50.474Z] "GET / HTTP/1.1" 200 - 0 17 3 2 "-" "curl/7.51.0-DEV" "6eb90d05-893d-4561-ba2b-be27ea111e50" "localhost:8081" "tcp://127.0.0.1:1111"

Note: it is possible for an unbound Envoy listener to use the same port the app is listening on.
In our example, the apps are listening on ports 1111 and 222; Envoy unbound listeners would also listen on 1111 and 2222, so that the proxying is transparent to the client. In that case, however, the iptables rules have to be tweaked, so that when the proxy tries to connect to the app on ports 1111 and 2222,
the connections are not intercepted and redirected back to the proxy.
This can be done by using the "-m owner" iptables option, which excludes connections originating from a process owned by a given UID/GID, using the PREROUTING chain, which is not traversed by locally generated packets, or using iptables MARKS.

More details in the doc referenced in the commit message.

All comments look straightforward except the one about downcasting Listener to ListenerImpl.
Please let me know what is your take on that.
I will fix that and incorporate the others in the meanwhile.

I have implemented the interface for ConnectionHandler and solved the dependency violation between source/common and source/server.

However, I haven't been able to get rid of the downcast from Listener to ListenerImpl in listener_impl.cc.

The options I have considered are:
(1) add the newConnection method defined on the ListenerImpl to the Listener interface
(2) push the job of redirecting a connection to the ConnectionHandler by having a method like ConnectionHandler::redirect(connection, socket_name)

both are reasonable in principle but have the drawback of adding low-level details about the connection (fd, sockaddr, ...) to interfaces.

At this point the best solution seems to be the dynamic_cast<>, so I went with that.

Please let me know if you have other alternatives in mind.

Thanks!

@enricoschiattarella I think you have some other changes in this change. Do you mind merging master or just rebasing and pushing? I can then give it a final pass.

@mattklein123 done. PTAL when you have time. Thanks!

@enricoschiattarella, @mattklein123

The documentation says:

If there is no listener associated with the original destination port,
the connection is handled by the [original] listener that receives it."

What is the desired behavior if the original listener has no filters like in the example above? I would expect the connection to be refused (no port matched from connecting client's perspective), but it seems the TCP connection is opened and then ... nothing.

Should a listener with empty filter list and use_original_dst=true reject TCP connection altogether? Filters array is normally required (since listener configuration doesn't make sense without them) but an empty filter list doesn't throw an error. And in this special case it does make some sense to have no filter's specified since it only serves as a redirect.

@ayj I think this is just a bug in general. If there are no filters, the connection handler should just close the connection. Feel free to fix or please open a GH issue and we can fix.