Skip to content

onUpstreamData(end_stream=true) is never raised when using cluster with TLS transport socket #13856

@PiotrSikora

Description

@PiotrSikora

It looks that onUpstreamData(..., end_stream=true) and onWrite(..., end_stream=true) events are never raised when using cluster with TLS transport socket.

Repro steps:

Start Envoy:

$ bazel-bin/source/exe/envoy-static -c /tmp/onwrite.yaml -l trace | grep end_stream=

Proxy HTTPS request as TCP using raw buffer (TLS established between curl and httpbin.org):

$ curl --resolve httpbin.org:9091:127.0.0.1 https://httpbin.org:9091/headers

Proxy HTTP request as TCP using TLS transport socket (TLS established between Envoy and httpbin.org):

$ curl --resolve httpbin.org:9092:127.0.0.1 httpbin.org:9092/headers

Config:

$ cat /tmp/onwrite.yaml
admin:
  access_log_path: "/dev/null"
  address:
    socket_address:
      protocol: TCP
      address: 127.0.0.1
      port_value: 9090

static_resources:
  listeners:
  - address:
      socket_address:
        protocol: TCP
        address: 0.0.0.0
        port_value: 9091
    filter_chains:
    - filters:
      - name: envoy.tcp_proxy
        config:
          stat_prefix: tcp_proxy
          cluster: httpbin
  - address:
      socket_address:
        protocol: TCP
        address: 0.0.0.0
        port_value: 9092
    filter_chains:
    - filters:
      - name: envoy.tcp_proxy
        config:
          stat_prefix: tcp_proxy
          cluster: httpbins

  clusters:
  - name: httpbin
    connect_timeout: 0.25s
    type: LOGICAL_DNS
    dns_lookup_family: V4_ONLY
    lb_policy: ROUND_ROBIN
    hosts:
      - socket_address:
          protocol: TCP
          address: httpbin.org
          port_value: 443
  - name: httpbins
    connect_timeout: 0.25s
    type: LOGICAL_DNS
    dns_lookup_family: V4_ONLY
    lb_policy: ROUND_ROBIN
    hosts:
      - socket_address:
          protocol: TCP
          address: httpbin.org
          port_value: 443
    tls_context:
      sni: "httpbin.org"

Logs:

When proxying HTTPS request as TCP using raw buffer (TLS established between curl and httpbin.org):

[C2] downstream connection received 517 bytes, end_stream=false
[C2] upstream connection received 5288 bytes, end_stream=false
[C2] downstream connection received 126 bytes, end_stream=false
[C2] upstream connection received 120 bytes, end_stream=false
[C2] downstream connection received 222 bytes, end_stream=false
[C2] downstream connection received 38 bytes, end_stream=false
[C2] upstream connection received 38 bytes, end_stream=false
[C2] upstream connection received 371 bytes, end_stream=false
[C2] downstream connection received 31 bytes, end_stream=false
[C2] downstream connection received 0 bytes, end_stream=true
[C2] upstream connection received 0 bytes, end_stream=true

When proxying HTTP request as TCP using TLS transport socket (TLS established between Envoy and httpbin.org):

[C4] downstream connection received 87 bytes, end_stream=false
[C4] upstream connection received 403 bytes, end_stream=false
[C4] downstream connection received 0 bytes, end_stream=true

The extra events when proxying HTTPS request as TCP using raw buffer are due to the TLS handshake and shutdown process, so that difference is expected and not an issue.

The issue is that when proxying HTTP request as TCP using TLS transport socket, there is no upstream connection received 0 bytes, end_stream=true event triggered, and the network filters cannot detect when upstream stream ended.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions