envoy validate Panics /source/server/config_validation/server.h

*Title*: *envoy validate Panics /source/server/config_validation/server.h*

*Description*:
>What issue is being seen? Describe what should be happening instead of
the bug, for example: Envoy should not crash, the expected value isn't
returned, etc.

When running envoy validate to validate the envoy_config it was passing until Monday afternoon. Starting Monday afternoon, it began to panic:
```
[2021-07-14 15:56:32.158][9][critical][assert] [./source/server/config_validation/server.h:113] panic: not implemented
[2021-07-14 15:56:32.158][9][critical][backtrace] [./source/server/backtrace.h:104] Caught Aborted, suspect faulting address 0x9
[2021-07-14 15:56:32.158][9][critical][backtrace] [./source/server/backtrace.h:91] Backtrace (use tools/stack_decode.py to get line numbers):
[2021-07-14 15:56:32.158][9][critical][backtrace] [./source/server/backtrace.h:92] Envoy version: a1606b267c27a6ffc057ac3abe7ae38734088265/1.19.0-dev/Clean/RELEASE/BoringSSL
[2021-07-14 15:56:32.158][9][critical][backtrace] [./source/server/backtrace.h:98] #0: [0x7f73b51cc3d0]
[2021-07-14 15:56:32.158][9][critical][backtrace] [./source/server/backtrace.h:98] #1: [0x55cc892e6764]
[2021-07-14 15:56:32.158][9][critical][backtrace] [./source/server/backtrace.h:98] #2: [0x55cc892e7df8]
[2021-07-14 15:56:32.158][9][critical][backtrace] [./source/server/backtrace.h:98] #3: [0x55cc8ab86fd6]
[2021-07-14 15:56:32.158][9][critical][backtrace] [./source/server/backtrace.h:98] #4: [0x55cc8ab8453b]
[2021-07-14 15:56:32.158][9][critical][backtrace] [./source/server/backtrace.h:98] #5: [0x55cc8ab81bd2]
[2021-07-14 15:56:32.158][9][critical][backtrace] [./source/server/backtrace.h:98] #6: [0x55cc8ab899df]
[2021-07-14 15:56:32.158][9][critical][backtrace] [./source/server/backtrace.h:98] #7: [0x55cc8ab3c859]
[2021-07-14 15:56:32.158][9][critical][backtrace] [./source/server/backtrace.h:98] #8: [0x55cc8a830fc4]
[2021-07-14 15:56:32.158][9][critical][backtrace] [./source/server/backtrace.h:98] #9: [0x55cc8ab4775c]
[2021-07-14 15:56:32.159][9][critical][backtrace] [./source/server/backtrace.h:98] #10: [0x55cc8ab47599]
[2021-07-14 15:56:32.159][9][critical][backtrace] [./source/server/backtrace.h:98] #11: [0x55cc8ab52616]
[2021-07-14 15:56:32.159][9][critical][backtrace] [./source/server/backtrace.h:98] #12: [0x55cc8ab36579]
[2021-07-14 15:56:32.159][9][critical][backtrace] [./source/server/backtrace.h:98] #13: [0x55cc8ab34230]
[2021-07-14 15:56:32.159][9][critical][backtrace] [./source/server/backtrace.h:98] #14: [0x55cc8ab42d2c]
[2021-07-14 15:56:32.159][9][critical][backtrace] [./source/server/backtrace.h:98] #15: [0x55cc8ab41b39]
[2021-07-14 15:56:32.159][9][critical][backtrace] [./source/server/backtrace.h:98] #16: [0x55cc8ab75df7]
[2021-07-14 15:56:32.159][9][critical][backtrace] [./source/server/backtrace.h:98] #17: [0x55cc8a82f1f8]
[2021-07-14 15:56:32.159][9][critical][backtrace] [./source/server/backtrace.h:98] #18: [0x55cc8a82e0b3]
[2021-07-14 15:56:32.159][9][critical][backtrace] [./source/server/backtrace.h:98] #19: [0x55cc8a82d5a5]
[2021-07-14 15:56:32.159][9][critical][backtrace] [./source/server/backtrace.h:98] #20: [0x55cc890d7210]
[2021-07-14 15:56:32.159][9][critical][backtrace] [./source/server/backtrace.h:98] #21: [0x55cc890d7ad4]
[2021-07-14 15:56:32.159][9][critical][backtrace] [./source/server/backtrace.h:98] #22: [0x55cc890d409c]
[2021-07-14 15:56:32.159][9][critical][backtrace] [./source/server/backtrace.h:96] #23: __libc_start_main [0x7f73b5019c8d]
Aborted
```

I tracked that down to this PR: https://github.com/envoyproxy/envoy/pull/16955
Which introduced this line: https://github.com/envoyproxy/envoy/blob/bceeb666247a1cf5cbc5ecc5de84a9301c0fa971/source/server/config_validation/server.h#L113


My question is, how can we fix or work around the panic? Or is this a bug that has broken the validator?

*Repro steps*:
> Include sample requests, environment, etc. All data and inputs
required to reproduce the bug.

Run:
envoy -c /etc/service-envoy.yaml --service-cluster service --mode validate






*Config*:
>Include the config used to configure Envoy.
```
static_resources:
  listeners:
  - address:
      socket_address:
        address: 0.0.0.0
        port_value: 8443
    filter_chains:
    - filters:
      - name: envoy.filters.network.http_connection_manager
        typed_config:
          "@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
          codec_type: auto
          stat_prefix: ingress_http
          access_log:
          - name: envoy.access_loggers.file
            typed_config:
              "@type": type.googleapis.com/envoy.extensions.access_loggers.file.v3.FileAccessLog
              path: /dev/stdout
              json_format:
                level: INFO
                date: "%START_TIME(%Y/%m/%dT%H:%M:%S%z)%"
                method: "%REQ(:METHOD)%"
                path: "%REQ(X-ENVOY-ORIGINAL-PATH?:PATH)%"
                protocol: "%PROTOCOL%"
                host: “%UPSTREAM_HOST%”
                response_code: "%RESPONSE_CODE%"
                response_flags: "%RESPONSE_FLAGS%"
                bytes_received: "%BYTES_RECEIVED%"
                bytes_sent: "%BYTES_SENT%"
                duration: "%DURATION%"
                response_time: "%RESP(X-ENVOY-UPSTREAM-SERVICE-TIME)%"
                request_fwd_for: "%REQ(X-FORWARDED-FOR)%"
                user_agent: "%REQ(USER-AGENT)%"
                request_id: "%REQ(X-REQUEST-ID)%"
                authority: "%REQ(:AUTHORITY)%"
                upstream_host: "%UPSTREAM_HOST%"
          route_config:
            name: local_route
            internal_only_headers: ["x-foo-bar-test"]
            virtual_hosts:
            - name: service
              domains:
              - "*"
              routes:
              - match:
                  prefix: "/health"
                route:
                  cluster: local_service
                typed_per_filter_config:
                  envoy.filters.http.ext_authz:
                    "@type": type.googleapis.com/envoy.extensions.filters.http.ext_authz.v3.ExtAuthzPerRoute
                    disabled: true
              - match:
                  prefix: "/env"
                route:
                  cluster: local_service
                typed_per_filter_config:
                  envoy.filters.http.ext_authz:
                    "@type": type.googleapis.com/envoy.extensions.filters.http.ext_authz.v3.ExtAuthzPerRoute
                    disabled: true
              - match:
                  prefix: "/docs"
                route:
                  cluster: local_service
                typed_per_filter_config:
                  envoy.filters.http.ext_authz:
                    "@type": type.googleapis.com/envoy.extensions.filters.http.ext_authz.v3.ExtAuthzPerRoute
                    disabled: true
              - match:
                  prefix: "/swagger"
                route:
                  cluster: local_service
                typed_per_filter_config:
                  envoy.filters.http.ext_authz:
                    "@type": type.googleapis.com/envoy.extensions.filters.http.ext_authz.v3.ExtAuthzPerRoute
                    disabled: true
              - match:
                  prefix: "/metrics"
                route:
                  cluster: local_service
                typed_per_filter_config:
                  envoy.filters.http.ext_authz:
                    "@type": type.googleapis.com/envoy.extensions.filters.http.ext_authz.v3.ExtAuthzPerRoute
                    disabled: true   
              - match:
                  prefix: "/"
                route:
                  cluster: local_service
          http_filters:
          - name: envoy.ext_authz
            typed_config:
              "@type": type.googleapis.com/envoy.extensions.filters.http.ext_authz.v3.ExtAuthz
              failure_mode_allow: false
              transport_api_version: V3
              status_on_error:
                code: 503
              grpc_service:
                envoy_grpc:
                  cluster_name: ext_authz-opa-service
                timeout: 0.5s
              with_request_body:
                max_request_bytes: 10240
                allow_partial_message: true
                pack_as_bytes: false
          - name: envoy.filters.http.router
            typed_config: {}
      transport_socket:
        name: envoy.transport_sockets.tls
        typed_config:
          "@type": type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.DownstreamTlsContext
          common_tls_context:
            tls_certificates:
              - certificate_chain:
                  filename: "/etc/crt.pem"
                private_key:
                  filename: "/etc/key.pem"
  - address:
      socket_address:
        address: 0.0.0.0
        port_value: 8080
    filter_chains:
    - filters:
      - name: envoy.filters.network.http_connection_manager
        typed_config:
          "@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
          codec_type: auto
          stat_prefix: ingress_http
          access_log:
          - name: envoy.access_loggers.file
            typed_config:
              "@type": type.googleapis.com/envoy.extensions.access_loggers.file.v3.FileAccessLog
              path: /dev/stdout
              json_format:
                level: INFO
                date: "%START_TIME(%Y/%m/%dT%H:%M:%S%z)%"
                method: "%REQ(:METHOD)%"
                path: "%REQ(X-ENVOY-ORIGINAL-PATH?:PATH)%"
                protocol: "%PROTOCOL%"
                host: “%UPSTREAM_HOST%”
                response_code: "%RESPONSE_CODE%"
                response_flags: "%RESPONSE_FLAGS%"
                bytes_received: "%BYTES_RECEIVED%"
                bytes_sent: "%BYTES_SENT%"
                duration: "%DURATION%"
                response_time: "%RESP(X-ENVOY-UPSTREAM-SERVICE-TIME)%"
                request_fwd_for: "%REQ(X-FORWARDED-FOR)%"
                user_agent: "%REQ(USER-AGENT)%"
                request_id: "%REQ(X-REQUEST-ID)%"
                authority: "%REQ(:AUTHORITY)%"
                upstream_host: "%UPSTREAM_HOST%"
          route_config:
            name: local_route
            internal_only_headers: ["x-foo-bar-test"]
            virtual_hosts:
            - name: service
              retry_policy:
                retry_on: 5xx
                num_retries: 2
              domains:
              - "*"
              routes:
              - match:
                  prefix: "/health"
                route:
                  cluster: local_service
                typed_per_filter_config:
                  envoy.filters.http.ext_authz:
                    "@type": type.googleapis.com/envoy.extensions.filters.http.ext_authz.v3.ExtAuthzPerRoute
                    disabled: true
              - match:
                  prefix: "/env"
                route:
                  cluster: local_service
                typed_per_filter_config:
                  envoy.filters.http.ext_authz:
                    "@type": type.googleapis.com/envoy.extensions.filters.http.ext_authz.v3.ExtAuthzPerRoute
                    disabled: true
              - match:
                  prefix: "/docs"
                route:
                  cluster: local_service
                typed_per_filter_config:
                  envoy.filters.http.ext_authz:
                    "@type": type.googleapis.com/envoy.extensions.filters.http.ext_authz.v3.ExtAuthzPerRoute
                    disabled: true
              - match:
                  prefix: "/swagger"
                route:
                  cluster: local_service
                typed_per_filter_config:
                  envoy.filters.http.ext_authz:
                    "@type": type.googleapis.com/envoy.extensions.filters.http.ext_authz.v3.ExtAuthzPerRoute
                    disabled: true
              - match:
                  prefix: "/metrics"
                route:
                  cluster: local_service
                typed_per_filter_config:
                  envoy.filters.http.ext_authz:
                    "@type": type.googleapis.com/envoy.extensions.filters.http.ext_authz.v3.ExtAuthzPerRoute
                    disabled: true
              - match:
                  prefix: "/"
                route:
                  cluster: local_service
          http_filters:
          - name: envoy.ext_authz
            typed_config:
              "@type": type.googleapis.com/envoy.extensions.filters.http.ext_authz.v3.ExtAuthz
              failure_mode_allow: false
              transport_api_version: V3
              status_on_error:
                code: 503
              grpc_service:
                envoy_grpc:
                  cluster_name: ext_authz-opa-service
                timeout: 0.5s
              with_request_body:
                max_request_bytes: 10240
                allow_partial_message: true
                pack_as_bytes: false
          - name: envoy.filters.http.router
            typed_config: {}
  clusters:
  - name: local_service
    connect_timeout: 30s
    type: strict_dns
    lb_policy: round_robin
    respect_dns_ttl: true
    health_checks:
      timeout: 1s
      interval: 2s
      interval_jitter: 1s
      unhealthy_threshold: 3
      healthy_threshold: 3
      no_traffic_interval: 60s
      event_log_path: /dev/stdout
      http_health_check:
        path: /health
    load_assignment:
      cluster_name: local_service
      endpoints:
      - lb_endpoints:
        - endpoint:
            address:
              socket_address:
                address: main
                port_value: 8080

  - name: ext_authz-opa-service
    type: strict_dns
    lb_policy: round_robin
    typed_extension_protocol_options:
      envoy.extensions.upstreams.http.v3.HttpProtocolOptions:
        "@type": type.googleapis.com/envoy.extensions.upstreams.http.v3.HttpProtocolOptions
        explicit_http_config:
          http2_protocol_options: {}
    health_checks:
      timeout: 1s
      interval: 3s
      interval_jitter: 1s
      initial_jitter: 3s
      unhealthy_threshold: 3
      healthy_threshold: 3
      no_traffic_interval: 5s
      no_traffic_healthy_interval: 10s
      event_log_path: /dev/stdout
      always_log_health_check_failures: true
      http_health_check:
        path: /health
    load_assignment:
      cluster_name: ext_authz-opa-service
      endpoints:
      - lb_endpoints:
        - endpoint:
            address:
              socket_address:
                address: opa
                port_value: ${OPA_PORT}
            health_check_config:
              port_value: ${OPA_HEALTH_PORT}

admin:
  access_log_path: "/dev/null"
  address:
    socket_address:
      address: 0.0.0.0
      port_value: 8081
```



Provide feedback

Saved searches

Use saved searches to filter your results more quickly

envoy validate Panics /source/server/config_validation/server.h #17344

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

envoy validate Panics /source/server/config_validation/server.h #17344

Description

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions