AWS SigV4 signer does not canonicalize URI and query string properly #16918
Description
Description:
When calculating the String to Sign for an incoming request, Envoy does not properly canonicalize the path and query parameters. This causes AWS to reject certain requests with the error: The request signature we calculated does not match the signature you provided. Check your AWS Secret Access Key and signing method. Consult the service documentation for details.
I've found the following three error cases so far, but there might be others I'm missing. In all cases below, the root problem seems to be a mismatch between the canonical request expected by AWS (first box) and the one calculated by Envoy (second box), either in the URI or query string lines (first two lines following GET)
-
Query parameters that are not alphabetically sorted
According to SigV4 docs (Step 3), query parameters should be sorted when building the canonical query string. Envoy seems to use the entire query string intact, without parsing or sorting the params. Requests succeed if the query params sent by the client happen to be correctly sorted, but fail with the signature error below otherwise:
$ curl "localhost:10000/v1/?a=test&b=value"
{"statusCode": 200, "body": "{}"}
$ curl "localhost:10000/v1/?b=value&a=test" (edited for clarity)
{"message":"The request signature we calculated does not match the signature you provided.
Check your AWS Secret Access Key and signing method. Consult the service documentation for
details. The Canonical String for this request should have been
GET
/v1/
a=test&b=value
accept:*/*
host:<redacted>.execute-api.us-west-2.amazonaws.com
...
[2021-06-09 16:21:57.845][252264][debug][http] [external/envoy/source/extensions/common/aws/signer_impl.cc:65] Canonical request:
GET
/v1/
b=value&a=test
accept:*/*
host:<redacted>.execute-api.us-west-2.amazonaws.com
-
Query parameters with no value
Per Step 3, parameters with no value specified (e.g.
https://my.site/?foo) should be canonicalized asfoo=, with a trailing equals sign. Because Envoy uses the original query string (foo) instead, these requests always fail.
$ curl "localhost:10000/v1/?a" (edited for clarity)
{"message":"The request signature we calculated does not match the signature you provided.
Check your AWS Secret Access Key and signing method. Consult the service documentation for
details. The Canonical String for this request should have been
GET
/v1/
a=
accept:*/*
host:<redacted>.execute-api.us-west-2.amazonaws.com
...
[2021-06-09 16:22:40.948][252264][debug][http] [external/envoy/source/extensions/common/aws/signer_impl.cc:65] Canonical request:
GET
/v1/
a
accept:*/*
host:<redacted>.execute-api.us-west-2.amazonaws.com
-
Path fragments that contain reserved characters
Per Step 2, path fragments should be double URI-encoded, unless the backend service is S3. Requests to S3 with reserved characters in the path succeed, but requests to other services do not.
$ curl "localhost:10000/v1/documents%20settings"
{"message":"The request signature we calculated does not match the signature you provided.
Check your AWS Secret Access Key and signing method. Consult the service documentation for
details. The Canonical String for this request should have been
GET
/v1/documents%2520settings
accept:*/*
host:<redacted>.execute-api.us-west-2.amazonaws.com
[2021-06-09 16:31:21.342][252272][debug][http] [external/envoy/source/extensions/common/aws/signer_impl.cc:65] Canonical request:
GET
/v1/documents%20settings
accept:*/*
host:<redacted>.execute-api.us-west-2.amazonaws.com
Steps to Repro
- Create an AWS API Gateway with AWS_IAM authorization enabled (any service that is not S3 should do)
- Run Envoy with AWS credentials that have access to the gateway loaded into the environment using the config below
- Run the curl commands above
Config
admin:
access_log_path: /tmp/admin_access.log
address:
socket_address: { address: 127.0.0.1, port_value: 9901 }
static_resources:
listeners:
- name: listener_0
address:
socket_address: { address: 127.0.0.1, port_value: 10000 }
filter_chains:
- filters:
- name: envoy.filters.network.http_connection_manager
typed_config:
"@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
stat_prefix: ingress_http
codec_type: AUTO
route_config:
name: local_route
virtual_hosts:
- name: local_service
domains: ["*"]
routes:
- match: { prefix: "/" }
route: { cluster: some_service }
http_filters:
- name: envoy.filters.http.aws_request_signing
typed_config:
"@type": type.googleapis.com/envoy.extensions.filters.http.aws_request_signing.v3.AwsRequestSigning
service_name: execute-api
region: us-west-2
host_rewrite: <redacted>.execute-api.us-west-2.amazonaws.com
- name: envoy.filters.http.router
clusters:
- name: some_service
connect_timeout: 5s
type: STRICT_DNS
lb_policy: ROUND_ROBIN
load_assignment:
cluster_name: some_service
endpoints:
- lb_endpoints:
- endpoint:
address:
socket_address:
address: <redacted>.execute-api.us-west-2.amazonaws.com
port_value: 443
transport_socket:
name: envoy.transport_sockets.tls