Description:

What issue is being seen? Describe what should be happening instead of
the bug, for example: The expected value isn't returned, etc.

should only be set when Retries are unset

Repro steps:

Include sample requests, environment, etc. All data and inputs
required to reproduce the bug.

Note: If there are privacy concerns, sanitize the data prior to
sharing.

Environment:

Include the environment like gateway version, envoy version and so on.

Logs:

Include the access logs and the Envoy logs.