Admission Control Filter

TL; DR: Probabilistically reject outbound requests based on upstream success-rate.

In scenarios where a single request does not have predictable resource utilization, it can be difficult to configure circuit breaker values. Additionally, circuit breakers are unable to protect against connection thrashing, so we rely on local ratelimiting which suffers from the same problems.

It is possible to protect upstreams more generically by detecting when a significant portion of recent requests to an upstream have been rejected and probabilistically reject requests destined for it before sending it on the wire. Finagle calls this admission control and the Google SRE book calls it client-side throttling. Envoy would benefit from a filter that provides this functionality.

The general approach is outlined here, but the idea is that you keep track of the request success-rate over a rolling window and reject outbound requests with a probability of:

We would be able to make use of a configurable measure of success, similar to the retry policy's retry_on parameter.