-
Notifications
You must be signed in to change notification settings - Fork 5.3k
Description
Title: Support for progressive traffic increase for new Pods (slow start mode)
Description:
TL;DR; It would be useful to have a slow start mode that allows us to add new pods without overwhelming them with a flood of requests. Similar to this from AWS: https://aws.amazon.com/about-aws/whats-new/2018/05/application-load-balancer-announces-slow-start-support/
We have a JVM-based web app behind Contour/Envoy/NLB, with horizontal pod auto scaling in place.
When a new pod gets created due to auto scaling, Contour/Envoy directs a proportional amount of traffic on that new pod. However, when the app that has just started is overwhelmed with a flood of requests, we're seeing consistent timeouts until it warms up (a couple of minutes). Because of this, whenever we scale out our app, we're losing data. While discussing this with other teams inside Adobe, we've noticed this a common problem with JVM-based apps.
(as you can see in the graph above, whenever a new pod gets created, requests start failing for a couple of minute)Any extra documentation required to understand the issue.
We tried the same scenario by using a Service type LoadBalancer, in EKS (with an Elastic Load Balancer in front) and we don't see the issue. The ELB is doing a progressive traffic increase on the new pod, as the graph seen below.
(in the graph above, you can see the number of requests received by the new pod from the ELB, which is gradually increasing)
