Skip to content

Progressive traffic increase for new Pods (slow start mode) #11050

@costimuraru

Description

@costimuraru

Title: Support for progressive traffic increase for new Pods (slow start mode)

Description:
TL;DR; It would be useful to have a slow start mode that allows us to add new pods without overwhelming them with a flood of requests. Similar to this from AWS: https://aws.amazon.com/about-aws/whats-new/2018/05/application-load-balancer-announces-slow-start-support/

We have a JVM-based web app behind Contour/Envoy/NLB, with horizontal pod auto scaling in place.
When a new pod gets created due to auto scaling, Contour/Envoy directs a proportional amount of traffic on that new pod. However, when the app that has just started is overwhelmed with a flood of requests, we're seeing consistent timeouts until it warms up (a couple of minutes). Because of this, whenever we scale out our app, we're losing data. While discussing this with other teams inside Adobe, we've noticed this a common problem with JVM-based apps.

Any extra documentation required to understand the issue.

75562220-e68b6500-5a50-11ea-87fb-5d5d27ef98f7

(as you can see in the graph above, whenever a new pod gets created, requests start failing for a couple of minute)

We tried the same scenario by using a Service type LoadBalancer, in EKS (with an Elastic Load Balancer in front) and we don't see the issue. The ELB is doing a progressive traffic increase on the new pod, as the graph seen below.

75562049-a4622380-5a50-11ea-8354-9af1fa096180

(in the graph above, you can see the number of requests received by the new pod from the ELB, which is gradually increasing)

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions