At KubeCon Europe in Berlin last week I presented some work we've done at Huawei scaling Kubernetes in-cluster load balancing to 50,000+ services and beyond, the challenges associated with doing this using the current iptables approach, and what we've achieved using an alternative IPVS-based approach. iptables is designed for firewalling, and based on in-kernel rule lists, while IPVS is designed for load balancing and based on in-kernel hash tables. IPVS also supports more sophisticated load balancing algorithms than iptables (least load, least conns, locality, weighted) as well as other useful features (e.g. health checking, retries etc).

After the presentation, there was strong support (a.k.a. a riot :-) ) for us to open source this work, which we are happy to do. We can use this issue to track that.

For those who were not able to be there, here is the video:

https://youtu.be/c7d_kD2eH4w

And the slides:

https://docs.google.com/presentation/d/1BaIAywY2qqeHtyGZtlyAp89JIZs59MZLKcFLxKE6LyM/edit?usp=sharing

We will follow up on this with a more formal design proposal, and a set of PR's, but in summary we added a about 680 lines of code to the existing 12,000 lines of kube-proxy (~5%), and added a third mode flag to it's command-line (mode=IPVS, to the existing mode=userspace and mode=iptables).
Performance improvement of load balancer updates is dramatic (update latency reduced from hours per rule to 2ms per rule). Network latency and variability also reduced dramatically for large numbers of services.

@kubernetes/sig-network-feature-requests
@kubernetes/sig-scalability-feature-requests
@thockin
@wojtek-t