We have a need for weighted LB that respects the (v2) locality weighting in addition to endpoint weighting. This needs to work initially for RR and Maglev LB, but eventually we would like any implementation to generalize to all LBs (at least at the locality level). This issue will track design discussion and implementation work.

We can weight RR via replication of hosts in the queue, at the expense of some memory use. Similarly, while populating the Maglev table, a similar approach can be taken.

One fundamental design question is whether we have true two-level hierarchical weighted LB (first pick locality, then pick endpoint within locality) or flat LB (combined locality weight and intra-locality endpoint weight for a single weight). Flat LB is conceptually simple, but may be expensive in terms of memory cost when we have a wide range of LB weights that combined weighting would imply. By having true hierarchical LB, we also get "for free" locality weighted LB for LBs that don't support weighting. The downside is we will need to implement at least a simple weighted locality LB and a consistent hash locality LB to support affinity and non-affinity second-level LBs.

I think based on the above it makes sense to do hierarchical LB. If this makes sense, I can work on weighted RR as a first step.

@mattklein123 @zuercher @alyssawilk WDYT?