Research

ICLR 2025 paper on unified routing + cascading shows consistent 4% improvement. Pattern: route to cheapest model first, run quality evaluator, escalate if below threshold. Complements existing Thompson Sampling and EMA routing.

Proposal

1. Add cascade mode to router: try cheapest provider first
2. Implement lightweight output quality classifier using self-learning data
3. Escalate to next provider only if quality prediction below threshold

Sources

• https://openreview.net/forum?id=AAl89VNNy1
• https://arxiv.org/abs/2406.18665