Skip to content

feat(routing): ASI coherence tracking and unified quality-gate cascading (#1841, #2165)#2713

Merged
bug-ops merged 1 commit intomainfrom
routing-asi-unified-cascading
Apr 6, 2026
Merged

feat(routing): ASI coherence tracking and unified quality-gate cascading (#1841, #2165)#2713
bug-ops merged 1 commit intomainfrom
routing-asi-unified-cascading

Conversation

@bug-ops
Copy link
Copy Markdown
Owner

@bug-ops bug-ops commented Apr 6, 2026

Summary

  • ASI coherence tracking (crates/zeph-llm/src/router/asi.rs new): per-provider sliding window of response embeddings; coherence score (cosine similarity vs window mean) penalizes Thompson beta and EMA score when responses drift; emits tracing::warn! on ASI drift below configurable threshold
  • Quality-gate cascading (router/mod.rs): after Thompson/EMA selection, optionally compute cosine similarity between query and response embeddings; below threshold escalates to next provider in pool; on exhaustion returns best response seen; fail-open on embed errors
  • Bounds validation for quality_gate (must be in (0.0, 1.0]) and ASI coherence_threshold/penalty_weight (clamped with warn log)
  • 4 new unit tests covering gate happy path, exhaustion fallback, and guard logic

Config

[llm.routing]
quality_gate = 0.75   # optional, disabled by default

[llm.routing.asi]
enabled = false
window = 5
coherence_threshold = 0.7
penalty_weight = 0.3

Test plan

  • cargo nextest run --workspace --lib --bins — 7690/7690 pass
  • cargo clippy --workspace -- -D warnings — clean
  • cargo +nightly fmt --check — clean
  • Live session test with quality_gate enabled — verify escalation logged at info
  • Live session test with ASI enabled and window=3 — verify drift detection fires

Closes #1841, #2165

@github-actions github-actions Bot added documentation Improvements or additions to documentation llm zeph-llm crate (Ollama, Claude) rust Rust code changes core zeph-core crate enhancement New feature or request size/XL Extra large PR (500+ lines) labels Apr 6, 2026
@bug-ops bug-ops enabled auto-merge (squash) April 6, 2026 18:13
@bug-ops bug-ops linked an issue Apr 6, 2026 that may be closed by this pull request
…ing (#1841, #2165)

Add two research-driven routing improvements to zeph-llm:

1. Agent Stability Index (ASI) — per-provider sliding window of response embeddings.
   Coherence score (cosine sim vs window mean) penalizes Thompson beta and EMA score
   when responses drift. Fires tracing::warn on ASI drift below threshold.
   Config: [llm.routing.asi] enabled/window/coherence_threshold/penalty_weight.

2. Quality-gate cascading — after Thompson/EMA selection, optionally compute cosine
   similarity between query and response embeddings. Below threshold, escalate to next
   provider; on pool exhaustion, return best response seen. Fail-open on embed errors.
   Config: [llm.routing] quality_gate = 0.75 (optional).

Bounds validation added for quality_gate (0.0, 1.0] and ASI params with warn log on
invalid values. Quality fallback logs at info level (not warn).
@bug-ops bug-ops force-pushed the routing-asi-unified-cascading branch from 1b4cecd to 1feb267 Compare April 6, 2026 18:23
@bug-ops bug-ops merged commit 8fb366a into main Apr 6, 2026
27 checks passed
@bug-ops bug-ops deleted the routing-asi-unified-cascading branch April 6, 2026 18:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

core zeph-core crate documentation Improvements or additions to documentation enhancement New feature or request llm zeph-llm crate (Ollama, Claude) rust Rust code changes size/XL Extra large PR (500+ lines)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

research(routing): unified routing+cascading framework (ICLR 2025) research(routing): Agent Stability Index for behavioral drift detection

1 participant