Skip to content

Refine router feedback parsing, reaction normalization, and deduplicated calibration metrics#175

Merged
dgarson merged 1 commit intocodex/implement-analytics-dashboard-frameworkfrom
codex/review-pr-for-data-quality-issues
Feb 24, 2026
Merged

Refine router feedback parsing, reaction normalization, and deduplicated calibration metrics#175
dgarson merged 1 commit intocodex/implement-analytics-dashboard-frameworkfrom
codex/review-pr-for-data-quality-issues

Conversation

@dgarson
Copy link
Owner

@dgarson dgarson commented Feb 24, 2026

Motivation

  • Implicit feedback parsing was too permissive and could infer tiers from descriptive text, producing false-positive corrections.
  • A single corrupted JSONL line could break persisted reads and disable downstream summary/review flows.
  • Duplicate feedback records were inflating calibration metrics and creating redundant open review items.

Description

  • Tightened implicit-tier extraction in parseImplicitFeedback to only infer T1..T4 when correction-oriented cues are present (change in src/routing/feedback-loop.ts).
  • Added normalizeReactionValue() and used it in fingerprintFeedback and reaction handling to ensure consistent reaction normalization (in src/routing/feedback-loop.ts).
  • Hardened readJsonl() to skip malformed JSONL lines with a warning and continue returning well-formed records (in src/routing/feedback-loop.ts).
  • Prevented duplicate feedback from creating review queue items and excluded duplicates from mismatch/false-escalation/latency/source aggregation while still counting duplicates (in src/routing/feedback-loop.ts).
  • Updated and extended tests in src/routing/feedback-loop.test.ts to cover descriptive-tier negatives, corrupted JSONL tolerance, and deduplicated summary expectations, and fixed formatting drift in src/gateway/hitl-request-store.ts.

Testing

  • Ran pnpm format and pnpm format:fix to apply repository formatting (succeeded).
  • Ran pnpm test src/routing/feedback-loop.test.ts and the file's test suite passed (7 tests passed).
  • Ran pnpm check, which reports pre-existing unrelated TypeScript/module issues in other areas of the repo that are not caused by this change.

Codex Task

@dgarson dgarson merged commit 5b2b163 into codex/implement-analytics-dashboard-framework Feb 24, 2026
2 of 10 checks passed
dgarson added a commit that referenced this pull request Feb 24, 2026
* Docs: fix router feedback analytics schema tables

* Routing: wire feedback loop into runtime surfaces

* Routing: deepen feedback telemetry and review lifecycle

* Routing: tighten implicit feedback calibration signals (#175)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant