Skip to content
This repository was archived by the owner on May 26, 2026. It is now read-only.

feat(kora): KR-FE-COST-TELEMETRY-PANEL — per-route cost visibility#164

Merged
rafe-walker merged 1 commit into
feature/phase2-upgradesfrom
feat/kora-KR-FE-COST-TELEMETRY-PANEL
May 24, 2026
Merged

feat(kora): KR-FE-COST-TELEMETRY-PANEL — per-route cost visibility#164
rafe-walker merged 1 commit into
feature/phase2-upgradesfrom
feat/kora-KR-FE-COST-TELEMETRY-PANEL

Conversation

@rafe-walker

Copy link
Copy Markdown
Owner

Summary

Per the cost-economy thesis (feedback-opus-escalation-must-be-earned + cheap-substrate-first): the per-route burn / escalation rate / cache effectiveness has to be operator-visible OR the substrate work is invisible value. This panel turns the snapshot v2 cost_telemetry section (PR #161) into operator UI.

Three windows × three subsections

CostTelemetryPage — 3 window states

Windows: Lifetime / Rolling 24h / Monthly

  • Lifetime is endpoint-only (/api/cost_telemetry) — process_lifetime is too unbounded for the on-disk snapshot. Auto-fetches when the operator picks the tab from the snapshot-success path.
  • Rolling 24h + Monthly default to the $0 snapshot read; Force-refresh button falls back to /api/cost_telemetry for fresh-as-of-now numbers.

Subsections per window:

  1. Cache effectiveness — horizontal stacked bar (cache_read / cache_creation / uncached). Total spend + total calls in the window header.
  2. Per-route breakdown — table with calls, in/out tokens, cost, escalation rate.
  3. Model breakdown — horizontal stacked bar in tier order (cheap → expensive: short_circuithaikusonnetopus) so the operator's mental model maps onto the cost ladder left-to-right.

Route disposition (1 active + 7 awaiting + 1 fallback)

Active today:

  • slack_dm — real counters from the live telemetry singleton

Reserved-no-consumer (clear "[Awaiting consumer]" marker + queued bucket cited inline so the panel doesn't look broken):

Route Queued bucket
email_inbound KR-EMAIL-COST-BILL
email_outbound_compose KR-EMAIL-OUTBOUND-COST-BILL
mcp_tool KR-MCP-COST-BILL
alert_investigation KR-ALERT-INVESTIGATION-COST-BILL
probe_investigation KR-PROBE-INVESTIGATION-COST-BILL
tool_loop_iteration KR-TOOL-LOOP-COST-BILL
scheduled_task KR-SCHEDULED-TASK-COST-BILL

Default fallback:

  • unknown — labeled "(agent-side fallback)" so operator knows these are unattributed (Kora's own tooling, not a real consumer route)

KNOWN_ROUTES literal matches kora_cli/telemetry/cost_telemetry.py:73-81 exactly — pinned by test_known_routes_match_backend_writer so FE/BE drift is caught at the test gate.

Escalation rate color bands

Per feedback-opus-escalation-must-be-earned:

  • Green (5–15%) — target band (Haiku-router earning Opus escalations at the right rate)
  • Yellow (<5% or 15–30%) — out of band, classifier needs tuning
  • Red (>30%) — classifier too loose; cost-economy broken

Pre-router state (today): escalation_count is always 0 because no router exists yet (KR-HAIKU-ROUTER is CC#3 in-flight). Renders "pre-router" instead of "0.0%" which would imply a (good!) 0% rate from a router that doesn't exist.

Chart-library choice: plain SVG / divs (no new dep)

Per spec §4 STOP-ASK preference. Bars are flex containers with per-segment divs whose widths come from inline style={{ width: \${pct}%` }}. Pinned by test_no_chart_library_added— verifiesrecharts/chart.js/victory/d3/echarts/plotlyare NOT inweb/package.json`.

Wired

Test plan

  • tests/kora_cli/test_cost_telemetry_panel.py20 source-pin tests: api wrapper, RouteCounters + CostTelemetryResponse types, snapshot type refinement, page exists + uses usePanelView, 3 windows declared, snapshot-first + force-refresh paths, Lifetime auto-fetch, KNOWN_ROUTES backend-match pin, ROUTE_BUCKET_REF citations, escalation color bands (green 5–15%, red >30%), pre-router state, cache-effectiveness zero-denom + uncached floor, route registered, no chart library added, CSS-width chart implementation.
  • pnpm tsc -b clean
  • pnpm build clean
  • Manual smoke: open /cost-telemetry against live daemon → 3 tabs functional, snapshot-source label updates per tab, force-refresh swaps to live source.

Refs

🤖 Generated with Claude Code

Per the cost-economy thesis (feedback-opus-escalation-must-be-
earned + cheap-substrate-first): the per-route burn / escalation
rate / cache effectiveness has to be operator-visible OR the
substrate work is invisible value. This panel turns the snapshot
v2 cost_telemetry section (PR #161) into operator-visible UI.

Three windows × three subsections
==================================

Windows: Lifetime / Rolling 24h / Monthly
  * Lifetime is endpoint-only (/api/cost_telemetry) — process_lifetime
    is too unbounded for the on-disk snapshot. Auto-fetches when
    the operator picks the tab from the snapshot-success path.
  * Rolling 24h + Monthly default to the $0 snapshot read; Force-
    refresh button falls back to /api/cost_telemetry for fresh-
    as-of-now numbers.

Subsections per window:
  * Cache effectiveness — horizontal stacked bar (cache_read /
    cache_creation / uncached). Total spend + total calls in
    the window header.
  * Per-route breakdown — table with calls, in/out tokens, cost,
    escalation rate. Reserved-no-consumer routes show
    "[Awaiting consumer]" with the queued bucket cited inline
    so the panel doesn't look broken before the wiring lands.
  * Model breakdown — horizontal stacked bar in tier order
    (cheap→expensive: short_circuit → haiku → sonnet → opus)
    so the operator's mental model maps onto the cost ladder
    left-to-right.

Route disposition (active vs awaiting vs fallback)
====================================================

Active today (1 route):
  * slack_dm — real counters

Reserved-no-consumer (7 routes — cite the queued bucket):
  * email_inbound           → KR-EMAIL-COST-BILL
  * email_outbound_compose  → KR-EMAIL-OUTBOUND-COST-BILL
  * mcp_tool                → KR-MCP-COST-BILL
  * alert_investigation     → KR-ALERT-INVESTIGATION-COST-BILL
  * probe_investigation     → KR-PROBE-INVESTIGATION-COST-BILL
  * tool_loop_iteration     → KR-TOOL-LOOP-COST-BILL
  * scheduled_task          → KR-SCHEDULED-TASK-COST-BILL

Default fallback (1 route):
  * unknown — labeled "agent-side fallback" so operator knows
    these are unattributed (Kora's own tooling, not a real
    consumer route)

KNOWN_ROUTES literal matches kora_cli/telemetry/cost_telemetry.py
:73-81 exactly — pinned by test_known_routes_match_backend_writer
so any drift between FE/BE is caught at the test gate.

Escalation rate color bands
============================

Per feedback-opus-escalation-must-be-earned:
  * Green (5–15%)  — target band (Haiku-router earning Opus
                     escalations at the right rate)
  * Yellow (<5% or 15–30%) — out of band, classifier needs tuning
  * Red (>30%)     — classifier too loose; cost-economy broken

Pre-router state (today): escalation_count is always 0 because
no router exists yet (KR-HAIKU-ROUTER is CC#3 in-flight per spec
§1). Render "pre-router" instead of "0.0%" which would imply a
(good!) 0% rate from a router that doesn't exist — pin in
test_pre_router_state_renders_gracefully.

Chart-library choice: plain SVG / divs (no new dep)
====================================================

Per spec §4 preference: bars are flex containers with per-
segment divs whose widths come from inline
style={{ width: `${pct}%` }}. Pinned by test_no_chart_library_
added (verifies recharts / chart.js / victory / d3 / echarts /
plotly are NOT in web/package.json).

Wired
======

  * api.getCostTelemetry() wrapper + RouteCounters +
    CostTelemetryResponse TS types
  * SnapshotResponse.cost_telemetry refined from opaque `unknown`
    (PR #162) to typed {rolling_24h, monthly}
  * web/src/pages/CostTelemetryPage.tsx — new top-level page
  * /cost-telemetry route + "Cost Telemetry" nav entry
    (BarChart3 icon; placed next to /cost-state in the sidebar
    since they're related domains — CostState = aggregate
    rung/budget, CostTelemetry = per-route breakdown)
  * usePanelView("CostTelemetryPage") for usage instrumentation
    (per PR #159 discipline)

Tests (tests/kora_cli/test_cost_telemetry_panel.py — 20 tests)
================================================================

Source-pin tests — no FE component-runner per the established
CC#2 pattern:
  * api.getCostTelemetry wrapper + RouteCounters +
    CostTelemetryResponse types + SnapshotResponse refinement
  * Page exists, uses usePanelView, declares 3 windows
  * loadFromSnapshot called first; forceLiveRefresh bypasses snapshot
  * Lifetime tab auto-fetches endpoint when picked from snapshot path
  * KNOWN_ROUTES literal matches backend writer exactly
  * ROUTE_BUCKET_REF cites the queued bucket for each reserved route
  * Escalation color bands match feedback thresholds (5-15% green,
    >30% red)
  * Pre-router state renders "pre-router" not "0.0%"
  * Cache effectiveness zero-denom edge case + uncached floor
  * Route registered in App.tsx + nav entry
  * No chart library added (banned: recharts / chart.js / victory /
    d3 / echarts / plotly)
  * Chart implementation uses CSS-width divs

Verification:
  * 20/20 source-pin tests pass
  * tsc -b clean
  * vite build clean
  * Screenshot rendered: web/docs/cost-telemetry/states.png
    (all 3 windows + preview HTML for reproducibility)

Refs:
  * rafe-walker/kora-docs 17_cc_bucket_prompts/KR-FE-COST-TELEMETRY-PANEL_per_route_visibility.md
  * PR #161 — cost telemetry snapshot v2 (data source)
  * PR #162 — snapshot wiring + getSnapshot wrapper
  * PR #159 — usePanelView instrumentation
  * feedback-opus-escalation-must-be-earned — escalation band
    rationale

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@rafe-walker rafe-walker merged commit 6af6637 into feature/phase2-upgrades May 24, 2026
@rafe-walker rafe-walker deleted the feat/kora-KR-FE-COST-TELEMETRY-PANEL branch May 24, 2026 02:18
rafe-walker added a commit that referenced this pull request May 24, 2026
…167)

New cockpit page surfaces current phrasebook (bundled OR override) + live regex tester. Three states demonstrated in committed screenshot: matched + $0 reply (snapshot fresh) / unmatched (would fall through to reasoning) / matched-but-would-fall-through (referenced field is 'unknown').

Two new read-only endpoints: GET /api/phrasebook/slack_dm + POST /api/phrasebook/slack_dm/test. Drift guard test pins _PHRASEBOOK_PLACEHOLDER_RE to dm_phrasebook._PLACEHOLDER_RE — FE/BE drift caught at test gate (same pattern as KNOWN_ROUTES pin from #164).

Security pin: unmatched POST does not echo snapshot contents (info-leakage defensive default).

Read-only v1; edit path deferred to KR-FE-PHRASEBOOK-EDITOR + KR-API-PHRASEBOOK-CRUD.

17/17 tests + tsc -b + vite build clean.
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant