Skip to content
This repository was archived by the owner on May 26, 2026. It is now read-only.

feat(kora): KR-SLACK-DM-PANEL-FLIP — endpoint reads from JSONL#137

Merged
rafe-walker merged 1 commit into
feature/phase2-upgradesfrom
feat/kora-KR-SLACK-DM-PANEL-FLIP
May 23, 2026
Merged

feat(kora): KR-SLACK-DM-PANEL-FLIP — endpoint reads from JSONL#137
rafe-walker merged 1 commit into
feature/phase2-upgradesfrom
feat/kora-KR-SLACK-DM-PANEL-FLIP

Conversation

@rafe-walker

Copy link
Copy Markdown
Owner

Summary

Flips /api/slack-dm/recent from the hardcoded 4-message stub (PR #120) to a live read of ${KORA_HOME}/slack_dm_log.jsonl. The handler writes inbound entries per slack_dm_handler.py:302-310 (PR #119) and outbound entries per slack_dm_handler.py:775-805 (PR #122 + #131 reasoning meta + #130 caller_actor_kind).

No FE changes needed. The TS SlackDMMessage type already matches the projected shape, and the stub banner auto-disappears on stub:false.

What's in here

  • Endpoint rewrite in kora_cli/web_server.py:
    • Direction discrimination per the actual writer keys: inbound has received_at + handled_status; outbound has sent_at + send_status. (K-DG drift caught: spec mentioned a direction: outbound key, but the writer doesn't include one.)
    • user_id_label derivation: env match → joshua; outbound → kora_bot; else → unknown_user. Raw user_id never reaches the wire.
    • handled_status projection: inbound passes through; outbound mapped to f"sent_{send_status}" so it lines up with the FE SlackDMHandledStatus enum (sent_ok / sent_failed).
    • Newest-first by timestamp descending; ISO-8601 lex order matches chronological for Z-suffixed UTC.
    • ?limit query param (default 50, capped at 200 to bound large-file reads; clamped to ≥ 1).
    • Aggregate counts (total_recent_24h, by_direction_24h, by_status_24h) use the full 24h window, not the limited slice — dashboard headline number reflects activity, not pagination.
    • Defensive parsing: missing file → empty list + stub:false; malformed JSON line / non-dict line / partial-shape entry / blank line → log + skip, other entries still parsed.
    • channel_id_truncated companion field: first 4 + last 4 chars masked (e.g. D012…CDEF); short IDs pass through. Backend-only for now — small FE follow-on bucket KR-SLACK-DM-PANEL-CHANNEL-MASK consumes it.

4-layer security contract preserved from PR #120

  1. user_id_label is a LABEL — raw U... Slack user IDs never reach the wire. Per-field assertion + walk-payload sweep.
  2. channel_id starts with D (DM channel) — backend test asserts. Group/public channel IDs would be a leak; only DMs belong in this panel.
  3. text rendered as PLAIN TEXT by FE — companion FE source-pin against dangerouslySetInnerHTML preserved from PR feat(kora): KR-SLACK-DM-PANEL — conversation view stub #120.
  4. Walk-payload guards for xoxb-/xoxp-/xapp- Slack token shapes and 32-char hex signing-secret shapes.

K-DG drift caught + handled

  • Spec K-DG block (§1) said outbound entries have a "direction": "outbound" key. Actual writer (slack_dm_handler.py:775-805) does NOT include one. Discriminate on sent_at/send_status presence — keys that DO exist on the entry. Tests pin the actual writer shape so a future spec-doc update lands cleanly.

Test plan

Fixture-isolation gotcha + fix worth noting

Initial implementation used set_kora_home_override() (the canonical mechanism per kora_constants.py:82), but the ContextVar leaks across pytest-xdist tests in the same worker process — token-based reset didn't restore cleanly when tests interleaved. Switched to direct monkeypatch.setattr of the get_kora_home symbol in all three module namespaces (kora_constants, kora_cli.config, kora_cli.web_server). The web_server case is critical because the endpoint resolves it via its own module namespace (from kora_cli.config import get_kora_home), not via a fresh import at call time. Worth noting for the next test author hitting this pattern.

Refs

🤖 Generated with Claude Code

Flips /api/slack-dm/recent from the hardcoded 4-message stub (PR
#120) to a live read of ${KORA_HOME}/slack_dm_log.jsonl. The
handler writes inbound entries per slack_dm_handler.py:302-310
(PR #119) and outbound entries per slack_dm_handler.py:775-805
(PR #122 + #131 reasoning meta + #130 caller_actor_kind). No FE
changes needed: TS SlackDMMessage type already matches the
projected shape, stub banner auto-disappears on stub:false.

Backend changes (kora_cli/web_server.py):
  * Replace stub body with JSONL reader + projection function
  * Direction discrimination per ACTUAL writer keys (not the spec
    K-DG note which mentioned a "direction" key — slack_dm_handler
    doesn't write one; we discriminate on received_at/handled_status
    vs sent_at/send_status presence)
  * user_id_label derivation:
      - matches KORA_SLACK_JOSHUA_USER_ID env → "joshua"
      - outbound entry → "kora_bot"
      - else → "unknown_user"
    Raw user_id NEVER reaches the wire. Mirrors the handler's own
    Joshua-check at slack_dm_handler.py:241.
  * handled_status: inbound passes through; outbound mapped to
    f"sent_{send_status}" so the FE's SlackDMHandledStatus enum
    (sent_ok / sent_failed) lines up.
  * Newest-first by timestamp descending; ISO-8601 lex order
    matches chronological for Z-suffixed UTC.
  * ?limit query param, default 50, capped at 200 to bound
    large-file reads. Clamped to >=1 defensively.
  * Aggregate counts (total_recent_24h, by_direction_24h,
    by_status_24h) use the FULL 24h window, not the limited
    slice — dashboard headline number must reflect activity, not
    pagination choice.
  * Defensive parsing: missing file → empty list + stub:false;
    malformed JSON line / non-dict line / partial-shape entry →
    log + skip; other entries still parsed.
  * channel_id_truncated companion field: first 4 + last 4 chars
    masked (e.g. "D012…CDEF"); short IDs pass through. Field is
    backend-only for now — small FE follow-on bucket
    KR-SLACK-DM-PANEL-CHANNEL-MASK consumes it.

4-layer SECURITY contract preserved from PR #120:
  1. user_id_label is a LABEL — raw U... Slack user IDs never
     reach the wire. Per-field assertion + walk-payload sweep.
  2. channel_id starts with "D" (DM channel) — backend test
     asserts. Group/public channel IDs would be a leak; only
     DMs belong in this panel.
  3. text rendered as PLAIN TEXT by FE — companion FE source-pin
     against dangerouslySetInnerHTML preserved from PR #120.
  4. Walk-payload guard for xoxb-/xoxp-/xapp- Slack token shapes
     and 32-char hex signing-secret shapes.

K-DG drift caught + handled:
  * Spec K-DG block said outbound entries have a "direction":
    "outbound" key. Actual writer (slack_dm_handler.py:775-805)
    does NOT include one. Discriminate on sent_at/send_status
    presence — keys that DO exist on the entry.

Tests (tests/kora_cli/test_web_server_slack_dm.py — full rewrite):
  * 33 tests covering: empty/missing file, inbound projection,
    outbound projection (including PR #130 caller_actor_kind +
    PR #131 reasoning meta forward-compat), user_id_label
    resolution (joshua / kora_bot / unknown_user), newest-first
    ordering, ?limit cap, malformed JSON / non-dict / partial-
    shape / blank-line tolerance, channel_id_truncated masking +
    short-ID passthrough, all 4 SECURITY guards (per-field +
    walk-payload), channel_id D-prefix enforcement, FE companion
    pins (no dangerouslySetInnerHTML), aggregate-window-vs-slice
    independence, cron-regression sanity.
  * Full admin-panel regression: 321/321 across 25 suites.
  * Fixture isolation gotcha encountered + fixed: ContextVar-
    based set_kora_home_override leaks across pytest-xdist tests
    in the same worker process; switched to direct
    monkeypatch.setattr of the get_kora_home symbol in all three
    module namespaces (kora_constants, kora_cli.config,
    kora_cli.web_server) since the endpoint resolves via its own
    namespace, not via a fresh import.

Refs: rafe-walker/kora-docs 17_cc_bucket_prompts/KR-SLACK-DM-PANEL-FLIP_stub_to_real.md
  * PR #119 — KR-FEAT-SLACK-DM ST1 (inbound writer)
  * PR #122 — KR-FEAT-SLACK-DM ST2 (outbound writer)
  * PR #131 — KR-FEAT-AI-RESPONSE-LOOP ST2 (reasoning meta fields)
  * PR #130 — KR-MCP-SEND-TOOLS (caller_actor_kind field)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@rafe-walker rafe-walker merged commit d97ab58 into feature/phase2-upgrades May 23, 2026
@rafe-walker rafe-walker deleted the feat/kora-KR-SLACK-DM-PANEL-FLIP branch May 23, 2026 01:13
rafe-walker added a commit that referenced this pull request May 23, 2026
… JSONL (#141)

3 panels flip at once: AGENT-ACTIVITY + REASONING + WEBHOOK-EVENTS now read from kora_audit_log.jsonl.

- NEW kora_cli/audit/jsonl_reader.py (shared helper) + 3 endpoint flips + 4 test files.

2 K-DG drifts caught + handled:
- §2 Flip 1 spec mismatch with actual mcp_tools.py:714-724 writer — handled by setting duration_ms=0, status=ok, using details.result for result_summary.
- §2 Flip 2 nullable cost_rung — handled by using lowercase unknown enum member to preserve PR #132 K-DG pin.

Reasoning grouping: collapses N tool calls sharing caller_session_id into 1 ReasoningCall with tools_used: [...]. Aggregate counts use individual rows (not groups) so headline reflects volume.

Webhook security: source_ip octet-masked at projection edge (audit writer passes RAW peer_ip; endpoint enforces mask). details sub-set to {reason, header_present} — never full audit details. IPv6/dash → defensive fallback.

Fixture-isolation from #137 applied across all 3 test files + reader tests: monkeypatch get_kora_home in all 3 module namespaces.

357/357 admin-panel tests pass across 27 suites.

Follow-on buckets cited: KR-REASONING-PANEL-MODEL-XREF (model/tokens cross-ref from slack_dm log) + KR-MCP-RUNTIME-SURFACE follow-on (extends mcp.tool_called audit with duration_ms + failure-path status).
rafe-walker added a commit that referenced this pull request May 23, 2026
Backend endpoint swap. Reads both email_inbound_log.jsonl (#138) + email_outbound_log.jsonl (#124), merges, projects to existing EmailMessage TS shape. FE auto-flips (stub badge gone).

- kora_cli/web_server.py: -106 stub + 330 LOC live endpoint + 2 projection helpers + status-mapping table.
- NEW tests/kora_cli/test_web_server_email_panel_flip.py: 596 LOC, 32 tests covering projection per direction + merge/sort + limit param + malformed tolerance + security walk-payload with message_id carve-out.

4-layer security + message_id carve-out: walk-payload sweep excludes only message_id + in_reply_to fields from email-regex check (RFC 5322 <id@operator-domain> is legitimate). test_no_email_addresses_outside_message_id_carve_out pins the boundary.

Status taxonomy collapse: handler filtered_paused/filtered_stopped → FE dropped_paused; filtered_non_joshua → filtered_non_allowlist. JSONL stays canonical; operator panel sees coarser FE union.

Fixture-isolation discipline applied: caught real cross-test bleed where original PR #121 fixture only patched upstream namespace; added 3-namespace get_kora_home monkeypatch per CC#2 #137 lesson.

Spoofing semantic flip: spoofing_warning = NOT entry.spoofing_check_skipped. Test pins behavior when future bucket adds real envelope-based detection.

47 new + 185 cross-bucket regression. Stub badge auto-disappears.
rafe-walker added a commit that referenced this pull request May 23, 2026
…g sources (#145)

Last major stub-flip in the cockpit. Replaces /api/alerts/current stub with live aggregator pulling from 5 sources (cost holder + OperationalStateHolder + HealthRollup + audit JSONL + heartbeat probe snapshots).

- NEW kora_cli/alerts/aggregator.py (390 LOC): Alert dataclass + 7 rule helpers + compute_active_alerts
- NEW kora_cli/alerts/__init__.py: public surface re-export
- kora_cli/web_server.py: -92 stub + 46 live aggregator call
- 28 aggregator tests + updated 15 preserved shape/security pins

§1 K-DG drift caught + corrected in module docstring:
- Spec get_operational_state_holder() → actual agent.operational_state_holder.get_holder()
- Spec get_health_holder() → actual agent.health_rollup_holder.get_health_rollup_holder()
- Cost .active_rung() method, OpState .current @Property, HealthRollup bare-field names — all verified.

Forward-compat documented: capability_denied_24h rules matcher targets details.result == capability_denied which todays audit emit doesnt produce (cap-gate at mcp.py:181 returns BEFORE audit emit at mcp_tools.py:714). Rule emits zero alerts in current state; activates automatically when audit-on-denial bucket lands. Test test_capability_denied_today_no_alert_since_audit_doesnt_emit_denials pins this expected behavior.

Fail-soft proven by 4 scenarios: cost holder raises / op holder None / audit reader raises / probe accessor raises — each test confirms OTHER rules still emit + aggregator never propagates exceptions. Defense-in-depth outer try/except catches helper-bypass failures.

3-layer security carry-forward preserved + extended: walk-payload sweep against diverse-triggered alerts catches PII (email/Slack-ID) + token shapes (Anthropic/Slack/hex/Bearer).

CC#2 #137 fixture-isolation applied: 3-namespace get_kora_home monkeypatch + reset all 5 aggregator sources to baseline no-alert state.

28 new + 15 preserved + 123/123 cross-bucket regression.
rafe-walker added a commit that referenced this pull request May 24, 2026
…t zero LLM cost (#157)

Per Council R3 Lock R3-4 item #3. Enables routine status queries
("burn this week?", "any alerts?", "what's open?") to be answered
at $0 LLM cost — engine reads the pre-computed snapshot instead
of tool-calling. Foundational infrastructure for probe-audit work
+ reasoning-engine routing-layer short-circuit (separate bucket
KR-SNAPSHOT-INTO-ROUTING wires the consumer side).

# New module: kora_cli/snapshot/

  * ``state_snapshot.py`` — pure projection from live read
    accessors (operational_state_holder + cost_state_holder +
    alerts aggregator + heartbeat probe snapshots). Per-source
    collectors are independently fail-soft: a single source
    failure degrades only that section to ``"unknown"`` (or null
    where shape requires).
  * ``__init__.py`` — public surface (compute / write / read /
    is_fresh / snapshot_path / run_snapshot_cycle / SCHEMA_VERSION)
    + ``get_snapshot_for_routing()`` convenience the future
    reasoning-engine routing-layer bucket consumes.

Snapshot file: ``${KORA_HOME}/cache/daemon_snapshot.json`` (atomic
write via existing ``utils.atomic_replace``, the same pattern
cron/jobs.py uses for jobs.json).

# Schema v1 — populated vs degraded

| Section | Field | Source | v1 disposition |
|---|---|---|---|
| operational_state | primary | get_holder().current.primary_state | ✅ populated |
| operational_state | paused | derived from primary | ✅ populated |
| operational_state | pause_reason | degradation_reasons[0].value when paused | ✅ populated (null when empty set) |
| alerts | active_count | len(compute_active_alerts()) | ✅ populated |
| alerts | by_severity | rollup of alerts | ✅ populated |
| alerts | by_category | rollup of alerts | ✅ populated |
| cost_ladder | current_tier | get_cost_holder().active_rung().name | ✅ populated |
| cost_ladder | monthly_budget_pct_used | get_cost_holder().current_pct_used() * 100 | ✅ populated |
| cost_ladder | model_default | dynamic per-call downshift (no holder field) | ⚠️ "unknown" v1 |
| service_health | {vercel,sentry,doppler,supabase,fly} | current_service_snapshots()[name].status | ✅ populated (per-probe degrade to "unknown" if absent) |
| tasks | open_count, in_progress_count | substrate Sea_Tickets read | ⏸️ "unknown" v1 (deferred per spec §4 — MCP call at 5-min cadence flagged ASK; follow-on bucket can wire cached substrate read) |

# Listener wiring

New ``kora_cli/listeners/snapshot_listener.py`` registers via
``register_daemon_listener("snapshot", factory)`` + the periodic
task ``snapshot.compute`` via ``register_periodic_task`` from the
heartbeat scheduler. Cadence default 300s (5 min);
``KORA_SNAPSHOT_INTERVAL_SEC`` env override.

Spec §2(b) says "extend cron/jobs.py OR new kora_cli/snapshot/__
init__.py"; picked the heartbeat-scheduler path (matches what
MCP-CONSUMPTION health-check, alert-notifier, email IMAP poll,
heartbeat probes all do — cheap in-process compute). cron/jobs.py
is the agent-driven cron for external worker processes;
overkill for a pure-Python state projection.

# Web endpoint

``GET /api/snapshot`` returns the snapshot dict verbatim when
fresh on disk; returns ``{"error": "no_snapshot", "stale": true}``
when missing or stale (>10 min). Cockpit + future routing layer
consume this without paying per-source fan-out.

# Read-only contract preserved

This module is a read-only consumer of every source holder. No
mutation of agent/operational_state_holder, agent/cost_state_
holder, kora_cli/heartbeat_probes, or alerts aggregator. The
snapshot is a projection, not a mirror — consumers wanting
authoritative state still read the source-of-truth accessors;
the snapshot is the cheap path for routine status queries.

# Tests

51 new tests pass:
  * 36 state_snapshot (shape + per-section degradation + full-
    degrade resilience + atomic write + read freshness gate +
    is_snapshot_fresh boundary tests + run_snapshot_cycle end-
    to-end + fail-soft on compute/write failure + get_snapshot_
    for_routing convenience)
  * 8 listener (registration in LISTENER_REGISTRY + periodic-task
    registration + cadence env resolution + lifecycle log lines)
  * 4 web endpoint (3-namespace get_kora_home fixture-isolation
    per CC#2 #137; fresh / missing / stale paths + shape pin)

437/437 cross-bucket regression (snapshot + alerts + all
test_listeners). Ruff clean.

Co-authored-by: CC#1 Kora Runtime <kora-pm@stormhavenenterprises.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant