Skip to content

fix: add overflow assembly diagnostics#512

Merged
jalehman merged 1 commit into
mainfrom
overflowc-b1896b39-495-overflow-diagnostics
Apr 28, 2026
Merged

fix: add overflow assembly diagnostics#512
jalehman merged 1 commit into
mainfrom
overflowc-b1896b39-495-overflow-diagnostics

Conversation

@jalehman

Copy link
Copy Markdown
Contributor

What

Adds compact, depersonalized overflow diagnostics to LCM assembly debug output for stressed prompt budgets. References #495.

Why

Large assembled prompts need observability that explains token pressure, duplicate context, and largest contributors without dumping private message or summary content.

Changes

  • Add overflow diagnostic payload
  • Track raw vs summary token split
  • Report duplicate clusters
  • Report top token contributors
  • Include recent bootstrap import count

Testing

  • npm test
  • npm run build

@jalehman jalehman merged commit 86f86b7 into main Apr 28, 2026
2 checks passed
jalehman pushed a commit to electricsheephq/lossless-claw-test that referenced this pull request May 18, 2026
…vations

Two bootstrap-side followups to v0.9.3:

* D1: PR Martian-Engineering#510 replaced bulk createMessagesBulk + appendContextMessages
  with N×ingestSingle inside a single withTransaction so the interceptor
  pipeline runs on first import.  Correct for sessions whose transcripts
  contain interceptor-triggering content (oversized files/images/tool-
  results); slow for the common case.  Pre-scan now restores the bulk-
  insert fast path when no message would trigger an interceptor;
  otherwise chunked transactions of K=100.

* F10: PR Martian-Engineering#512's recentBootstrapImportsByConversation Map had no time-
  based expiry — stale "7 imports" tags trailed conversations
  indefinitely (LRU only evicts at 100 conversations).  Add 30-minute
  TTL gated in formatOverflowDiagnosticsForLog so operator log lines
  reflect recent activity only.

Closes Martian-Engineering#568.  Refs Martian-Engineering#489, Martian-Engineering#510, Martian-Engineering#512, Martian-Engineering#495.
jalehman pushed a commit to electricsheephq/lossless-claw-test that referenced this pull request May 18, 2026
…vations

Two bootstrap-side followups to v0.9.3:

* D1: PR Martian-Engineering#510 replaced bulk createMessagesBulk + appendContextMessages
  with N×ingestSingle inside a single withTransaction so the interceptor
  pipeline runs on first import.  Correct for sessions whose transcripts
  contain interceptor-triggering content (oversized files/images/tool-
  results); slow for the common case.  Pre-scan now restores the bulk-
  insert fast path when no message would trigger an interceptor;
  otherwise chunked transactions of K=100.

* F10: PR Martian-Engineering#512's recentBootstrapImportsByConversation Map had no time-
  based expiry — stale "7 imports" tags trailed conversations
  indefinitely (LRU only evicts at 100 conversations).  Add 30-minute
  TTL gated in formatOverflowDiagnosticsForLog so operator log lines
  reflect recent activity only.

Closes Martian-Engineering#568.  Refs Martian-Engineering#489, Martian-Engineering#510, Martian-Engineering#512, Martian-Engineering#495.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant