fix: preserve LCM history across session rotation#396
Merged
Conversation
Session-file rotation was purging the existing conversation and then reseeding it through the normal first-bootstrap cap. That left only a small suffix of the rotated transcript in LCM, which in turn made assembly fall back to live context while incremental compaction and manual compaction still evaluated the truncated persisted state. Treat rotation reseeds as full transcript replacements instead of first-time bootstraps so LCM keeps coverage parity with the live session. Add a regression test that uses a tiny bootstrapMaxTokens value and verifies a rotated transcript is replayed in full. Regeneration-Prompt: | User reported that a live lossless-claw session showed enormous actual context usage, but incremental compaction and /compact both believed the conversation was comfortably under target. Logs showed the session repeatedly falling back to live context while compaction decisions reported rawTokensOutsideTail=0. Trace the mismatch through bootstrap, assemble, and compaction. The key production clue is a session-file rotation event followed by an initial import that only reloaded a tiny suffix of a much larger rotated transcript. Preserve the intended first-bootstrap cap for genuinely new conversations, but do not reapply that cap when a known conversation is being reseeded after session-file rotation. In that case, replay the full rotated transcript so persisted LCM coverage matches the live session again. Add a focused regression test that would fail if rotation reseeds were trimmed by bootstrapMaxTokens.
Remove the automatic bootstrap purge path that deleted persisted conversation data whenever the session file path changed. Rotation now only invalidates the stored bootstrap checkpoint and reconciles forward from the existing conversation state, preserving messages, summaries, lineage, and context items. Update the rotation regression tests to assert that existing conversation state survives file changes and that only the missing tail messages are imported after rotation. Regeneration-Prompt: | Lossless Claw should never delete persisted conversation history just because the backing session JSONL file path changes. We already traced a production data-loss incident to the rotation handling added in PR #366: on file-path mismatch, bootstrap hard-deleted messages, summaries, context items, lineage, and telemetry, then rebuilt from the new file. The earlier fix kept the full rotated transcript during reseed, but that still left an automatic destructive purge in the code, which violates the product contract that lossless is lossless unless the user explicitly invokes a destructive command. Remove the purge-on-rotation behavior entirely while keeping rotation detection/logging. Treat a rotated file as another transcript source for the same conversation: invalidate the old bootstrap checkpoint and reconcile from the persisted conversation state so only genuinely missing tail messages are imported. Preserve existing summaries and context items. Update regression tests to prove that file rotation does not wipe conversation history and does not reapply the first-bootstrap budget cap to an existing conversation.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
This PR fixes two regressions in LCM bootstrap handling when a session JSONL file rotates. It preserves the full rotated transcript for existing conversations and removes the automatic purge path so session-file changes no longer delete persisted history.
Why
Production logs and DB inspection showed that session-file rotation was deleting messages and summaries from stable conversations, which broke incremental compaction, confused
/compact, and violated the lossless contract. Existing data surgery is separate, but future rotations need to stop causing data loss.Changes
Testing
npx vitest run test/engine.test.ts -t "rotation|bootstrapMaxTokens|reconciles missing tail messages|conversation already has messages"npx vitest run test/engine.test.ts