Skip to content

fix: reset bootstrap state on session file rotation#366

Merged
jalehman merged 2 commits into
Martian-Engineering:mainfrom
electricsheephq:codex/fix-bootstrap-rotation
Apr 10, 2026
Merged

fix: reset bootstrap state on session file rotation#366
jalehman merged 2 commits into
Martian-Engineering:mainfrom
electricsheephq:codex/fix-bootstrap-rotation

Conversation

@100yenadmin

Copy link
Copy Markdown
Collaborator

Summary

This fixes the stale-bootstrap loop described in #233.

When OpenClaw rotates a session file to a new UUID.jsonl, the current bootstrap path still treats the existing LCM conversation as authoritative even if its checkpoint points at the old file. With no overlap between the new file and the old DB tail, bootstrap falls through to already bootstrapped, new transcript data never enters LCM, and compaction never gets a chance to run.

What changed

  • detect session-file path rotation immediately after loading bootstrap state
  • purge the stale conversation-scoped bootstrap data for the rotated transcript
  • reset bootstrapped_at so the existing first-import path can seed from the new file
  • clear related summaries, context items, compaction telemetry, bootstrap state, large files, and FTS rows for that conversation before re-import
  • add a regression test proving that rotated sessions discard stale summaries and import the new transcript cleanly

Why this is safe

A rotated session file is a transcript identity change. Reusing summaries and bootstrap checkpoints from the previous file is what creates the dead loop. Resetting the conversation-scoped LCM state keeps the conversation record itself but drops stale derived data so bootstrap can rebuild from the current transcript.

Tests

  • pnpm exec vitest run test/engine.test.ts test/bootstrap-flood-regression.test.ts

Adds focused coverage for:

  • session-file rotation purging stale messages and summaries
  • bootstrapping cleanly from the replacement transcript
  • existing append-only/import-cap bootstrap regressions remaining green

Closes #233.

@100yenadmin 100yenadmin marked this pull request as ready for review April 10, 2026 15:16
Copilot AI review requested due to automatic review settings April 10, 2026 15:16

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR fixes a bootstrap dead-loop that can occur when a session transcript file is rotated to a new path (e.g., UUID.jsonl changes) by detecting the rotation early and resetting conversation-scoped derived state so bootstrap can import from the new transcript.

Changes:

  • Detect session file path rotation immediately after loading bootstrap state during LcmContextEngine.bootstrap().
  • Purge conversation-scoped derived data (messages, summaries, context items, compaction telemetry, bootstrap state, large files, and optional FTS rows) and reset bootstrapped_at to force a clean re-bootstrap.
  • Add a regression test ensuring stale summaries/context are discarded and the new transcript imports cleanly after rotation.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File Description
src/engine.ts Adds rotation detection and a purge/reset routine to ensure bootstrap re-seeds from the new session file.
test/engine.test.ts Adds a regression test covering session-file rotation and verifying stale derived data is removed.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Add a patch changeset for PR Martian-Engineering#366 and extend the bootstrap rotation regression to cover the stable sessionKey path where the runtime session UUID changes. This keeps the existing rotation fix intact while proving it also works for the production-shaped conversation reuse case that survives via sessionKey across session-file rotation.

Regeneration-Prompt: |\n  Update the contributor branch for PR Martian-Engineering#366 after review. The branch already fixes bootstrap dead loops when session files rotate; add the missing patch changeset required by repo policy and add one focused regression test for the stable sessionKey case where a conversation is reused across a new runtime sessionId and a new session file. Keep the code change additive, run the targeted bootstrap tests, and avoid unrelated edits.
@jalehman jalehman merged commit f4177ec into Martian-Engineering:main Apr 10, 2026
1 check passed
@github-actions github-actions Bot mentioned this pull request Apr 10, 2026
jalehman added a commit that referenced this pull request Apr 11, 2026
Remove the automatic bootstrap purge path that deleted persisted conversation data whenever the session file path changed. Rotation now only invalidates the stored bootstrap checkpoint and reconciles forward from the existing conversation state, preserving messages, summaries, lineage, and context items.

Update the rotation regression tests to assert that existing conversation state survives file changes and that only the missing tail messages are imported after rotation.

Regeneration-Prompt: |
  Lossless Claw should never delete persisted conversation history just because
  the backing session JSONL file path changes. We already traced a production
  data-loss incident to the rotation handling added in PR #366: on file-path
  mismatch, bootstrap hard-deleted messages, summaries, context items, lineage,
  and telemetry, then rebuilt from the new file. The earlier fix kept the full
  rotated transcript during reseed, but that still left an automatic destructive
  purge in the code, which violates the product contract that lossless is
  lossless unless the user explicitly invokes a destructive command.

  Remove the purge-on-rotation behavior entirely while keeping rotation
  detection/logging. Treat a rotated file as another transcript source for the
  same conversation: invalidate the old bootstrap checkpoint and reconcile from
  the persisted conversation state so only genuinely missing tail messages are
  imported. Preserve existing summaries and context items. Update regression
  tests to prove that file rotation does not wipe conversation history and does
  not reapply the first-bootstrap budget cap to an existing conversation.
jalehman added a commit that referenced this pull request Apr 11, 2026
* fix: replay full transcript after session rotation

Session-file rotation was purging the existing conversation and then reseeding it through the normal first-bootstrap cap. That left only a small suffix of the rotated transcript in LCM, which in turn made assembly fall back to live context while incremental compaction and manual compaction still evaluated the truncated persisted state.

Treat rotation reseeds as full transcript replacements instead of first-time bootstraps so LCM keeps coverage parity with the live session. Add a regression test that uses a tiny bootstrapMaxTokens value and verifies a rotated transcript is replayed in full.

Regeneration-Prompt: |
  User reported that a live lossless-claw session showed enormous actual context usage, but incremental compaction and /compact both believed the conversation was comfortably under target. Logs showed the session repeatedly falling back to live context while compaction decisions reported rawTokensOutsideTail=0.

  Trace the mismatch through bootstrap, assemble, and compaction. The key production clue is a session-file rotation event followed by an initial import that only reloaded a tiny suffix of a much larger rotated transcript. Preserve the intended first-bootstrap cap for genuinely new conversations, but do not reapply that cap when a known conversation is being reseeded after session-file rotation. In that case, replay the full rotated transcript so persisted LCM coverage matches the live session again. Add a focused regression test that would fail if rotation reseeds were trimmed by bootstrapMaxTokens.

* fix: stop purging history on session rotation

Remove the automatic bootstrap purge path that deleted persisted conversation data whenever the session file path changed. Rotation now only invalidates the stored bootstrap checkpoint and reconciles forward from the existing conversation state, preserving messages, summaries, lineage, and context items.

Update the rotation regression tests to assert that existing conversation state survives file changes and that only the missing tail messages are imported after rotation.

Regeneration-Prompt: |
  Lossless Claw should never delete persisted conversation history just because
  the backing session JSONL file path changes. We already traced a production
  data-loss incident to the rotation handling added in PR #366: on file-path
  mismatch, bootstrap hard-deleted messages, summaries, context items, lineage,
  and telemetry, then rebuilt from the new file. The earlier fix kept the full
  rotated transcript during reseed, but that still left an automatic destructive
  purge in the code, which violates the product contract that lossless is
  lossless unless the user explicitly invokes a destructive command.

  Remove the purge-on-rotation behavior entirely while keeping rotation
  detection/logging. Treat a rotated file as another transcript source for the
  same conversation: invalidate the old bootstrap checkpoint and reconcile from
  the persisted conversation state so only genuinely missing tail messages are
  imported. Preserve existing summaries and context items. Update regression
  tests to prove that file rotation does not wipe conversation history and does
  not reapply the first-bootstrap budget cap to an existing conversation.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Session file rotation breaks bootstrap — compaction never triggers (PR #190)

3 participants