Skip to content

feat: add SQLite-aware bootstrap frontier#646

Open
100yenadmin wants to merge 1 commit intoMartian-Engineering:mainfrom
100yenadmin:feat/lcm-645-sqlite-frontier
Open

feat: add SQLite-aware bootstrap frontier#646
100yenadmin wants to merge 1 commit intoMartian-Engineering:mainfrom
100yenadmin:feat/lcm-645-sqlite-frontier

Conversation

@100yenadmin
Copy link
Copy Markdown
Contributor

@100yenadmin 100yenadmin commented May 9, 2026 •
edited
Loading

Summary

This starts the LCM-side SQLite migration by making bootstrap frontier state source-aware and SQLite-capable, while keeping JSONL compatibility as a fallback.

The immediate goal is not to solve the full Lossless Claw migration in one PR. The goal here is narrower:

  • stop hard-depending on file mtime / size / offset for the canonical bootstrap frontier
  • allow LCM to bootstrap and resume from an OpenClaw SQLite transcript cursor when the newer session-store seam is available
  • preserve current JSONL behavior on older OpenClaw runtimes

This is the first implementation slice for #645 and part of the vNext umbrella in #641.

Why this is better

OpenClaw is moving toward SQLite-backed session/transcript state. LCM's current bootstrap checkpoint model is still file-era:

  • session_file_path
  • last_seen_size
  • last_seen_mtime_ms
  • last_processed_offset

That works for append-only JSONL recovery, but it does not map cleanly onto:

  • same-session transcript rewrites
  • full transcript replacement
  • runtimes where the transcript file is no longer the canonical operational source

This PR gives LCM a real SQLite replay frontier without forcing the rest of the migration into the same slice.

What changed

Core bootstrap frontier

  • adds SQLite-aware bootstrap metadata to conversation_bootstrap_state
  • stores source kind/identity plus SQLite frontier fields alongside the legacy JSONL fields
  • migrates existing rows forward as jsonl source records

Engine replay path

  • adds an optional SQLite bootstrap path in LcmContextEngine.bootstrap(...)
  • uses OpenClaw SQLite frontier + delta reads when the host exposes them
  • avoids stat(sessionFile) as the first truth source on SQLite-capable hosts
  • keeps the existing JSONL fast paths untouched as the compatibility fallback

Runtime adapter

  • wires an optional adapter in src/plugin/index.ts
  • dynamically loads openclaw/plugin-sdk/session-store-runtime when available
  • consumes the #79904-style frontier/delta seam without hard-breaking older OpenClaw installs

Regression coverage

  • adds engine coverage for:
    • fileless SQLite bootstrap
    • SQLite cursor append-resume
  • adds flood-regression coverage for:
    • SQLite reset replay against repeated-content history
  • updates migration coverage for the new bootstrap-state schema

Non-goals

This PR does not attempt to solve:

It is intentionally the narrow frontier/replay slice.

Validation

cd /Volumes/LEXAR/repos/lossless-claw-645-sqlite-frontier
npm test -- test/engine.test.ts test/bootstrap-flood-regression.test.ts test/migration.test.ts
npm run build

Results on this branch:

  • 250/250 targeted tests passed
  • build passed

Stack placement

This PR is the first downstream Lossless Claw implementation slice that consumes the OpenClaw SQLite migration work.

Related stack:

Why this is separate:

  • frontier correctness is the narrowest and safest migration step
  • identity, message-parts, and maintenance remapping stay gated until the upstream seam contracts settle

Related

This was referenced May 9, 2026
Open
Open
Open
Open
Open
@100yenadmin
Copy link
Copy Markdown
Contributor Author

100yenadmin commented May 9, 2026

Architecture fit for this slice:

flowchart TD
    A["OpenClaw session-store runtime seam"] --> B["frontier / delta when available"]
    B --> C["LCM bootstrap engine"]
    C --> D["source-aware bootstrap state"]
    D --> E["resume from SQLite cursor"]
    A --> F["JSONL fallback on older hosts"]
Loading

Why this slice exists:

  • LCM's old frontier model is file-era and cannot be the long-term source of truth once OpenClaw moves to SQLite
  • the first migration step is to make frontier state source-aware and able to resume from OpenClaw's canonical runtime seam
  • keeping JSONL fallback avoids breaking older OpenClaw hosts while the upstream seam rolls out

Why it was chosen before identity or message-part migration:

  • frontier correctness is the narrowest and most foundational dependency
  • family/segment identity and message_parts fidelity are easier to reason about after replay is SQLite-native

Stack placement:

  • LCM umbrella: #641
  • child issue: #645
  • upstream dependency: openclaw/openclaw#79904
  • later dependent slices: #642, #643, #644

Lossless Claw connection:

  • this is the first real code move from JSONL-era bootstrap to SQLite-native ingest
  • it preserves the companion model: OpenClaw remains canonical operational storage, and LCM remains the richer downstream lossless store

This was referenced May 9, 2026
Open
Open
Open
Open
@jetd1 jetd1 mentioned this pull request May 10, 2026
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@100yenadmin