feat: run LCM observed extraction in maintenance (7/10)#540
feat: run LCM observed extraction in maintenance (7/10)#540100yenadmin wants to merge 45 commits into
Conversation
There was a problem hiding this comment.
Pull request overview
Wires deterministic observed-work and event extraction into LcmContextEngine.maintain() so all extraction/mutation happens only on the explicit maintenance/write path (not via read tools), and introduces/extends supporting storage + tool surfaces for temporal rollups, observed-work density, and event search.
Changes:
- Run rollup building + observed-work/event extraction during
maintain(), with logging and failure isolation/pending-rebuild preservation. - Add SQLite migrations + stores for rollups, observed work, event observations, and inert task-bridge suggestions.
- Register/read tools for temporal recaps (
lcm_recent), work density (lcm_work_density), event search (lcm_event_search), and operator-gated rollup inspection (lcm_rollup_debug), with updated docs/tests.
Reviewed changes
Copilot reviewed 36 out of 36 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| test/task-bridge-suggestion-store.test.ts | Adds coverage for inert task-bridge suggestion storage semantics. |
| test/plugin-prompt-hook.test.ts | Updates prompt-hook assertions for new recall-order guidance. |
| test/observed-work-store.test.ts | Adds observed-work + event observation + tool behavior tests (redaction, cursoring, bounded reads). |
| test/engine.test.ts | Adds regression test ensuring extraction runs only during maintain() (read-path remains non-mutating). |
| src/tools/lcm-work-density-tool.ts | Implements lcm_work_density tool with period filtering, redaction, and bounded scope behavior. |
| src/tools/lcm-rollup-debug-tool.ts | Adds operator-gated rollup inspection tool with optional source inclusion. |
| src/tools/lcm-event-search-tool.ts | Implements lcm_event_search tool for deterministic event observations (sources hidden by default). |
| src/timezone-windows.ts | Provides shared timezone/date-window helpers used by period resolution. |
| src/store/task-bridge-suggestion-store.ts | Adds inert suggestion ledger store with validation and review-state updates. |
| src/store/rollup-store.ts | Adds rollup persistence/state APIs and leaf-summary range reads for rollup building. |
| src/store/observed-work-store.ts | Adds observed-work persistence, density queries, provenance, and processing state handling. |
| src/store/index.ts | Re-exports EventObservationStore types for broader consumption. |
| src/store/event-observation-store.ts | Adds event observation persistence and query APIs with source redaction. |
| src/rollup-builder.ts | Adds deterministic daily + weekly/monthly rollup building and fingerprinting logic. |
| src/plugin/index.ts | Registers new tools and extends system prompt guidance; gates rollup debug tool by config. |
| src/observed-work-extractor.ts | Implements deterministic extraction from leaf summaries with cursoring + optional event upserts. |
| src/engine.ts | Instantiates new stores/builders and runs rollup + extraction work during maintain(). |
| src/db/migration.ts | Adds idempotent migrations for rollups, observed work, task-bridge suggestions, and event observations. |
| src/db/config.ts | Adds rollupDebugEnabled config/env surface for gating rollup debug tool. |
| specs/lcm-temporal-memory-plan.md | Documents temporal-memory/rollup plan and tool-selection guidance. |
| specs/lcm-task-bridge-option-c-experimental.md | Documents experimental task-bridge approach and inert-store intent. |
| specs/lcm-observed-work-density-option-b.md | Documents observed-work density model and tool contract. |
| skills/lossless-claw/references/recall-tools.md | Updates recall-tool reference with availability-gated temporal guidance. |
| skills/lossless-claw/SKILL.md | Updates skill description and workflow guidance to include lcm_recent when available. |
| openclaw.plugin.json | Adds config schema/help for rollupDebugEnabled. |
| docs/configuration.md | Documents LCM_ROLLUP_DEBUG_ENABLED / rollupDebugEnabled. |
| README.md | Documents LCM_ROLLUP_DEBUG_ENABLED environment variable. |
| .changeset/lcm-temporal-skill-docs.md | Changeset for skill-doc guidance updates. |
| .changeset/lcm-temporal-rollups.md | Changeset for temporal rollups + tools. |
| .changeset/lcm-task-bridge-suggestions.md | Changeset for inert task-bridge suggestion storage. |
| .changeset/lcm-observed-work-extraction.md | Changeset for observed-work extraction addition. |
| .changeset/lcm-observed-work-density.md | Changeset for observed-work density model + tool. |
| .changeset/lcm-extraction-maintenance.md | Changeset for maintenance-path extraction wiring. |
| .changeset/lcm-event-observations.md | Changeset for event observations + tool. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
Review-thread triage at current head
Local focused gates passed: |
Final adversarial pass - merge order 7/10, after #538Score: mergeability 9.4/10, functionality 9.0/10, description 9.5/10. Functionality audit: Works as production maintenance wiring for observed-work/event extraction. Read tools do not mutate; maintenance writes are gated by config; retry failures preserve pending state. The final hardening moved event observation writes into the per-summary savepoint so failed observed-work writes do not leak event rows. Scenario coverage: Makes the observed-work/event layers useful at runtime through explicit maintenance, while preserving read/write-path separation. Final state: Residual caveat: Writes happen through maintenance only, not user read tools. |
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 40 out of 40 changed files in this pull request and generated 7 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
Final May 1 hardening/status update:
The PR description top note has been refreshed with current head SHA, visible-vs-logical LOC breakdown, scope, caveats, and validation status. LCM remains an evidence layer: recaps, observed work, events, episodes, and suggestions are entry points, not proof or task authority. |
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 41 out of 41 changed files in this pull request and generated 3 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| rollups: includeSources | ||
| ? rollups | ||
| : rollups.map((rollup) => { | ||
| const { source_summary_ids: omitted, ...redacted } = rollup; | ||
| void omitted; | ||
| return redacted; | ||
| }), |
| const limit = typeof p.limit === "number" ? Math.trunc(p.limit) : 5; | ||
| const detailLevel = typeof p.detailLevel === "number" ? Math.trunc(p.detailLevel) : 1; | ||
| const topic = typeof p.topic === "string" && p.topic.trim() ? p.topic.trim() : undefined; | ||
| const minConfidence = typeof p.minConfidence === "number" ? p.minConfidence : undefined; | ||
| const store = lcm.getObservedWorkStore(); |
| if (query) { | ||
| const likeQuery = `%${escapeLikePattern(query)}%`; | ||
| where.push( | ||
| "(lower(coalesce(query_key, '')) = ? OR lower(title) LIKE ? ESCAPE '\\' OR lower(coalesce(description, '')) LIKE ? ESCAPE '\\')" |
PR #540: Maintenance wiring for extraction and event processing
Maintainer Quick Read
800354d63f3b2cc1c2504d4b261bf45a94e87ab3Why Maintainers Should Care
This connects extraction/event processing to maintenance so LCM can update observed-work and event observations outside user-facing reads. The key product point is trust: asking a question does not secretly mutate memory.
Stack Map
How It Works
What Ships In This PR
What This PR Intentionally Does Not Do
How An Agent Uses It
Engineering Boundaries
lcm_describe,lcm_expand,lcm_expand_query, raw messages, GitHub, logs, tickets, or the relevant authority.includeSourcesor debug affordance.Primary Review Files
src/engine.tssrc/maintenance.tssrc/observed-work-extractor.tssrc/event-observation-extractor.tstest/observed-work-store.test.tsSize And Review Surface
These are intentionally large LCM changes, so the body separates the GitHub-visible diff from the intended logical review slice. Later PRs target
main, which makes the GitHub Files tab include parent-stack code; reviewers should use the logical parent listed above when judging the slice.main#538Logical slice breakdown:
Review Concerns Already Folded Into The Code
julianday()correctness and now use matching expression indexes for indexed range/order plans.Validation Evidence
npm test -- --run test/observed-work-store.test.ts test/rollup-store-builder.test.ts test/task-bridge-suggestion-store.test.tsnpm testnpm run buildgit diff --checktestcheck: successSuggested Maintainer Review Path
mainwhile the series is logically stacked.