fix: preserve unpersisted volatile live input in LCM assembly#688
Conversation
|
Upstream RFC for the proper contract fix: openclaw/openclaw#82137 To frame what this PR is and isn't: This is a Track 1 hot fix. The current LCM That's a real production fix and we should ship it, but the underlying issue is a contract gap, not an LCM bug. OpenClaw already has the structured metadata internally:
The RFC proposes plumbing these three (existing) signals through to // Subagent / task-completion notification — currently we detect via
// "[Internal task completion event]" text match.
if (params.internalEvents?.[i]?.some(e =>
e.type === AGENT_INTERNAL_EVENT_TYPE_TASK_COMPLETION
)) {
// structured, robust against marker renames
}Same for When the RFC lands and a version of OpenClaw ships the signals, we'll feature-detect and migrate. The marker constants in this PR stay as a fallback until the floor OpenClaw version in the wild has the new fields, then we delete them. Tracking that migration as Track 3 in the three-track plan referenced in the PR description. |
45f76a5 to
5e8abf7
Compare
When OpenClaw's LCM context engine assembles context from the durable DB frontier, messages that were never persisted (inter-session subagent announcements, retry/overflow prompts with suppressPromptPersistence) are absent from the DB. The assembled output therefore omits them entirely, causing the model to miss live input events like subagent completions. This patch adds a reconciliation step after DB assembly: detect volatile live input in params.messages that is not represented in the assembled output, and append it within the token budget. Key design choices: - Volatile live inputs are never covered by summary substring matches. A summary containing similar text captures a *past* turn; the current volatile event is a distinct occurrence the model must see explicitly. - Only exact assembled-message matches count as coverage for volatile inputs. This prevents stale summaries from consuming live events. - Occurrence-level bipartite maximum matching (DFS augmenting-path) deduplicates volatile inputs against assembled context. - Live input takes priority: if appending volatile input exceeds the token budget, evict from the front of assembled DB context first. - Tool-result/tool-call pairs are kept intact during budget trimming; protected fresh-tail messages and exact live anchors are never evicted. - Tool names that are null/undefined/""/"unknown" are normalized to equivalent in coverage signatures, fixing anchor mismatches when the assembler fills missing tool names with "unknown". Fixes: subagent completion announcements lost from model context Related: openclaw/openclaw#80760, openclaw/openclaw#81164
5e8abf7 to
5965151
Compare
|
Thank you! |
Problem
When
contextEngine.assemble()rebuilds context from the DBcontext_itemstable, messages that were never persisted to DB are absent from the assembled output. This includes:suppressPromptPersistence=trueforinputProvenance.kind === "inter_session")suppressPromptPersistenceOnRetry)The DB frontier lags behind
params.messages, so the model never sees the current turn's volatile event — e.g., a subagent completion announcement that should trigger a response.Root Cause
LCM's assembler rebuilds from DB only. OpenClaw's
shouldSuppressAgentPromptPersistence()marks certain messages as non-persistable, so they never entercontext_items. The assembled output has no way to include them.Fix
After DB assembly, detect volatile live input in
params.messagesthat is not represented in the assembled output and append it within the token budget.Key design choices
Volatile inputs are never covered by summary substring matches. A summary containing similar text captures a past turn; the current volatile event is a distinct occurrence the model must see explicitly. Only exact assembled-message matches count as coverage for volatile inputs.
Occurrence-level bipartite maximum matching (DFS augmenting-path) for deduplication. Two-layer slot allocation: exact-match slots first, then normalized-overlap generic slots.
Live input takes priority. If appending volatile input exceeds the token budget, evict from the front of assembled DB context. Protected fresh-tail messages and exact live anchors are never evicted.
Tool-result/tool-call pair integrity.
expandProtectedToolPairIndexes()ensures protected toolResults and their paired assistant tool-call messages are both preserved during budget trimming.Tool name normalization.
normalizeToolNameForCoverage()treatsnull/undefined/""/"unknown"as equivalent in coverage signatures, fixing anchor mismatches when the assembler fills missing tool names with"unknown".System volatile normalization. System-role volatile messages are projected to
userrepresentation before coverage/anchor/append logic.Fresh-tail hash protection. Assembler provides per-message hashes for fresh-tail messages; engine protects these alongside exact live anchors during volatile trimming.
Files Changed
src/engine.ts— core reconciliation logic (appendUncoveredVolatileLiveInputsWithinBudget, coverage matching, budget eviction)src/assembler.ts— fresh-tail message hash export for protectiontest/engine.test.ts— 931 tests (278 engine-specific), including regression tests for volatile coverage, budget eviction, tool-pair integrity, and tool-name normalization.changeset/assemble-live-tail.md— patch bump.gitignore— addednode_modulesRelated
assemble()contract) and Track 3 (upstream PR for live-input fields) are separate.