feat(sessions): structured compaction summary with monotonic boundary#597
Merged
Conversation
6 tasks
12451a5 to
795037c
Compare
Rework session compaction based on source-level reads of four production
LLM harnesses (Aider f09d706, sst/opencode 2719063, cline/cline a0faf7c,
Claude Code docs). Three adopted ideas:
- Cline's 9-section structured summary template with anti-drift rule
(Task Evolution section requires direct quotes from user messages) to
replace the free-form bullet list that was losing grounding across
successive compactions
- OpenCode's truncate-only-at-user-message-boundaries rule, enforced in
ExtractiveSessionReducer via backward walk — guarantees tool call/
result pair integrity which the openspec netclaw-session "pair
integrity" requirement already mandated but the code did not enforce
- Self-session-id disambiguation in the observer system prompt so the
observer can mark foreign session IDs from tool-call history (the
failure mode that triggered this rework — ArdyBot lost its own
session ID after a compaction that followed a turn investigating
another session)
Summary messages now use a `[session-summary session:{id}]` header.
This is the only recognition marker: consumers (observer, reducer, UI)
walk history looking for the header prefix rather than consulting a
separately-persisted index. The observer prompt tells the model to
preserve any prior summary block verbatim on successive compactions —
belt-and-suspenders defense against summary-over-summary decay paired
with the reducer's backward walk that won't chop a prior summary.
Journal compatibility: no new fields on `SessionCompacted` or
`SessionSnapshot`. Old-format compactions (User-role `[observations
from earlier in this session]` messages) still deserialize cleanly and
are folded into the new format by the next compaction via
`WrapObservations`.
Refs Aaronontheweb/netclaw#595 (session CWD tracking + project
identity re-read) and #596 (authoritative session CWD for path-taking
tools) which are tracked follow-ups on milestone 0.12.
Research and architectural discussion captured in OpenSpec change
artifacts at openspec/changes/compaction-rework/. Working-context
grounding (durable RecentFiles / OpenGoals / ProgressMarkers on
SessionState) ships as a separate OpenSpec change in PR2.
Tests: 890/890 pass. Slopwatch clean.
795037c to
0f3f24c
Compare
Aaronontheweb
added a commit
that referenced
this pull request
Apr 11, 2026
Follow-up to #597: sync the compaction-rework delta spec into the canonical netclaw-session spec and move the change directory to the archive. Consolidates the stale 2026-03-25 refactor-llm-session-actor delta fragment for "Conversation compaction" into the canonical requirement — that fragment had been sitting un-merged since the earlier refactor change was archived, and this rewrite supersedes its content. "Decoupled immutable session state" fragment from the same archive is left alone (unrelated to compaction work).
Aaronontheweb
added a commit
that referenced
this pull request
Apr 11, 2026
Stacks on the compaction-rework change (merged as #597). Adds a `WorkingContext` field to `SessionState` that tracks recent files the agent has read/written/edited, survives compaction, actor recovery, and daemon restart without depending on the observer LLM to reconstruct it. This introduces a new tier of session state that sits parallel to existing tiers: - Skills: stateless how-to reference (already exists) - Memory: cross-session facts (already exists, via memory store) - WorkingContext: in-session mutable state (new) - Conversation: turn history (already exists) `WorkingContext` carries a single field: `RecentFiles` — a bounded ring buffer of 10 file paths, most-recent-first, deduped on repeat access. Updated automatically by `LlmSessionActor` when a path-taking tool (file_read, file_write, file_edit, etc.) completes. The path is extracted from the tool call's `ArgumentsJson` via `WorkingContextUpdater.TryExtractFilePath` which probes well-known JSON field names (`path`, `file_path`, `filePath`, `file`, `filename`, `fileName`) — no tool-name allowlist, so first-party tools and MCP filesystem tools participate automatically. Security: `AddRecentFile` rejects paths containing control characters (newline, carriage return, null byte). Without this, an attacker- crafted path containing a literal newline would break out of the `recent_files:` section in the `[working-context]` block and inject arbitrary content into the LLM's system prompt. Rejected paths are silently dropped at the earliest point in the ingestion pipeline. Injection: `WorkingContext` is emitted as a `[working-context]` block via `InjectDynamicContextLayers` on every LLM call when non-empty, adjacent to the existing `[session]` block. Suppressed entirely when `WorkingContext.IsEmpty` so the model doesn't see a barren header. Explicitly NOT in this change: - OpenGoals / ProgressMarkers fields and the observer-output parser that would populate them — deferred per "no hypothetical future requirements" rule. Can be added in a follow-up change when the parser lands. - CurrentWorkingDirectory / ActiveProjectPath — GitHub #595. - Authoritative CWD for path-taking tools — GitHub #596. Journal compatibility: `WorkingContext` is a new optional field on `SessionSnapshot` (ProtoMember 6) and `SessionCompacted` (ProtoMember 7). ProtoMember 5/6 respectively are reserved holes — formerly `CompactionBoundaryIndex`, removed before compaction-rework merged. Old snapshots and events without the field deserialize to `WorkingContext.Empty` via `SessionState.FromSnapshot` / `Apply(SessionCompacted)`. Precedent: Cline's `TaskState.currentFocusChainChecklist` in `src/core/task/focus-chain/` is the closest comparable pattern — a persistent markdown todo list tracked on disk, re-attached to every LLM call, explicitly protected from re-summarization. `WorkingContext` is the Netclaw equivalent, stored on `SessionState` rather than a standalone file. Tests: 933/933 pass (+11 new). Slopwatch clean (SW003 on the JsonException catch is explicitly ignored via inline directive with justification — LLM-generated tool args can be malformed and "no path tracked" is the correct handling). Refs Aaronontheweb/netclaw#595, Aaronontheweb/netclaw#596
Aaronontheweb
added a commit
that referenced
this pull request
Apr 11, 2026
Stacks on the compaction-rework change (merged as #597). Adds a `WorkingContext` field to `SessionState` that tracks recent files the agent has read/written/edited, survives compaction, actor recovery, and daemon restart without depending on the observer LLM to reconstruct it. This introduces a new tier of session state that sits parallel to existing tiers: - Skills: stateless how-to reference (already exists) - Memory: cross-session facts (already exists, via memory store) - WorkingContext: in-session mutable state (new) - Conversation: turn history (already exists) `WorkingContext` carries a single field: `RecentFiles` — a bounded ring buffer of 10 file paths, most-recent-first, deduped on repeat access. Updated automatically by `LlmSessionActor` when a path-taking tool (file_read, file_write, file_edit, etc.) completes. The path is extracted from the tool call's `ArgumentsJson` via `WorkingContextUpdater.TryExtractFilePath` which probes well-known JSON field names (`path`, `file_path`, `filePath`, `file`, `filename`, `fileName`) — no tool-name allowlist, so first-party tools and MCP filesystem tools participate automatically. Security: `AddRecentFile` rejects paths containing control characters (newline, carriage return, null byte). Without this, an attacker- crafted path containing a literal newline would break out of the `recent_files:` section in the `[working-context]` block and inject arbitrary content into the LLM's system prompt. Rejected paths are silently dropped at the earliest point in the ingestion pipeline. Injection: `WorkingContext` is emitted as a `[working-context]` block via `InjectDynamicContextLayers` on every LLM call when non-empty, adjacent to the existing `[session]` block. Suppressed entirely when `WorkingContext.IsEmpty` so the model doesn't see a barren header. Explicitly NOT in this change: - OpenGoals / ProgressMarkers fields and the observer-output parser that would populate them — deferred per "no hypothetical future requirements" rule. Can be added in a follow-up change when the parser lands. - CurrentWorkingDirectory / ActiveProjectPath — GitHub #595. - Authoritative CWD for path-taking tools — GitHub #596. Journal compatibility: `WorkingContext` is a new optional field on `SessionSnapshot` (ProtoMember 6) and `SessionCompacted` (ProtoMember 7). ProtoMember 5/6 respectively are reserved holes — formerly `CompactionBoundaryIndex`, removed before compaction-rework merged. Old snapshots and events without the field deserialize to `WorkingContext.Empty` via `SessionState.FromSnapshot` / `Apply(SessionCompacted)`. Precedent: Cline's `TaskState.currentFocusChainChecklist` in `src/core/task/focus-chain/` is the closest comparable pattern — a persistent markdown todo list tracked on disk, re-attached to every LLM call, explicitly protected from re-summarization. `WorkingContext` is the Netclaw equivalent, stored on `SessionState` rather than a standalone file. Tests: 933/933 pass (+11 new). Slopwatch clean (SW003 on the JsonException catch is explicitly ignored via inline directive with justification — LLM-generated tool args can be malformed and "no path tracked" is the correct handling). Refs Aaronontheweb/netclaw#595, Aaronontheweb/netclaw#596
Aaronontheweb
added a commit
that referenced
this pull request
Apr 11, 2026
Stacks on the compaction-rework change (merged as #597). Adds a `WorkingContext` field to `SessionState` that tracks recent files the agent has read/written/edited, survives compaction, actor recovery, and daemon restart without depending on the observer LLM to reconstruct it. This introduces a new tier of session state that sits parallel to existing tiers: - Skills: stateless how-to reference (already exists) - Memory: cross-session facts (already exists, via memory store) - WorkingContext: in-session mutable state (new) - Conversation: turn history (already exists) `WorkingContext` carries a single field: `RecentFiles` — a bounded ring buffer of 10 file paths, most-recent-first, deduped on repeat access. Updated automatically by `LlmSessionActor` when a path-taking tool (file_read, file_write, file_edit, etc.) completes. The path is extracted from the tool call's `ArgumentsJson` via `WorkingContextUpdater.TryExtractFilePath` which probes well-known JSON field names (`path`, `file_path`, `filePath`, `file`, `filename`, `fileName`) — no tool-name allowlist, so first-party tools and MCP filesystem tools participate automatically. Security: `AddRecentFile` rejects paths containing control characters (newline, carriage return, null byte). Without this, an attacker- crafted path containing a literal newline would break out of the `recent_files:` section in the `[working-context]` block and inject arbitrary content into the LLM's system prompt. Rejected paths are silently dropped at the earliest point in the ingestion pipeline. Injection: `WorkingContext` is emitted as a `[working-context]` block via `InjectDynamicContextLayers` on every LLM call when non-empty, adjacent to the existing `[session]` block. Suppressed entirely when `WorkingContext.IsEmpty` so the model doesn't see a barren header. Explicitly NOT in this change: - OpenGoals / ProgressMarkers fields and the observer-output parser that would populate them — deferred per "no hypothetical future requirements" rule. Can be added in a follow-up change when the parser lands. - CurrentWorkingDirectory / ActiveProjectPath — GitHub #595. - Authoritative CWD for path-taking tools — GitHub #596. Journal compatibility: `WorkingContext` is a new optional field on `SessionSnapshot` (ProtoMember 6) and `SessionCompacted` (ProtoMember 7). ProtoMember 5/6 respectively are reserved holes — formerly `CompactionBoundaryIndex`, removed before compaction-rework merged. Old snapshots and events without the field deserialize to `WorkingContext.Empty` via `SessionState.FromSnapshot` / `Apply(SessionCompacted)`. Precedent: Cline's `TaskState.currentFocusChainChecklist` in `src/core/task/focus-chain/` is the closest comparable pattern — a persistent markdown todo list tracked on disk, re-attached to every LLM call, explicitly protected from re-summarization. `WorkingContext` is the Netclaw equivalent, stored on `SessionState` rather than a standalone file. Tests: 933/933 pass (+11 new). Slopwatch clean (SW003 on the JsonException catch is explicitly ignored via inline directive with justification — LLM-generated tool args can be malformed and "no path tracked" is the correct handling). Refs Aaronontheweb/netclaw#595, Aaronontheweb/netclaw#596
Aaronontheweb
added a commit
that referenced
this pull request
Apr 11, 2026
* feat(sessions): durable WorkingContext grounding Stacks on the compaction-rework change (merged as #597). Adds a `WorkingContext` field to `SessionState` that tracks recent files the agent has read/written/edited, survives compaction, actor recovery, and daemon restart without depending on the observer LLM to reconstruct it. This introduces a new tier of session state that sits parallel to existing tiers: - Skills: stateless how-to reference (already exists) - Memory: cross-session facts (already exists, via memory store) - WorkingContext: in-session mutable state (new) - Conversation: turn history (already exists) `WorkingContext` carries a single field: `RecentFiles` — a bounded ring buffer of 10 file paths, most-recent-first, deduped on repeat access. Updated automatically by `LlmSessionActor` when a path-taking tool (file_read, file_write, file_edit, etc.) completes. The path is extracted from the tool call's `ArgumentsJson` via `WorkingContextUpdater.TryExtractFilePath` which probes well-known JSON field names (`path`, `file_path`, `filePath`, `file`, `filename`, `fileName`) — no tool-name allowlist, so first-party tools and MCP filesystem tools participate automatically. Security: `AddRecentFile` rejects paths containing control characters (newline, carriage return, null byte). Without this, an attacker- crafted path containing a literal newline would break out of the `recent_files:` section in the `[working-context]` block and inject arbitrary content into the LLM's system prompt. Rejected paths are silently dropped at the earliest point in the ingestion pipeline. Injection: `WorkingContext` is emitted as a `[working-context]` block via `InjectDynamicContextLayers` on every LLM call when non-empty, adjacent to the existing `[session]` block. Suppressed entirely when `WorkingContext.IsEmpty` so the model doesn't see a barren header. Explicitly NOT in this change: - OpenGoals / ProgressMarkers fields and the observer-output parser that would populate them — deferred per "no hypothetical future requirements" rule. Can be added in a follow-up change when the parser lands. - CurrentWorkingDirectory / ActiveProjectPath — GitHub #595. - Authoritative CWD for path-taking tools — GitHub #596. Journal compatibility: `WorkingContext` is a new optional field on `SessionSnapshot` (ProtoMember 6) and `SessionCompacted` (ProtoMember 7). ProtoMember 5/6 respectively are reserved holes — formerly `CompactionBoundaryIndex`, removed before compaction-rework merged. Old snapshots and events without the field deserialize to `WorkingContext.Empty` via `SessionState.FromSnapshot` / `Apply(SessionCompacted)`. Precedent: Cline's `TaskState.currentFocusChainChecklist` in `src/core/task/focus-chain/` is the closest comparable pattern — a persistent markdown todo list tracked on disk, re-attached to every LLM call, explicitly protected from re-summarization. `WorkingContext` is the Netclaw equivalent, stored on `SessionState` rather than a standalone file. Tests: 933/933 pass (+11 new). Slopwatch clean (SW003 on the JsonException catch is explicitly ignored via inline directive with justification — LLM-generated tool args can be malformed and "no path tracked" is the correct handling). Refs Aaronontheweb/netclaw#595, Aaronontheweb/netclaw#596 * feat(sessions): observability + compaction-pipeline integration test for WorkingContext WorkingContextUpdater now takes an optional ILoggingAdapter and emits debug logs for both tracked files and schema-drift (tool call had string-valued arguments but none of the probed field names matched). Operators hunting "why isn't this file in [working-context]" get a concrete signal in debug logs. Adds an end-to-end integration test that drives a file_read tool call through the real compaction pipeline (3 turns: populate → compact → probe) and verifies the [working-context] block still contains src/Rect.cs after compaction. This exercises WorkingContextUpdater, the SessionCompacted event path, and InjectDynamicContextLayers together — not just direct Apply() on an in-memory state. Refs #600 (sub-agent gap discovered during this work).
This was referenced Apr 12, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Rework session compaction based on source-level reads of four production LLM harnesses:
sst/opencode@2719063Aider-AI/aider@f09d706cline/cline@a0faf7ccode.claude.com/docs/en/*Three adopted ideas:
ExtractiveSessionReducervia a backward walk. Guarantees tool call/result pair integrity, which the openspecnetclaw-session"pair integrity" requirement already mandated but the code did not actually enforce.Also adds
SessionState.CompactionBoundaryIndexas monotonic metadata pointing at the most recent[session-summary session:{id}]marker. Informational for MVP; the infrastructure supports future checkpoint rollback and multi-tier compaction without a data migration.Summary messages now use a
[session-summary session:{id}]header. The observer prompt tells the model to preserve any prior summary block verbatim on successive compactions — belt-and-suspenders defense against summary-over-summary decay paired with the reducer's backward walk that won't chop a prior summary.Journal compatibility:
SessionCompactedevents written before this change deserialize cleanly. The newCompactionBoundaryIndexfield is optional andApply(SessionCompacted)defaults to the post-system offset when absent.OpenSpec
This PR ships the full OpenSpec change at
openspec/changes/compaction-rework/(proposal, design,netclaw-sessiondelta spec, tasks).Related
WorkingContextgrounding — PR2)Test plan
dotnet build src/Netclaw.Actors.Tests/Netclaw.Actors.Tests.csproj— zero warningsdotnet test src/Netclaw.Actors.Tests/Netclaw.Actors.Tests.csproj— 890/890 passdotnet slopwatch analyze— cleanObservationPromptBuilderTests.cs: 9-section layout, self-session-id, preserve-prior-summary rule, direct-quotes-in-task-evolutionExtractiveSessionReducerTests.cs: backward walk to user boundary, skip system nudges, keep-zero edge case, degenerate fallbackCompactionIntegrationTests.cscases pass (session-id header, observer self-session-id, second-compaction buffer drain, session recovery after compaction+kill)