Summary

• Problem: Bootstrap files (SOUL.md, AGENTS.md, MEMORY.md, etc.) are re-read from disk on every agent turn. Memory flush itself writes new content to memory/YYYY-MM-DD.md. If these files are modified they will bust the cache.
• Why it matters: Each prefix cache miss forces full prompt re-processing on every subsequent turn. Local models feel this significantly. Additionally, a lot of the time we modify these files, we're writing out lessons learned from a long context life. Not a great time to invalidate the cache.
• What changed: New bootstrap-cache.ts module snapshots bootstrap files on first load per session key. All subsequent calls within the same session — including compaction's resolveBootstrapContextForRun() and the memory flush sub-run — return the snapshot instead of re-reading disk.
• What did NOT change: Memory flush still writes to disk normally (available to next session). Post-compaction context (readPostCompactionContext) still reads AGENTS.md fresh from disk — intentional, it's re-orienting the agent after compaction, and we're already busting the cache pretty hard. No config changes, no new hooks, no changes to the hook system.

This change was mostly planned by Opus and executed by Sonnet, and then the human went and told them to delete 2/3 of what they wrote, so, at this point, it's her problem, not theirs.

Change Type

• Bug fix
• Feature
• Refactor
• Docs
• Security hardening
• Chore/infra

Scope

• Gateway / orchestration
• Skills / tool execution
• Auth / tokens
• Memory / storage
• Integrations
• API / contracts
• UI / DX
• CI/CD / infra

Linked Issue/PR

User-visible / Behavior Changes

Memory written during a session (via memory flush or tool calls to memory/*.md) will not appear in the system prompt until the next session. Neither will anything written to SOUL, AGENTS, etc.

Additionally, models all seem to think this stuff doesn't update live. For example, just in this section alone I had to delete a line the model wrote about how it "was already true in most cases." It's uncanny.

Anyway, if you think about it, right now (before this change), when we ask a model to write something in one of these files, we're then killing that context and replacing it with one where it always thought that. It must be very weird to read a conversation about something you know the ending of already.

Security Impact

• New permissions/capabilities? No
• Secrets/tokens handling changed? No
• New/changed network calls? No
• Command/tool execution surface changed? No
• Data access scope changed? No

Repro + Verification

Locally tested before/after and validated that caching behavior is much nicer when working with memory documents.

Environment

• OS: Linux (Debian)
• Runtime/container: Node.js
• Model/provider: Anthropic (claude-* models with prefix caching)
• Integration/channel: any
• Relevant config: default

Steps

1. Work in a long session.
2. Ask the agent to modify its memory.
3. Send another message.

Expected

• System prompt same in this session for next turn

Actual (before this PR)

• System prompt changed (updated MEMORY) so cache is invalidated

Evidence

• Perf numbers (if relevant)
  100% cache hit rate after mod of memory, well worth it especially on local.

Human Verification

• Verified scenarios: cache serves stable content across multiple calls with same session key; different session keys are isolated; long haul test doesn't leak memory all over the floor
• Edge cases checked: restart of gateway with pending change causes cache invalidation (expected result; not a regression from today's behavior anyway)
• What you did not verify: Massively parallel long sessions. But I have done a couple shorter parallel ones.

Compatibility / Migration

• Backward compatible? Yes
• Config/env changes? No
• Migration needed? No

Failure Recovery

• How to disable/revert this change quickly: revert bootstrap-files.ts to use loadWorkspaceBootstrapFiles() directly (one line)
• Files/config to restore: src/agents/bootstrap-files.ts
• Known bad symptoms to watch for: The initial onboarding experience might get weird.

Risks and Mitigations

• Risk: Cache entries accumulate in long-running processes with many sessions
  • Mitigation: None currently. A previous version had a TTL system but this was discarded in favor of simplicity. If the sessions leave memory then so do the cache entries, and we delete the cache when we reset a session or delete it too.

Greptile Summary

Implements session-scoped caching for bootstrap files (AGENTS.md, MEMORY.md, SOUL.md, etc.) to preserve prompt cache when these files are modified during a session.

Key changes:

• New bootstrap-cache.ts module provides simple Map-based caching keyed by sessionKey
• resolveBootstrapFilesForRun in bootstrap-files.ts uses cached files when sessionKey is provided
• Cache is cleared in sessions.ts during session reset/delete via clearBootstrapSnapshot
• Post-compaction context (readPostCompactionContext) continues to read AGENTS.md fresh from disk, as mentioned in PR description

Impact:

• Memory writes during a session (e.g., via memory flush) will not appear in the system prompt until the next session
• Maintains 100% cache hit rate after memory modifications within a session
• Memory accumulates in the cache Map for the lifetime of the gateway process, cleared only on session reset/delete

Confidence Score: 4/5

• Safe to merge with careful monitoring of cache behavior in production
• Implementation is clean and straightforward with good test coverage. The cache invalidation is properly wired to session lifecycle events (reset/delete). However, the cache uses only sessionKey without workspaceDir, which relies on the assumption that a session key is always bound to a single workspace - this was an intentional simplification per commit de20b01. The PR explicitly documents the behavior change where memory modifications won't appear until next session. One minor concern is long-running processes with many sessions could accumulate cache entries, but the PR acknowledges this and notes that cache entries are cleared when sessions are deleted or reset.
• No files require special attention - implementation is clean and well-tested

_{Last reviewed commit: aa82281}

_{(4/5) You can add custom instructions or style guidelines for the agent here!}