Merged
Conversation
Leiter's distillation pipeline had three weaknesses that compounded: 1. The pre-processing stripped too much context — tool_use blocks were silently dropped, so the distilling agent couldn't see what tools were used when the user corrected something. A user saying "no, don't do that" after an [assistant tool]: Edit(src/main.rs) line is far more informative than just seeing the correction in isolation. 2. The writing guidelines didn't help the agent decide *whether* to record an observation. Every correction got recorded equally, leading to one-off debugging decisions polluting the soul alongside genuine lasting preferences. 3. The user never saw what distillation changed. The sub-agent would silently modify the soul and the user had no visibility into what was learned, making it hard to catch bad entries or confirm good ones. Changes: - Richer pre-processing: assistant messages with tool_use blocks now emit [assistant tool]: Name(param) summary lines. Key parameter is chosen heuristically (file_path > command > pattern). Tool results remain dropped — the tool name from the assistant side is sufficient. - Recording judgment guidelines: new subsection in SOUL_WRITING_GUIDELINES teaching the distiller to prefer patterns over one-offs, use tool context to understand corrections, skip ephemeral decisions, and capture implicit positive signals. - Distillation feedback: sub-agent now ends with a summary of what it changed, and the main agent relays it to the user after mark-distilled. Works identically for manual and auto-distill (foreground vs background sub-agent). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
bde4172 to
c270b47
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

Leiter's distillation pipeline had three weaknesses that compounded:
silently dropped, so the distilling agent couldn't see what tools
were used when the user corrected something. A user saying "no, don't
do that" after an [assistant tool]: Edit(src/main.rs) line is far
more informative than just seeing the correction in isolation. This
behavior was introduced earlier when we added log pre-processing
to cut down on token use use - but that was primarly from tool output,
which we are still filtering out.
record an observation. Every correction got recorded equally, leading
to one-off debugging decisions polluting the soul alongside genuine
lasting preferences.
silently modify the soul and the user had no visibility into what
was learned, making it hard to catch bad entries or confirm good ones.
Changes:
emit [assistant tool]: Name(param) summary lines. Key parameter is
chosen heuristically (file_path > command > pattern). Tool results
remain dropped — the tool name from the assistant side is sufficient.
teaching the distiller to prefer patterns over one-offs, use tool
context to understand corrections, skip ephemeral decisions, and
capture implicit positive signals.
changed, and the main agent relays it to the user after mark-distilled.
Works identically for manual and auto-distill (foreground vs background
sub-agent).
changelog: include