fix(compression): pre-pass v2 — avoid blind LLM summarization for stale skill views by dolphin-creator · Pull Request #44166 · NousResearch/hermes-agent

dolphin-creator · 2026-06-11T09:20:27Z

Context

Complement to PR #32562 (P0/P1 Ghost Skill mitigation).

Current bug: LLM compression triggers as soon as the context exceeds the threshold, even when the primary cause is the presence of multiple skill dumps (skill_view) of thousands of chars that are no longer useful. The LLM summarizer is invoked unnecessarily (token cost + latency) and tends to dilute the [SKILL_PRUNED] markers by paraphrasing them, which breaks the P1 rule (Skill Safety Rule) and causes hallucinations (Ghost Skill).

What this PR does:
Instead of dumping everything into the LLM summarizer, we introduce a heuristic Pre-pass v2 that distinguishes useful skills from stale ones before calculating the token budget.

Solution

1. Pre-pass v2 — The main fix

Smart heuristic: Before Phase 1 (pruning tool results), the scan identifies single-use vs reused skills.
- Skill called 1x = prunable → replaced by [SKILL_PRUNED]
- Skill called 2+ times or in the last 5 messages = protected
Early return: If this pruning alone is enough to bring tokens below the threshold, LLM compression is avoided entirely. No unnecessary LLM call, no latency, no wasted tokens.
Cost: 0 tokens (pure heuristic analysis, no LLM call)

2. Summary P2 — Defense in depth

If despite the pre-pass, LLM compression is triggered (very long session), P2 ensures that [SKILL_PRUNED] markers survive the pass through the summarizer:

Extract pruned skill names before the LLM call
Directive in the template to preserve markers verbatim
Post-LLM re-injection if the summarizer paraphrased the markers

Impact

Performance: Avoids the LLM summarizer call in a large number of sessions where skill dumps were the only cause of exceeding the threshold.
Stability: Keeps [SKILL_PRUNED] markers in the final summary, preserving the activation of the P1 rule.
Zero regression: Does not change the API, only the internal compression logic.

Fixes #32106

…ompression Complements PR NousResearch#32562 (P0/P1 Ghost Skill mitigation). Pre-pass v2: - New _prune_stale_skill_views() runs BEFORE _prune_old_tool_results - Heuristic: single-use skills pruned, reused/recent skills protected - Early return if pruning brings tokens below threshold (saves LLM call) Summary P2: - Extract [SKILL_PRUNED] markers before _serialize_for_summary() - Add '## Pruned Skills' section to summary template with verbatim directive - Post-LLM: re-inject markers if summarizer paraphrased them away - Fixes NousResearch#32106: LLM summary was diluting [SKILL_PRUNED] into vague prose

liuhao1024 · 2026-06-11T09:46:51Z

Review: P2 regex never matches v2 pre-pass placeholders

The P2 [SKILL_PRUNED] extraction regex targets a format the v2 pre-pass doesn't produce:

P2 regex (context_compressor.py ~line 1584):

r"\[SKILL_PRUNED:.*?reload with skill_view\(name='([^']+)'\)"

v2 pre-pass placeholder (_prune_stale_skill_views, line ~161):

f"[SKILL_PRUNED: content lost in compression; reload with skill_view before relying on it]"

The placeholder says reload with skill_view before relying on it but the regex expects reload with skill_view(name='X') — they will never match. As a result:

_pruned_skill_names is always empty → the LLM summary prompt never receives a ## Pruned Skills section.
The re-injection fallback (if _pruned_skill_names) also never fires, so the LLM output loses the marker entirely.

The pruned content does get a summary placeholder ([skill_view] name=X ... [SKILL_PRUNED: ...]), but after summarization, the LLM may paraphrase it away since it receives no instruction to preserve it.

Fix: Either update the regex to match the v2 format, or change the v2 placeholder to include skill_view(name='{skill_name}') so the regex matches. The latter also makes the summary output more actionable for the agent (it can call skill_view with the exact name).

Also: the SKILL_PRUNED in cont guard in is a substring check — if any non-skill tool result happens to contain the literal text [SKILL_PRUNED], it would be incorrectly skipped from pruning. Consider prefix-matching (cont.startswith('[SKILL_PRUNED]') or cont.startswith('[skill_view]')) instead.

…_view(name) The P1 Skill Safety Rule only triggers on [SKILL_PRUNED]. Checking skill_view(name='...') in the LLM summary is insufficient — the LLM can mention a skill name without the canonical marker, causing the re-injection to be skipped while P1 remains inactive. Use the simpler and more correct check: '[SKILL_PRUNED]' not in summary. Fixes NousResearch#32106.

Even if a skill is loaded only once (count==1), if the user mentions its name in recent messages, we keep it protected instead of pruning. This prevents false positives where a single-use skill is actually the focus of the current task (e.g. user says 'use the doc-builder skill'). Cost: zero — simple string matching on recent user messages. Fixes NousResearch#32106.

…mpactions After successive compressions, the same skill may appear as [SKILL_PRUNED] multiple times in head/tail/summary. Without guidance, the LLM could redundantly reload the same skill multiple times. Add an explicit note in the summary: one reload is sufficient. Fixes NousResearch#32106.

…ompaction After multiple compactions, the same skill may appear as [SKILL_PRUNED] multiple times. Without guidance, the LLM could redundantly reload the same skill on every turn, seeing the old markers as 'unresolved'. Add: - P1-DEDUP: System rule to ignore remaining markers after one reload - P2-Dedup note: Summary reminder that markers are historical artifacts Fixes NousResearch#32106.

…anagement Allows listing currently loaded skills and manually pruning specific ones to free context space. Complements the automatic pre-pass v2 pruning. Usage via Telegram: - 'Liste les skills chargés' - 'Prune le skill hermes-architecture'

The session_search tool returns a JSON string, not a dict. Added json.loads() to parse the result before extracting skill_view calls. Now correctly shows loaded skills in session.

@liuhao1024

…view(name='X') The P2 extraction regex expects skill_view(name='X') but the v2 pre-pass placeholder used 'reload with skill_view before relying on it'. This mismatch meant _pruned_skill_names was always empty, so the P2 defense (pre-LLM extraction + post-LLM re-injection) never fired. Align both Phase 1 and pre-pass v2 placeholders to use the callable skill_view(name='X') format so the regex matches and the agent can reload the pruned skill directly. Addresses review comment from @liuhao1024 on PR NousResearch#44166.

dolphin-creator · 2026-06-11T13:15:13Z

@liuhao1024 Thanks for catching this! You're absolutely right — the P2 regex expected skill_view(name='X') but the v2 pre-pass placeholder used a different format.

Fixed in commit 8d86b2b:

Updated both Phase 1 and pre-pass v2 placeholders to include skill_view(name='{skill_name}') so the regex matches
This also makes the placeholder actionable for the agent — it can call skill_view with the exact skill name directly

Regarding the SKILL_PRUNED in cont guard at line 880: it's already protected by cont.startswith("[skill_view]") which runs first. The SKILL_PRUNED check is a secondary catch for edge cases where content was already pruned by an earlier pass. Both conditions use OR logic so they're complementary.

The fix ensures the full P2 defense chain works: extraction → template directive → re-injection.

dolphin-creator added 2 commits June 10, 2026 16:39

fix: ghost skill P0/P1 mitigation - [SKILL_PRUNED] marker + safety rule

536b0b1

alt-glitch added type/bug Something isn't working P3 Low — cosmetic, nice to have comp/agent Core agent loop, run_agent.py, prompt builder labels Jun 11, 2026

dolphin-creator changed the title ~~fix: pre-pass v2 + summary P2 — preserve [SKILL_PRUNED] through LLM compression~~ fix(compression): pre-pass v2 — avoid blind LLM summarization for stale skill views Jun 11, 2026

dolphin-creator added 6 commits June 11, 2026 12:13

fix: session_skills_list now parses session_search JSON result correctly

70a2576

The session_search tool returns a JSON string, not a dict. Added json.loads() to parse the result before extracting skill_view calls. Now correctly shows loaded skills in session.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix(compression): pre-pass v2 — avoid blind LLM summarization for stale skill views#44166

fix(compression): pre-pass v2 — avoid blind LLM summarization for stale skill views#44166
dolphin-creator wants to merge 9 commits into
NousResearch:mainfrom
dolphin-creator:fix-ghost-skill-prepass-v2-and-summary-p2

dolphin-creator commented Jun 11, 2026 •

edited

Loading

Uh oh!

liuhao1024 commented Jun 11, 2026

Uh oh!

dolphin-creator commented Jun 11, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

dolphin-creator commented Jun 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Context

Solution

1. Pre-pass v2 — The main fix

2. Summary P2 — Defense in depth

Impact

Uh oh!

liuhao1024 commented Jun 11, 2026

Review: P2 regex never matches v2 pre-pass placeholders

Uh oh!

dolphin-creator commented Jun 11, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

dolphin-creator commented Jun 11, 2026 •

edited

Loading