Skip to content

fix: context_compressor.py - Ghost Skill P0/P1 mitigation (#32106)#32562

Open
dolphin-creator wants to merge 2 commits into
NousResearch:mainfrom
dolphin-creator:fix-ghost-skill-p0-p1
Open

fix: context_compressor.py - Ghost Skill P0/P1 mitigation (#32106)#32562
dolphin-creator wants to merge 2 commits into
NousResearch:mainfrom
dolphin-creator:fix-ghost-skill-p0-p1

Conversation

@dolphin-creator

@dolphin-creator dolphin-creator commented May 26, 2026

Copy link
Copy Markdown

Contexte

Linked issue: #32106

Bug 'Ghost Skill' : Lors de la compression contextuelle, les skills chargés via skill_view() sont tronqués et réduits à des placeholders.
L'LLM interprète ces placeholders comme des skills disponibles, alors que leur contenu est perdu, ce qui génère des réponses hallucinées. La règle SOUL.md est contournée car le modèle croit le skill chargé.

Solution (Quick-Win P0 + P1)

Ce fix injecte deux couches de sécurité minimalistes pour briser la boucle d'hallucination sans modifier l'architecture de compression.

P0 — Marqueur explicite [SKILL_PRUNED]

Fichier : agent/context_compressor.py
Ajout d'un suffixe explicite lors du pruning d'un skill compressé :

[SKILL_PRUNED: content lost in compression; reload with skill_view before relying on it]

Cela force le modèle à reconnaître que le skill est injoignable.

P1 — Règle système d'invalidation

Fichier : agent/prompt_builder.py
Injection d'une règle ## Skill Safety Rule dans SKILLS_GUIDANCE :

  1. UNAVAILABLE : Si un placeholder contient [SKILL_PRUNED], le skill est considéré comme inaccessible.
  2. RELOAD : Le modèle doit rappeler skill_view(name='...') avant toute action dépendant du skill.
  3. WAIT : En cas de chargement en cours, le modèle doit attendre la confirmation de succès.

Impact

  • Zéro régression : Ne modifie que les strings de fallback et la guidance système.
  • Coût négligeable : ~10 tokens ajoutés par skill pruné.
  • Efficacité : Brise immédiatement les boucles 'Ghost Skill' observées en production.

@alt-glitch alt-glitch added type/bug Something isn't working P2 Medium — degraded but workaround exists comp/agent Core agent loop, run_agent.py, prompt builder labels May 26, 2026
@alt-glitch

alt-glitch commented May 26, 2026

Copy link
Copy Markdown
Collaborator

Competing fix with #32375 for issue #32106 — both add [SKILL_PRUNED] marker to compressed skill_view results. #32375 is more targeted (only marks skill_view outputs, preserves skills_list/skill_manage as metadata-only). This PR also injects a ## Skill Safety Rule system prompt addition in prompt_builder.py. Recommend coordinating with #32375.

@dolphin-creator

dolphin-creator commented May 26, 2026

Copy link
Copy Markdown
Author

After reviewing both PRs side by side, I believe they're complementary rather than competing.

Identical on context_compressor.py — both separate skill_view from skills_list/skill_manage and add the [SKILL_PRUNED] marker.

What each brings the other doesn't have:

Where this PR sits in the issue roadmap (#32106):

  • ✅ P0 — Explicit [SKILL_PRUNED] marker → covered here
  • ✅ P1 — System prompt invalidation rule → covered here
  • 🔜 P2 — Runtime skill-state tracking → follow-up PR
  • 🔜 P3 — Task-aware skill recovery → follow-up PR
  • 🔜 P4 — Execution loop invalidation → follow-up PR

Proposal: I'd like to incorporate the tests from #32375 into #32562, then #32375 could be closed as superseded. The combined PR would deliver P0 + P1 with proper test coverage in a single PR.

Happy to coordinate — @LeonSGP43 if you're comfortable with this approach, I'll add your tests and credit you in the commit.

@dolphin-creator dolphin-creator mentioned this pull request May 26, 2026
Open
@dolphin-creator
test(ghost-skill): add SKILL_PRUNED marker tests + Skill Safety Rule …
b7aaf2a 
…coverage (NousResearch#32106)

- TestToolResultSummaries: skill_view emits [SKILL_PRUNED], skills_list/skill_manage remain metadata-only
- TestGuidanceConstants: SKILLS_GUIDANCE includes ## Skill Safety Rule with reload instruction
- Credits: test patterns from LeonSGP43 (PR NousResearch#32375), adapted for merged PR
@dolphin-creator

dolphin-creator commented May 29, 2026

Copy link
Copy Markdown
Author

Tests added (commit b7aaf2a) — this PR now covers both P0 and P1 with test coverage:

  • TestToolResultSummaries — skill_view emits [SKILL_PRUNED], skills_list/skill_manage remain metadata-only (3 tests)
  • TestGuidanceConstants — SKILLS_GUIDANCE includes ## Skill Safety Rule with reload instruction (1 test)

Test patterns adapted from @LeonSGP43 PR #32375 with credit in the commit. All 4 pass.

Proposal: Since this PR now includes the tests that were unique to #32375, plus the Skill Safety Rule system prompt that was only in #32562, #32375 could be closed as superseded. The combined PR delivers P0 + P1 with proper test coverage in a single PR.

@alt-glitch happy to coordinate on the merge if this looks good.

@dolphin-creator

dolphin-creator commented May 29, 2026

Copy link
Copy Markdown
Author

Production validation: I've been running these exact P0+P1 patches on my own Hermes Agent instance since May 26th (3 days in production).

Zero issues, zero ghost skill loops, zero crashes. The [SKILL_PRUNED] marker + Skill Safety Rule system prompt work reliably in real-world long sessions.

Happy to provide more data points if needed.

@dolphin-creator dolphin-creator mentioned this pull request Jun 11, 2026
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

comp/agent Core agent loop, run_agent.py, prompt builder P2 Medium — degraded but workaround exists type/bug Something isn't working

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@dolphin-creator @alt-glitch