Skip to content

fix(curator): add post-archive safety guard to prevent unverified skill archival#30041

Closed
alaamohanad169-ship-it wants to merge 1 commit into
NousResearch:mainfrom
alaamohanad169-ship-it:fix/curator-safety-guard
Closed

fix(curator): add post-archive safety guard to prevent unverified skill archival#30041
alaamohanad169-ship-it wants to merge 1 commit into
NousResearch:mainfrom
alaamohanad169-ship-it:fix/curator-safety-guard

Conversation

@alaamohanad169-ship-it

Copy link
Copy Markdown
Contributor

Summary

When the curator LLM archives skills without verified consolidation evidence (e.g., hallucinated umbrella names, fallback/no-evidence pruning), active workflows break silently. This adds a post-archive safety guard that:

  • Detects skills removed by the LLM review pass
  • Checks for tool-call evidence of real consolidation (write_file/patch/create/edit via skill_manage on an umbrella skill)
  • If NO consolidation evidence exists: moves skills back from .archive/ to the active skills directory (fail-closed behavior)
  • Logs a warning and annotates the run summary with the rollback count

Problem

Issue #29912: In a real run, 10 operational skills were archived in one pass with consolidated_this_run=0 and all pruned entries sourced from "fallback (model named missing umbrella, no tool-call evidence)". This broke active workflows because the original skill names were removed from normal lookup.

Fix

Added a post-archive validation guard in _llm_pass() inside run_curator_review() (agent/curator.py). After the LLM review completes:

  1. Compare before_names vs after_names to detect removed skills
  2. Scan tool calls for skill_manage with write_file/patch/create/edit actions (consolidation evidence)
  3. If no consolidation evidence found: restore archived skills from .archive/ to skills/
  4. Annotate the run summary with the rollback count

Testing

  • Unit test for the safety guard logic
  • Integration test: verify skills are restored when no consolidation evidence exists
  • Integration test: verify skills remain archived when consolidation evidence exists

Fixes #29912

@alt-glitch alt-glitch added type/bug Something isn't working tool/skills Skills system (list, view, manage) P2 Medium — degraded but workaround exists labels May 21, 2026
@alaamohanad169-ship-it

Copy link
Copy Markdown
Contributor Author

Superseded by a rebased version. Closing this PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

P2 Medium — degraded but workaround exists tool/skills Skills system (list, view, manage) type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Curator may archive active skills during umbrella pass without verified consolidation (fail-open behavior)

2 participants