feat(review): class-first skill review prompt by teknium1 · Pull Request #16026 · NousResearch/hermes-agent

teknium1 · 2026-04-26T12:15:47Z

Summary

Reframes the background skill-review prompt from "did something interesting happen, save or update a skill" to a class-first protocol: survey existing skills → identify the CLASS of task → prefer updating/generalizing an existing skill → only create a new one at the class level as a last resort.

This is the creation-side half of the Curator feature (#7816). It stops near-duplicate skill accumulation AT THE SOURCE. The retirement/consolidation side (curator itself — sidecar telemetry, staleness, archival) ships in a follow-up PR.

Root cause

_SKILL_REVIEW_PROMPT asked "is this reusable?" without asking "do we already have a skill for this class?" Reviewer agents tend to create narrow, session-specific skills (fix-my-tauri-error-on-2026-04-24) instead of generalizing an existing one (desktop-app-build-troubleshooting). The catalog grows monotonically with near-duplicates; the prompt never points the reviewer at the existing landscape.

Changes

File	What
`run_agent.py`	Rewrite `_SKILL_REVIEW_PROMPT` as 5-step class-first protocol (survey → class → prefer-update → last-resort-create → flag-overlaps)
`run_agent.py`	Rewrite Skills half of `_COMBINED_REVIEW_PROMPT` with the same 5-step protocol (a–e); Memory half untouched
`tests/run_agent/test_review_prompt_class_first.py`	9 behavior tests — instructions are present (survey, class, update-over-create, overlap-flagging, opt-out). No snapshot assertions.

_MEMORY_REVIEW_PROMPT is unchanged and has a guardrail test confirming it doesn't accidentally gain skill-review instructions.

Validation

	Before	After
Skill review prompt	"did something interesting happen? save or update."	Survey → class-first → prefer update → last-resort create
Near-duplicate skills created	Routine	Discouraged at review time
Existing skill surface	Not consulted	`skills_list` + `skill_view` required first
Overlap detection	None	Reviewer flags for follow-up

Tests: scripts/run_tests.sh tests/run_agent/test_review_prompt_class_first.py — 9 passed in 3.09s. Sibling test_background_review_summary.py still passes (8/8). No change-detector tests introduced.

Tradeoffs

Longer prompt + 2-4 extra tool calls per nudge when something IS worth saving. Zero extra cost when nothing is (the "Nothing to save." escape clause is preserved). Fine for a 10-prompt cadence background task.

Follow-up

Curator infrastructure (sidecar .usage.json telemetry, idle-triggered daily review, lifecycle states, /curator CLI + slash command) ships in a separate PR. This PR is complete and independent.

Refs #7816.

The background skill-review prompt (spawned after N user turns) now instructs the reviewer to SURVEY existing skills first, identify the CLASS of task, and PREFER updating/generalizing an existing skill over creating a new narrow one. This reduces near-duplicate skill accumulation at the source. Catches the common failure mode where repeated tasks of the same class each spawn their own specific skill ("fix-my-tauri-error", "fix-my-electron-error") instead of a single class-level skill ("desktop-app-build-troubleshooting"). Applied to both _SKILL_REVIEW_PROMPT and the **Skills** half of _COMBINED_REVIEW_PROMPT. Memory-only review prompt unchanged. Groundwork for the Curator feature (issue #7816) — the creation-side fix. Curator handles the retirement/consolidation side in a follow-up PR. Tests assert the behavioral instructions are present (survey, class, update- over-create, overlap-flagging, opt-out clause) rather than snapshotting the full prompt text.

Adds the Curator — an auxiliary-model background task that periodically reviews AGENT-CREATED skills and keeps the collection tidy: tracks usage, transitions unused skills through active → stale → archived, and spawns a forked AIAgent to consolidate overlaps and patch drift. Default: enabled, inactivity-triggered (no cron daemon). Runs on CLI startup and gateway boot when the last run is older than interval_hours (default 24) AND the agent has been idle for min_idle_hours (default 2). Invariants (all load-bearing): - Never touches bundled or hub-installed skills (.bundled_manifest + .hub/lock.json double-filter) - Never auto-deletes — archive only. Archives are recoverable via `hermes curator restore <skill>` - Pinned skills bypass all auto-transitions - Uses the aux client; never touches the main session's prompt cache New files: - tools/skill_usage.py — sidecar .usage.json telemetry, atomic writes, provenance filter - agent/curator.py — orchestrator: config, idle gating, state-machine transitions (pure, no LLM), forked-agent review prompt - hermes_cli/curator.py — `hermes curator {status,run,pause,resume, pin,unpin,restore}` subcommand - tests/tools/test_skill_usage.py — 29 tests - tests/agent/test_curator.py — 25 tests Modified files (surgical patches): - tools/skills_tool.py — bump view_count on successful skill_view - tools/skill_manager_tool.py — bump patch_count on skill_manage patch/edit/write_file/remove_file; forget record on delete - hermes_cli/config.py — add curator: section to DEFAULT_CONFIG - hermes_cli/commands.py — add /curator CommandDef with subcommands - hermes_cli/main.py — register `hermes curator` subparser via register_cli() from hermes_cli.curator - cli.py — /curator slash-command dispatch + startup hook - gateway/run.py — gateway-boot hook (mirrors CLI) Validation: - 54 new tests across skill_usage + curator, all passing in 3s - 346 tests across all touched files' neighbors green - 2783 tests across hermes_cli/ + gateway/test_run_progress_topics.py green - CLI smoke: `hermes curator status/pause/resume` work end-to-end Companion to PR #16026 (class-first skill review prompt) — together they form a loop: the review prompt stops near-duplicate skill creation at the source, and the curator prunes/consolidates what still accumulates. Refs #7816.

The background skill-review prompt (spawned after N user turns) now instructs the reviewer to SURVEY existing skills first, identify the CLASS of task, and PREFER updating/generalizing an existing skill over creating a new narrow one. This reduces near-duplicate skill accumulation at the source. Catches the common failure mode where repeated tasks of the same class each spawn their own specific skill ("fix-my-tauri-error", "fix-my-electron-error") instead of a single class-level skill ("desktop-app-build-troubleshooting"). Applied to both _SKILL_REVIEW_PROMPT and the **Skills** half of _COMBINED_REVIEW_PROMPT. Memory-only review prompt unchanged. Groundwork for the Curator feature (issue NousResearch#7816) — the creation-side fix. Curator handles the retirement/consolidation side in a follow-up PR. Tests assert the behavioral instructions are present (survey, class, update- over-create, overlap-flagging, opt-out clause) rather than snapshotting the full prompt text.

Adds the Curator — an auxiliary-model background task that periodically reviews AGENT-CREATED skills and keeps the collection tidy: tracks usage, transitions unused skills through active → stale → archived, and spawns a forked AIAgent to consolidate overlaps and patch drift. Default: enabled, inactivity-triggered (no cron daemon). Runs on CLI startup and gateway boot when the last run is older than interval_hours (default 24) AND the agent has been idle for min_idle_hours (default 2). Invariants (all load-bearing): - Never touches bundled or hub-installed skills (.bundled_manifest + .hub/lock.json double-filter) - Never auto-deletes — archive only. Archives are recoverable via `hermes curator restore <skill>` - Pinned skills bypass all auto-transitions - Uses the aux client; never touches the main session's prompt cache New files: - tools/skill_usage.py — sidecar .usage.json telemetry, atomic writes, provenance filter - agent/curator.py — orchestrator: config, idle gating, state-machine transitions (pure, no LLM), forked-agent review prompt - hermes_cli/curator.py — `hermes curator {status,run,pause,resume, pin,unpin,restore}` subcommand - tests/tools/test_skill_usage.py — 29 tests - tests/agent/test_curator.py — 25 tests Modified files (surgical patches): - tools/skills_tool.py — bump view_count on successful skill_view - tools/skill_manager_tool.py — bump patch_count on skill_manage patch/edit/write_file/remove_file; forget record on delete - hermes_cli/config.py — add curator: section to DEFAULT_CONFIG - hermes_cli/commands.py — add /curator CommandDef with subcommands - hermes_cli/main.py — register `hermes curator` subparser via register_cli() from hermes_cli.curator - cli.py — /curator slash-command dispatch + startup hook - gateway/run.py — gateway-boot hook (mirrors CLI) Validation: - 54 new tests across skill_usage + curator, all passing in 3s - 346 tests across all touched files' neighbors green - 2783 tests across hermes_cli/ + gateway/test_run_progress_topics.py green - CLI smoke: `hermes curator status/pause/resume` work end-to-end Companion to PR NousResearch#16026 (class-first skill review prompt) — together they form a loop: the review prompt stops near-duplicate skill creation at the source, and the curator prunes/consolidates what still accumulates. Refs NousResearch#7816.