feat(skills): quality gate prompt + wire up dead bump_use counter#17598
Closed
ZenoWangzy wants to merge 3 commits into
Closed
feat(skills): quality gate prompt + wire up dead bump_use counter#17598ZenoWangzy wants to merge 3 commits into
ZenoWangzy wants to merge 3 commits into
Conversation
Contributor
|
Thanks @ZenoWangzy — closing this one. The bump_use wiring concern is now covered by #17932 (merged), which calls bump_use() directly from the three canonical activation paths (slash command, --skill preload, skill_view tool call) rather than the session-end scan approach here. The direct approach has simpler failure modes and immediate signal. The skills.extra_review_prompt piece is unrelated scope and doesn't have a concrete driver right now — if you have a specific quality gate you want the background reviewer to enforce, happy to look at a focused PR for just that piece with a motivating use case in the description. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Wire up dead
bump_usecounter —tools/skill_usage.py:277definesbump_usebut it had zero production call sites. Addedagent/outcome_tracker.pythat scans session messages forskill_viewtool calls (OpenAI-format) and callsbump_usefor each unique skill at session end. Hooked intoAIAgent.shutdown_memory_provider(called at CLI exit, gateway session expiry,/reset). The Curator now gets realuse_count/last_used_atdata for lifecycle decisions.Configurable skill review quality gate — The background skill review prompt (
_SKILL_REVIEW_PROMPT) previously had no filter criteria. Addedskills.extra_review_promptconfig key (read from~/.hermes/config.yaml) that appends a user-defined quality gate to the review prompt wheneverreview_skills=True. Default is empty (no change in behavior). When set, the reviewer must pass all three checks before writing a skill.Files changed
agent/outcome_tracker.pyrecord_session_outcome()+_extract_skill_views()run_agent.py_spawn_background_review,shutdown_memory_provider)tests/agent/test_outcome_tracker.pytests/agent/test_skill_review_extra.pyTest plan
python -m pytest tests/agent/test_outcome_tracker.py -v→ 13 passedpython -m pytest tests/agent/test_skill_review_extra.py -v→ 5 passedpython -m pytest tests/gateway/test_unknown_command.py -q→ 6 passed~/.hermes/skills/.usage.json—use_countincrements andlast_used_atis setskills.extra_review_promptin~/.hermes/config.yaml, trigger a skill review, verify the quality gate criteria appear in the review prompt