feat: skill prerequisites — hide skills with unmet runtime dependencies#659
Conversation
4c10afa to
d1e67ff
Compare
…dependencies Skills can now declare runtime prerequisites (env vars, CLI binaries) via YAML frontmatter. Skills with unmet prerequisites are excluded from the system prompt so the agent never claims capabilities it can't deliver, and skill_view() warns the agent about what's missing. Three layers of defense: - build_skills_system_prompt() filters out unavailable skills - _find_all_skills() flags unmet prerequisites in metadata - skill_view() returns prerequisites_warning with actionable details Tagged 12 bundled skills that have hard runtime dependencies: gif-search (TENOR_API_KEY), notion (NOTION_API_KEY), himalaya, imessage, apple-notes, apple-reminders, openhue, duckduckgo-search, codebase-inspection, blogwatcher, songsee, mcporter. Closes NousResearch#658 Fixes NousResearch#630
d1e67ff to
f210510
Compare
|
Merged in 5a20c48 — clean merge, all 2209 tests pass (21 new). Great work! The prerequisite system brings skills to parity with tool availability checks. Thanks @kshitijk4poor! |
|
@teknium1 since the PR was closed before the latest review follow-ups were pushed. I’ve continued the work locally and there are a few additional fixes that should go in if this is reopened or superseded by a new PR. These changes are already implemented and I’m currently testing them locally, will push shortly. Additional changes beyond the earlier backend-aware prerequisite work:
|
…th unmet runtime dependencies Authored by kshitijk4poor. Fixes NousResearch#630.
…th unmet runtime dependencies Authored by kshitijk4poor. Fixes NousResearch#630.
…th unmet runtime dependencies Authored by kshitijk4poor. Fixes NousResearch#630.
…th unmet runtime dependencies Authored by kshitijk4poor. Fixes NousResearch#630.
Summary
Skills can now declare runtime prerequisites (
env_vars,commands) via YAML frontmatter. Skills with unmet prerequisites are automatically hidden from the system prompt so the agent never claims capabilities it can't deliver, andskill_view()returns an actionable warning about what's missing.This is the generalized fix for the class of bugs where the agent promises a skill-based capability (GIF search, email, Notion, etc.) then fails at execution time because a CLI tool isn't installed or an API key isn't set.
Closes #658
Fixes #630
Motivation
Tools already gate themselves behind availability checks —
web_searchwon't appear withoutFIRECRAWL_API_KEY,browser_navigatewon't appear withoutagent-browseron PATH,image_generaterequiresFAL_KEY, etc. Each tool registers acheck_fnwith the registry, andregistry.get_definitions()excludes tools that fail their check.Skills had no equivalent mechanism. Every skill in
~/.hermes/skills/was unconditionally advertised in the system prompt regardless of whether its dependencies were present. This meant the agent would promise capabilities (GIF search, email via himalaya, Notion integration) then fail with crypticcommand not founderrors at execution time.This PR brings skills to parity with tools by adding the same check semantics in a format appropriate for Markdown-based skills (declarative YAML frontmatter instead of Python callbacks).
How it works
The
prerequisitesfieldBoth sub-fields are optional. Skills without
prerequisitesbehave exactly as before (backward compatible).Comparison with tool availability checks
check_fncallback (Python)prerequisitesfrontmatter (YAML)os.getenv("KEY")os.getenv("KEY")shutil.which("bin")/ customshutil.which("bin")registry.get_definitions()build_skills_system_prompt()<available_skills>promptskill_view()returnsprerequisites_warningSame semantics, appropriate format for each: tools are Python code (callback), skills are Markdown files (declarative YAML).
Three layers of defense
agent/prompt_builder.py<available_skills>— LLM never sees themskills_list()tools/skills_tool.pyprerequisites_met: false+prerequisites_missingin metadataskill_view()tools/skills_tool.pyprerequisites_warningtelling agent to inform the userWhat changed
Core implementation
tools/skills_tool.py—check_skill_prerequisites(frontmatter)function, updated_find_all_skills()andskill_view(), updated module docstring with specagent/prompt_builder.py—_skill_prerequisites_met()helper, filtering inbuild_skills_system_prompt()12 bundled skills tagged with prerequisites
gifs/gif-searchTENOR_API_KEYproductivity/notionNOTION_API_KEYemail/himalayahimalayaapple/imessageimsgapple/apple-notesmemoapple/apple-remindersremindctlsmart-home/openhueopenhueresearch/duckduckgo-searchddgsgithub/codebase-inspectionpygountfeeds/blogwatcherblogwatchermusic-creation/songseesongseemcp/mcporternpxgif-search skill updated
${TENOR_API_KEY}env varDocs
CONTRIBUTING.md—prerequisitesfield added to SKILL.md format spec + dedicated "Skill prerequisites" section with guidance on when to declare themTests (21 new, 104 total)
tests/tools/test_skills_tool.py—TestCheckSkillPrerequisites(9),TestFindAllSkillsPrerequisites(3),TestSkillViewPrerequisites(3)tests/agent/test_prompt_builder.py—TestSkillPrerequisitesMet(4), prerequisite filtering inTestBuildSkillsSystemPrompt(2)Files changed (17)
tools/skills_tool.py— core prerequisite checkingagent/prompt_builder.py— prompt-level filteringCONTRIBUTING.md— spec docsskills/*/SKILL.md— prerequisite declarationstests/— 21 new test casesTest plan
skill_view("gif-search")shows warning whenTENOR_API_KEYunsetprerequisitesfield are unaffected