Skip to content

feat: skill prerequisites — hide skills with unmet runtime dependencies#659

Merged
teknium1 merged 1 commit into
NousResearch:mainfrom
kshitijk4poor:feat/skill-prerequisites
Mar 8, 2026
Merged

feat: skill prerequisites — hide skills with unmet runtime dependencies#659
teknium1 merged 1 commit into
NousResearch:mainfrom
kshitijk4poor:feat/skill-prerequisites

Conversation

@kshitijk4poor

@kshitijk4poor kshitijk4poor commented Mar 8, 2026

Copy link
Copy Markdown
Collaborator

Summary

Skills can now declare runtime prerequisites (env_vars, commands) via YAML frontmatter. Skills with unmet prerequisites are automatically hidden from the system prompt so the agent never claims capabilities it can't deliver, and skill_view() returns an actionable warning about what's missing.

This is the generalized fix for the class of bugs where the agent promises a skill-based capability (GIF search, email, Notion, etc.) then fails at execution time because a CLI tool isn't installed or an API key isn't set.

Closes #658
Fixes #630

Motivation

Tools already gate themselves behind availability checks — web_search won't appear without FIRECRAWL_API_KEY, browser_navigate won't appear without agent-browser on PATH, image_generate requires FAL_KEY, etc. Each tool registers a check_fn with the registry, and registry.get_definitions() excludes tools that fail their check.

Skills had no equivalent mechanism. Every skill in ~/.hermes/skills/ was unconditionally advertised in the system prompt regardless of whether its dependencies were present. This meant the agent would promise capabilities (GIF search, email via himalaya, Notion integration) then fail with cryptic command not found errors at execution time.

This PR brings skills to parity with tools by adding the same check semantics in a format appropriate for Markdown-based skills (declarative YAML frontmatter instead of Python callbacks).

How it works

The prerequisites field

---
name: gif-search
description: Search GIFs from Tenor
prerequisites:
  env_vars: [TENOR_API_KEY]     # checked via os.getenv()
  commands: [curl, jq]          # checked via shutil.which()
---

Both sub-fields are optional. Skills without prerequisites behave exactly as before (backward compatible).

Comparison with tool availability checks

Aspect Tools Skills (this PR)
Gate mechanism check_fn callback (Python) prerequisites frontmatter (YAML)
Env var check os.getenv("KEY") os.getenv("KEY")
Binary check shutil.which("bin") / custom shutil.which("bin")
Exclusion point registry.get_definitions() build_skills_system_prompt()
Effect Tool removed from LLM tool list Skill removed from <available_skills> prompt
Second safety net N/A (tool dispatch returns error) skill_view() returns prerequisites_warning

Same semantics, appropriate format for each: tools are Python code (callback), skills are Markdown files (declarative YAML).

Three layers of defense

Layer File Behavior
System prompt agent/prompt_builder.py Skills with unmet prereqs excluded from <available_skills> — LLM never sees them
skills_list() tools/skills_tool.py Returns prerequisites_met: false + prerequisites_missing in metadata
skill_view() tools/skills_tool.py Returns prerequisites_warning telling agent to inform the user

What changed

Core implementation

  • tools/skills_tool.pycheck_skill_prerequisites(frontmatter) function, updated _find_all_skills() and skill_view(), updated module docstring with spec
  • agent/prompt_builder.py_skill_prerequisites_met() helper, filtering in build_skills_system_prompt()

12 bundled skills tagged with prerequisites

Skill Prereq type Value
gifs/gif-search env_var TENOR_API_KEY
productivity/notion env_var NOTION_API_KEY
email/himalaya command himalaya
apple/imessage command imsg
apple/apple-notes command memo
apple/apple-reminders command remindctl
smart-home/openhue command openhue
research/duckduckgo-search command ddgs
github/codebase-inspection command pygount
feeds/blogwatcher command blogwatcher
music-creation/songsee command songsee
mcp/mcporter command npx

gif-search skill updated

  • Replaced hardcoded Tenor public demo API key with ${TENOR_API_KEY} env var
  • Added setup instructions for obtaining a free key

Docs

  • CONTRIBUTING.mdprerequisites field added to SKILL.md format spec + dedicated "Skill prerequisites" section with guidance on when to declare them

Tests (21 new, 104 total)

  • tests/tools/test_skills_tool.pyTestCheckSkillPrerequisites (9), TestFindAllSkillsPrerequisites (3), TestSkillViewPrerequisites (3)
  • tests/agent/test_prompt_builder.pyTestSkillPrerequisitesMet (4), prerequisite filtering in TestBuildSkillsSystemPrompt (2)

Files changed (17)

  • tools/skills_tool.py — core prerequisite checking
  • agent/prompt_builder.py — prompt-level filtering
  • CONTRIBUTING.md — spec docs
  • 12× skills/*/SKILL.md — prerequisite declarations
  • tests/ — 21 new test cases

Test plan

  • All 104 tests pass (83 existing + 21 new)
  • Verify skill with missing env var doesn't appear in system prompt
  • Verify skill_view("gif-search") shows warning when TENOR_API_KEY unset
  • Verify skill appears normally when prerequisites are satisfied
  • Verify skills without prerequisites field are unaffected

@kshitijk4poor kshitijk4poor marked this pull request as draft March 8, 2026 07:27
@kshitijk4poor kshitijk4poor force-pushed the feat/skill-prerequisites branch 2 times, most recently from 4c10afa to d1e67ff Compare March 8, 2026 07:46
…dependencies

Skills can now declare runtime prerequisites (env vars, CLI binaries) via
YAML frontmatter. Skills with unmet prerequisites are excluded from the
system prompt so the agent never claims capabilities it can't deliver, and
skill_view() warns the agent about what's missing.

Three layers of defense:
- build_skills_system_prompt() filters out unavailable skills
- _find_all_skills() flags unmet prerequisites in metadata
- skill_view() returns prerequisites_warning with actionable details

Tagged 12 bundled skills that have hard runtime dependencies:
gif-search (TENOR_API_KEY), notion (NOTION_API_KEY), himalaya, imessage,
apple-notes, apple-reminders, openhue, duckduckgo-search, codebase-inspection,
blogwatcher, songsee, mcporter.

Closes NousResearch#658
Fixes NousResearch#630
@teknium1

teknium1 commented Mar 8, 2026

Copy link
Copy Markdown
Contributor

Merged in 5a20c48 — clean merge, all 2209 tests pass (21 new). Great work! The prerequisite system brings skills to parity with tool availability checks. Thanks @kshitijk4poor!

@kshitijk4poor

Copy link
Copy Markdown
Collaborator Author

@teknium1 since the PR was closed before the latest review follow-ups were pushed. I’ve continued the work locally and there are a few additional fixes that should go in if this is reopened or superseded by a new PR.

These changes are already implemented and I’m currently testing them locally, will push shortly.

Additional changes beyond the earlier backend-aware prerequisite work:

  1. Avoid shared sandbox creation during taskless skill discovery
    build_skills_system_prompt() and similar taskless callers no longer create/reuse a backend sandbox just to evaluate prerequisites. For non-local backends without a real task_id, prerequisite status now degrades to unknown instead of booting a sandbox and risking shared state leakage.

  2. Remove implicit python3 dependency in probes
    The earlier probe assumed python3 existed in every backend. It now relies on shell builtins and command -v, making it safe for SSH or custom container images.

  3. Fix shell-injection risk in env-var probing
    prerequisites.env_vars (from skill frontmatter) is now validated before use. Invalid names are treated as missing and never interpolated into the probe command.

  4. Preserve configured backend default timeout
    The probe previously could initialize the shared environment with a 30s timeout if it was the first tool touching the task. Environment creation now preserves the configured default while the probe keeps its own 30s execution budget.

  5. Memoize prerequisite probes during skills_list()
    Adds a per-call probe cache to avoid repeated backend round-trips and sandbox startup attempts when checking multiple skills.

angelburgosrosado pushed a commit to angelburgosrosado/hermes-agent that referenced this pull request Apr 27, 2026
…th unmet runtime dependencies

Authored by kshitijk4poor. Fixes NousResearch#630.
02356abc pushed a commit to 02356abc/hermes-agent that referenced this pull request May 14, 2026
…th unmet runtime dependencies

Authored by kshitijk4poor. Fixes NousResearch#630.
olympus-terminal pushed a commit to olympus-terminal/hermes-agent that referenced this pull request May 16, 2026
…th unmet runtime dependencies

Authored by kshitijk4poor. Fixes NousResearch#630.
Egavasyug pushed a commit to Egavasyug/hermes-agent that referenced this pull request Jun 10, 2026
…th unmet runtime dependencies

Authored by kshitijk4poor. Fixes NousResearch#630.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

2 participants