feat(webhook): per-subscription --toolset override (closes #32899)#32902
feat(webhook): per-subscription --toolset override (closes #32899)#32902rmaffeo wants to merge 1 commit into
Conversation
c7030c8 to
e91b3ba
Compare
Webhook subscriptions default to the safe hermes-webhook toolset (web_search, web_extract, vision_analyze, clarify) introduced in NousResearch#30745 to prevent prompt-injection from third-party payloads (GitHub, Stripe, etc.) reaching local execution tools. That default is correct, but it broke trusted, authenticated LAN routes (Home Assistant rest_command + X-Gitlab-Token, internal CI runners) whose prompts genuinely need terminal/file/send_message access — there was no way to widen toolsets on those routes short of widening the entire platform globally. Add --toolset (repeatable) on 'hermes webhook subscribe', persist as toolsets: [...] on the route, propagate through MessageEvent .enabled_toolsets_override to gateway._run_agent, which resolves composite names (e.g. 'hermes-cli') against CONFIGURABLE_TOOLSETS the same way _get_platform_tools does. Default behavior unchanged when --toolset is omitted; unknown toolset names are rejected at subscribe time and logged-and-ignored at request time. Closes NousResearch#32899.
e91b3ba to
3f9f660
Compare
|
Pushed two follow-up fixes after a self-review:
New test file
Full webhook + tools-config + run-helper suite (216 tests) still green. PR diff is now 9 files, +442/-1. Still deferred to a follow-up PR if maintainers want it:
|
Closes #32899.
Summary
Adds a per-subscription
--toolsetflag onhermes webhook subscribeso trusted, authenticated webhook routes can opt out of the safe-default toolset locked down in #30745, without widening the platform globally viaplatform_toolsets.webhookinconfig.yaml.Motivation
#30745 (May 24 2026) correctly restricted the default
hermes-webhooktoolset to a safe subset (web_search,web_extract,vision_analyze,clarify) to keep prompt-injection from third-party webhook payloads — GitHub PR titles/comments, Stripe metadata, ntfy publishers — out ofterminal,write_file,send_message, and other local-execution tools.That's the right default. But Hermes's own bundled skills (
ha-hermes-webhook-bridge,ha-cross-sensor-snapshot-gate,webhook-subscriptions) explicitly document the webhook adapter as the integration point for trusted, authenticated event sources too — Home Assistant viarest_command+ per-routeX-Gitlab-Token, Hubitat virtual switches, internal CI/cron. Those flows depend onterminal/write_file/send_message(fetch a snapshot, save to disk, runvision_analyze, hand it off to Telegram viasend_message ... MEDIA:<path>). After #30745 every such route went silently broken: the gateway dispatches an agent that has no way to complete the task and either fails clearly or hallucinates"ok".The only existing workaround is
platform_toolsets.webhook: [hermes-cli]inconfig.yaml, which widens every webhook route — including the third-party-payload ones the lockdown was designed to protect.Design
Per-subscription
--toolsetoverride:hermes webhook subscribe ha-front-door \ --toolset hermes-cli \ --deliver log \ --prompt "..."--toolsetarg, repeatable (action="append"), accepts any toolset name inTOOLSETS(validated at subscribe time, rejected with a friendly error if unknown).toolsets: [...]in the subscription JSON. Subscriptions without it (the overwhelming majority — anything that takes payloads from outside the operator's network) keep the safe default.WebhookAdapterpopulatesMessageEvent.enabled_toolsets_override(new optional field on the dataclass) fromroute_config["toolsets"]. Empty list and missing field both normalise toNoneso the platform default still applies.GatewayRunner._run_agentacceptsenabled_toolsets_override. When set, composite names likehermes-cliare expanded to theirCONFIGURABLE_TOOLSETSmembers the same way_get_platform_toolsdoes, so callers can pass a single composite and get the right granular runtime set (not the raw composite name, which the agent runtime doesn't register as an individual toolset)._handle_messageforwardsevent.enabled_toolsets_overrideinto_run_agent. No other adapter sets the field, so behaviour everywhere else is unchanged.hermes webhook listand the subscribe-output banner display the configured toolsets when present.Proxy mode (
_run_agent_via_proxy) is intentionally out of scope — the override is only meaningful for the local-agent path. The proxy variant still uses the platform default.Security posture
The default for new subscriptions is unchanged. The widening is opt-in, per-route, requires the operator to explicitly pass
--toolset, and is documented in the updatedwebhook-subscriptionsskill with explicit guidance on when it's safe to use (authenticated LAN routes only) and when it's not (any webhook that receives payloads from outside operator control). The skill's "When NOT to use" section is deliberately blunt: "Anything the agent reads from those payloads becomes prompt-injectable text; withterminalorsend_messageexposed, that is a remote-code-execution surface."The HMAC auth, idempotency, rate limiting, signature validation, and other safety rails from previous webhook hardening (#30745, #30200, dbf73e9, 15aa688) are untouched.
Tests
New cases (all passing):
tests/hermes_cli/test_webhook_cli.py:test_toolset_override_persisted— single--toolsetis stored.test_toolset_override_repeatable— multiple--toolsetflags accumulate.test_toolset_unknown_rejected— unknown name aborts subscribe and the route isn't persisted.test_no_toolset_omits_key— default case writes notoolsetsfield (so the gateway falls through to the platform default resolver).tests/gateway/test_webhook_integration.py:test_toolset_field_propagated_to_event— route withtoolsets: ["hermes-cli"]yieldsevent.enabled_toolsets_override == ["hermes-cli"].test_no_toolset_field_leaves_override_none— route without the field yieldsNone.test_empty_toolset_list_treated_as_unset—toolsets: []normalises toNone.Existing webhook + tools_config suites all still pass (126 tests in the immediate-vicinity suites, 160 in the broader sweep, no regressions).
Backwards compatibility
toolsetsfield → resolve toNone→ platform default → safehermes-webhooktoolset. No behaviour change.MessageEvent.enabled_toolsets_overrideis an optional field with a default ofNone; no adapter except the webhook one sets it._run_agentadds the new kwarg withdefault=None; the second internal call site (followup_result, around line 17843) preserves None implicitly, which is correct for a different-event continuation.Notes for reviewers
website/docs/user-guide/skills/bundled/devops/devops-webhook-subscriptions.mdis not included in this PR —website/scripts/generate-skill-docs.pyrewrote line endings on every skill page (LF vs the existing CRLF on disk), which would have made the diff unreviewable. A maintainer with the right local line-ending setup should rerun the generator post-merge to refresh the bundled page from the updatedSKILL.md. The source skill is updated and the generator will pick it up.--toolsetresolver importstoolsets.TOOLSETSandhermes_cli.tools_config.{CONFIGURABLE_TOOLSETS, _toolset_allowed_for_platform}lazily inside the override branch so the cold path stays zero-cost when the override isn't set.