feat(watchers): interval-polling watcher engine with watermark dedup by teknium1 · Pull Request #21497 · NousResearch/hermes-agent

teknium1 · 2026-05-07T20:16:04Z

Summary

Adds hermes watch — the pull-based sibling of webhooks. Watchers poll an external source on an interval, dedup new items via a watermark, and either deliver them verbatim or hand them to a short-lived agent.

Inspired by Vellum Assistant's watcher system.

Mental model

Trigger	Source	When
Webhook	external push	whenever the source calls you
Cron	time-based	on schedule, regardless of change
Watcher	pull on interval	only when new data is detected

New surface

watchers/ package: engine + store + providers registry
Built-in providers: http_json (any JSON endpoint with configurable id_field/items_path), rss (RSS 2.0 + Atom), github (issues/pulls/releases/commits/search)
hermes watch {add,list,remove,run,reset,tick} CLI
Custom providers plug in via watchers.providers.register() at import time

Integration

Piggybacks on cron's tick loop — if cron runs, watchers run. Own file lock prevents double-fire across gateway/daemon/manual ticks.
Delivery reuses cron's _deliver_result, so multi/all/platform targets (incl. feat(cron): routing intent — deliver=all fans out to every connected channel #21495's routing intents) all work out of the box.
First poll of a new watcher records a baseline — it never replays the existing feed. Only items that appear after the baseline are delivered.
Watermark is a bounded ID set (max_seen=500) stored per-watcher at ~/.hermes/watchers/<name>.watermark.json so hermes watch reset can nuke a single one without touching others.

Quick-start examples

# Watch GitHub issues on a repo, agent-mode
hermes watch add hermes-issues \\
  --provider github --repo NousResearch/hermes-agent --scope issues \\
  --interval 300 --deliver origin

# Watch an RSS feed, deliver raw (no LLM cost)
hermes watch add hn --provider rss \\
  --url https://news.ycombinator.com/rss \\
  --interval 900 --deliver telegram --deliver-only

# Watch any JSON endpoint
hermes watch add api-events \\
  --provider http_json --url https://api.example.com/events \\
  --arg id_field=event_id --arg items_path=data.events --interval 60

Validation

scripts/run_tests.sh tests/watchers/                   # 46 passed
scripts/run_tests.sh tests/cron/test_scheduler.py      # 115 passed (piggyback hook safe)
HERMES_HOME=/tmp/... hermes watch --help    # CLI parses cleanly
HERMES_HOME=/tmp/... hermes watch list      # empty list prints setup help

Changes

watchers/{__init__,store,providers,engine}.py: new package (~680 LOC)
hermes_cli/watchers.py: new CLI command (~210 LOC)
hermes_cli/main.py: argparse wiring + cmd_watch entrypoint (~90 LOC)
cron/scheduler.py: best-effort watcher tick after cron tick (~15 LOC)
tests/watchers/: 46 tests across store/providers/engine
website/docs/user-guide/features/watchers.md + sidebar entry

No breaking changes. Watcher subsystem is entirely new; existing cron/webhook paths untouched except for the tick-loop hook (import-guarded, best-effort, can't break cron).

Adds 'hermes watch' — the pull-based sibling of webhooks. Watchers poll an external source on an interval, dedup new items via a per-watcher watermark, and either deliver the raw items verbatim or hand them to a short-lived agent. Inspired by Vellum Assistant's watcher system. ## New surface - `watchers/` package: engine + store + providers registry - 3 built-in providers: `http_json` (any JSON endpoint), `rss` (RSS 2.0 + Atom), `github` (issues/pulls/releases/commits or search) - `hermes watch {add,list,remove,run,reset,tick}` CLI - Custom providers: `from watchers.providers import register` ## Integration - Piggybacks on cron's tick loop — if cron runs, watchers run. Own file lock prevents double-fire across gateway/daemon/manual ticks. - Delivery reuses cron's `_deliver_result` so `multi`/`all`/platform targets all work out of the box. - First poll of a new watcher **records a baseline** — it never replays the existing feed. Only items that appear after baseline are delivered. - Watermark is a bounded ID set (max_seen, default 500) to cap memory. Stored per-watcher at `~/.hermes/watchers/<name>.watermark.json`. ## Changes - watchers/{__init__,store,providers,engine}.py: new - hermes_cli/watchers.py: new CLI command - hermes_cli/main.py: argparse wiring + cmd_watch entrypoint - cron/scheduler.py: best-effort watcher tick after cron tick - tests/watchers/{test_store,test_providers,test_engine}.py: 46 tests - website/docs/user-guide/features/watchers.md + sidebar entry ## Validation ``` scripts/run_tests.sh tests/watchers/ # 46 passed scripts/run_tests.sh tests/cron/test_scheduler.py # 115 passed (piggyback hook safe) HERMES_HOME=/tmp/... python -m hermes_cli.main watch --help # CLI parses HERMES_HOME=/tmp/... python -m hermes_cli.main watch list # empty list prints help text ```

github-actions · 2026-05-07T20:17:13Z

🔎 Lint report: `feat/watchers` vs `origin/main`

ruff

Total: 0 on HEAD, 0 on base (➖ 0)

🆕 New issues: none

✅ Fixed issues: none

Unchanged: 0 pre-existing issues carried over.

ty (type checker)

Total: 7676 on HEAD, 7670 on base (🆕 +6)

🆕 New issues (18):

Rule	Count
`invalid-argument-type`	7
`unresolved-import`	4
`unresolved-attribute`	3
`unsupported-operator`	3
`unused-type-ignore-comment`	1

First entries

tests/run_agent/test_provider_attribution_headers.py:65: [unresolved-attribute] unresolved-attribute: Attribute `startswith` is not defined on `dict[str, str]` in union `Unknown | str | dict[str, str]`
tests/agent/test_codex_cloudflare_headers.py:181: [unsupported-operator] unsupported-operator: Operator `in` is not supported between objects of type `Literal["originator"]` and `(Unknown & ~AlwaysFalsy) | (str & ~AlwaysFalsy) | (dict[str, str] & ~AlwaysFalsy) | ... omitted 3 union elements`
run_agent.py:8167: [invalid-argument-type] invalid-argument-type: Argument to function `build_anthropic_client` is incorrect: Expected `str`, found `str | Unknown | dict[Unknown | str, Unknown | str | dict[str, str]] | int | dict[Unknown, Unknown]`
tests/run_agent/test_provider_attribution_headers.py:130: [unsupported-operator] unsupported-operator: Operator `not in` is not supported between objects of type `Literal["X-OpenRouter-Cache"]` and `Unknown | str | dict[str, str] | ... omitted 3 union elements`
run_agent.py:8084: [invalid-argument-type] invalid-argument-type: Argument to bound method `ContextCompressor.update_model` is incorrect: Expected `int`, found `str | Unknown | dict[Unknown | str, Unknown | str | dict[str, str]] | int | dict[Unknown, Unknown]`
watchers/engine.py:36: [unused-type-ignore-comment] unused-type-ignore-comment: Unused blanket `type: ignore` directive
run_agent.py:6478: [invalid-argument-type] invalid-argument-type: Argument to function `_codex_cloudflare_headers` is incorrect: Expected `str`, found `Unknown | str | dict[str, str] | ... omitted 3 union elements`
run_agent.py:6649: [invalid-argument-type] invalid-argument-type: Argument to function `build_anthropic_client` is incorrect: Expected `str`, found `str | dict[Unknown, Unknown] | Any | ... omitted 3 union elements`
tests/run_agent/test_provider_attribution_headers.py:131: [unsupported-operator] unsupported-operator: Operator `not in` is not supported between objects of type `Literal["X-OpenRouter-Cache-TTL"]` and `Unknown | str | dict[str, str] | ... omitted 3 union elements`
run_agent.py:12528: [invalid-argument-type] invalid-argument-type: Argument to function `len` is incorrect: Expected `Sized`, found `(str & ~AlwaysFalsy) | (dict[Unknown, Unknown] & ~AlwaysFalsy) | (Any & ~AlwaysFalsy) | ... omitted 3 union elements`
run_agent.py:11373: [invalid-argument-type] invalid-argument-type: Argument to function `apply_anthropic_cache_control` is incorrect: Expected `bool`, found `int | str | Unknown | dict[Unknown | str, Unknown | str | dict[str, str]] | dict[Unknown, Unknown]`
tests/watchers/test_store.py:8: [unresolved-import] unresolved-import: Cannot resolve imported module `pytest`
run_agent.py:12525: [invalid-argument-type] invalid-argument-type: Argument to function `_is_oauth_token` is incorrect: Expected `str`, found `str | dict[Unknown, Unknown] | Any | ... omitted 3 union elements`
watchers/engine.py:161: [unresolved-import] unresolved-import: Cannot resolve imported module `agent.skill_tools`
tests/agent/test_codex_cloudflare_headers.py:163: [unresolved-attribute] unresolved-attribute: Attribute `startswith` is not defined on `dict[str, str]` in union `Unknown | str | dict[str, str]`
tests/agent/test_codex_cloudflare_headers.py:163: [unresolved-attribute] unresolved-attribute: Attribute `get` is not defined on `str & ~AlwaysFalsy`, `int & ~AlwaysFalsy` in union `(Unknown & ~AlwaysFalsy) | (str & ~AlwaysFalsy) | (dict[str, str] & ~AlwaysFalsy) | ... omitted 3 union elements`
tests/watchers/test_engine.py:12: [unresolved-import] unresolved-import: Cannot resolve imported module `pytest`
tests/watchers/test_providers.py:8: [unresolved-import] unresolved-import: Cannot resolve imported module `pytest`

✅ Fixed issues (17):

Rule	Count
`invalid-argument-type`	10
`unresolved-attribute`	4
`unsupported-operator`	3

First entries

run_agent.py:7846: [unresolved-attribute] unresolved-attribute: Attribute `strip` is not defined on `dict[Unknown | str, Unknown | str | dict[str, str]] & ~AlwaysFalsy`, `int & ~AlwaysFalsy`, `dict[Unknown, Unknown] & ~AlwaysFalsy` in union `Divergent | Divergent | (str & ~AlwaysFalsy) | ... omitted 5 union elements`
run_agent.py:6649: [invalid-argument-type] invalid-argument-type: Argument to function `build_anthropic_client` is incorrect: Expected `str`, found `str | dict[Unknown, Unknown] | Any | ... omitted 5 union elements`
run_agent.py:8084: [invalid-argument-type] invalid-argument-type: Argument to bound method `ContextCompressor.update_model` is incorrect: Expected `int`, found `Divergent | Divergent | str | ... omitted 4 union elements`
tests/run_agent/test_provider_attribution_headers.py:130: [unsupported-operator] unsupported-operator: Operator `not in` is not supported between objects of type `Literal["X-OpenRouter-Cache"]` and `Unknown | str | dict[str, str] | ... omitted 4 union elements`
run_agent.py:12525: [invalid-argument-type] invalid-argument-type: Argument to function `_is_oauth_token` is incorrect: Expected `str`, found `str | dict[Unknown, Unknown] | Any | ... omitted 5 union elements`
run_agent.py:11373: [invalid-argument-type] invalid-argument-type: Argument to function `apply_anthropic_cache_control` is incorrect: Expected `bool`, found `int | Divergent | Divergent | ... omitted 4 union elements`
run_agent.py:12528: [invalid-argument-type] invalid-argument-type: Argument to function `len` is incorrect: Expected `Sized`, found `(str & ~AlwaysFalsy) | (dict[Unknown, Unknown] & ~AlwaysFalsy) | (Any & ~AlwaysFalsy) | ... omitted 5 union elements`
run_agent.py:8087: [invalid-argument-type] invalid-argument-type: Argument to bound method `ContextCompressor.update_model` is incorrect: Expected `str`, found `Divergent | Divergent | str | ... omitted 4 union elements`
tests/run_agent/test_provider_attribution_headers.py:65: [unresolved-attribute] unresolved-attribute: Attribute `startswith` is not defined on `dict[str, str]` in union `Unknown | str | Divergent | dict[str, str]`
tests/agent/test_codex_cloudflare_headers.py:163: [unresolved-attribute] unresolved-attribute: Attribute `startswith` is not defined on `dict[str, str]` in union `Unknown | str | Divergent | dict[str, str]`
tests/agent/test_codex_cloudflare_headers.py:163: [unresolved-attribute] unresolved-attribute: Attribute `get` is not defined on `str & ~AlwaysFalsy`, `int & ~AlwaysFalsy` in union `(Unknown & ~AlwaysFalsy) | (str & ~AlwaysFalsy) | (dict[str, str] & ~AlwaysFalsy) | ... omitted 4 union elements`
tests/agent/test_codex_cloudflare_headers.py:181: [unsupported-operator] unsupported-operator: Operator `in` is not supported between objects of type `Literal["originator"]` and `(Unknown & ~AlwaysFalsy) | (str & ~AlwaysFalsy) | (dict[str, str] & ~AlwaysFalsy) | ... omitted 4 union elements`
run_agent.py:8167: [invalid-argument-type] invalid-argument-type: Argument to function `build_anthropic_client` is incorrect: Expected `str`, found `Divergent | Divergent | str | ... omitted 4 union elements`
run_agent.py:8168: [invalid-argument-type] invalid-argument-type: Argument to function `get_provider_request_timeout` is incorrect: Expected `str`, found `Divergent | Divergent | str | ... omitted 4 union elements`
tests/run_agent/test_provider_attribution_headers.py:131: [unsupported-operator] unsupported-operator: Operator `not in` is not supported between objects of type `Literal["X-OpenRouter-Cache-TTL"]` and `Unknown | str | dict[str, str] | ... omitted 4 union elements`
run_agent.py:8168: [invalid-argument-type] invalid-argument-type: Argument to function `get_provider_request_timeout` is incorrect: Expected `str | None`, found `Divergent | Divergent | str | ... omitted 4 union elements`
run_agent.py:6478: [invalid-argument-type] invalid-argument-type: Argument to function `_codex_cloudflare_headers` is incorrect: Expected `str`, found `Unknown | str | dict[str, str] | ... omitted 4 union elements`

Unchanged: 4015 pre-existing issues carried over.

Diagnostics are surfaced as warnings — this check never fails the build.

teknium1 · 2026-05-08T13:16:25Z

Closing in favor of shipping this as an optional skill — see upcoming PR. The value here (built-in provider code for RSS/HTTP JSON/GitHub + watermark dedup) is real, but it doesn't justify a parallel subsystem to cron. cron's no_agent=True + script already provides scheduling, delivery, and [SILENT] suppression; we just need reusable template scripts for the common fetch-and-dedup patterns. A skill captures that without adding a new core CLI verb or piggyback tick.

… no-agent (#21881) * feat(skills): watchers skill — poll RSS / HTTP JSON / GitHub via cron no-agent Ships three reusable polling scripts plus a shared watermark helper as an optional skill. Users wire them into the existing cron (no_agent=True) mode rather than learning a new subsystem. Supersedes the closed PR #21497 (parallel watcher subsystem). Same value, zero new core surface. ## What ships - optional-skills/devops/watchers/SKILL.md: pattern + three example cron commands - optional-skills/devops/watchers/scripts/_watermark.py: shared helper (atomic state writes, bounded ID set, first-run baseline) - optional-skills/devops/watchers/scripts/watch_rss.py: RSS 2.0 + Atom - optional-skills/devops/watchers/scripts/watch_http_json.py: any JSON endpoint with configurable id_field / items_path / headers - optional-skills/devops/watchers/scripts/watch_github.py: issues / pulls / releases / commits (uses GITHUB_TOKEN if present) ## Invariants enforced by the shared helper - First run records baseline, emits nothing (never replays existing feed) - Watermark file is <state_dir>/<name>.json, atomic replace on write - Bounded to 500 IDs (configurable) - Empty stdout when no new items — cron treats that as silent delivery ## Validation - watch_rss.py against news.ycombinator.com/rss first run → empty stdout, watermark populated - Removed one seen-id, second run → emitted exactly that item - No DeprecationWarnings (ET element truth-value footgun dodged explicitly) End-user pattern: 'hermes cron create my-feed --schedule "*/15 * * * *" --no-agent --script $HERMES_HOME/skills/devops/watchers/scripts/watch_rss.py --script-args "--name hn --url https://news.ycombinator.com/rss" --deliver telegram' * docs(skills/watchers): tighten description to match peer optional skills * docs(skills/watchers): align frontmatter + structure with peer optional skills * docs(skills/watchers): gate to linux/macos (shell syntax in examples)

… no-agent (NousResearch#21881) * feat(skills): watchers skill — poll RSS / HTTP JSON / GitHub via cron no-agent Ships three reusable polling scripts plus a shared watermark helper as an optional skill. Users wire them into the existing cron (no_agent=True) mode rather than learning a new subsystem. Supersedes the closed PR NousResearch#21497 (parallel watcher subsystem). Same value, zero new core surface. ## What ships - optional-skills/devops/watchers/SKILL.md: pattern + three example cron commands - optional-skills/devops/watchers/scripts/_watermark.py: shared helper (atomic state writes, bounded ID set, first-run baseline) - optional-skills/devops/watchers/scripts/watch_rss.py: RSS 2.0 + Atom - optional-skills/devops/watchers/scripts/watch_http_json.py: any JSON endpoint with configurable id_field / items_path / headers - optional-skills/devops/watchers/scripts/watch_github.py: issues / pulls / releases / commits (uses GITHUB_TOKEN if present) ## Invariants enforced by the shared helper - First run records baseline, emits nothing (never replays existing feed) - Watermark file is <state_dir>/<name>.json, atomic replace on write - Bounded to 500 IDs (configurable) - Empty stdout when no new items — cron treats that as silent delivery ## Validation - watch_rss.py against news.ycombinator.com/rss first run → empty stdout, watermark populated - Removed one seen-id, second run → emitted exactly that item - No DeprecationWarnings (ET element truth-value footgun dodged explicitly) End-user pattern: 'hermes cron create my-feed --schedule "*/15 * * * *" --no-agent --script $HERMES_HOME/skills/devops/watchers/scripts/watch_rss.py --script-args "--name hn --url https://news.ycombinator.com/rss" --deliver telegram' * docs(skills/watchers): tighten description to match peer optional skills * docs(skills/watchers): align frontmatter + structure with peer optional skills * docs(skills/watchers): gate to linux/macos (shell syntax in examples)

alt-glitch added type/feature New feature or request P3 Low — cosmetic, nice to have comp/cron Cron scheduler and job management comp/cli CLI entry point, hermes_cli/, setup wizard labels May 7, 2026

teknium1 closed this May 8, 2026

teknium1 mentioned this pull request May 8, 2026

feat(skills): watchers skill — poll RSS / HTTP JSON / GitHub via cron no-agent #21881

Merged

teknium1 mentioned this pull request Jun 7, 2026

feat(monitor): proactive urgency-classifying monitor (skill + aux task) #41122

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(watchers): interval-polling watcher engine with watermark dedup#21497

feat(watchers): interval-polling watcher engine with watermark dedup#21497
teknium1 wants to merge 1 commit into
mainfrom
feat/watchers

teknium1 commented May 7, 2026

Uh oh!

github-actions Bot commented May 7, 2026

Uh oh!

teknium1 commented May 8, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

teknium1 commented May 7, 2026

Summary

Mental model

New surface

Integration

Quick-start examples

Validation

Changes

Uh oh!

github-actions Bot commented May 7, 2026

🔎 Lint report: feat/watchers vs origin/main

ruff

ty (type checker)

Uh oh!

teknium1 commented May 8, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

🔎 Lint report: `feat/watchers` vs `origin/main`