Skip to content

feat: harden roadmap tier2 tools#1

Merged
xbrrr merged 371 commits into
mainfrom
feat/roadmap-tier2-2605
May 26, 2026
Merged

feat: harden roadmap tier2 tools#1
xbrrr merged 371 commits into
mainfrom
feat/roadmap-tier2-2605

Conversation

@xbrrr

@xbrrr xbrrr commented May 26, 2026

Copy link
Copy Markdown
Owner

Summary

  • Add dry-run support to patch/write_file; large write_file calls produce diff+lint without writing by default.
  • Add smart-window behavior for read_file after search context, with language-aware overview for Python/JS/TS.
  • Add opt-in MCP progressive loading via mcp_progressive_loading/progressive_loading, default off to avoid workflow breakage.

Verification

  • py_compile tools/file_tools.py tools/mcp_tool.py
  • targeted tests: 115 passed
  • regression pack: 227 passed, 1 skipped

Rollout

  • No restart needed until enabling/rolling out runtime behavior.
  • MCP progressive loading remains opt-in/off by default.

novax635 and others added 30 commits May 23, 2026 23:08
Adds `hermes kanban promote <task_id>` for manual lifecycle recovery
when an auto-promote daemon misses the parent-done transition (issue
NousResearch#28822). Refuses promotion unless every parent dep is done/archived
(override with --force). Emits a `promoted_manual` audit event distinct
from the automatic `promoted` kind, so audit consumers can filter
human-driven from system-driven promotions. Supports --dry-run and
--json for orchestration. Does not mutate assignee/claim state — the
dispatcher picks the card up via its normal ready polling path.

Closes NousResearch#28822.
…29464

Adds an --ids flag to 'hermes kanban promote' mirroring the existing
block/schedule convention, so the marquee use case from issue NousResearch#28822
(promote all children of a closed organizational parent in one shot)
doesn't require a shell loop. Single-id JSON output stays a flat
object for back-compat; bulk emits a list. Dedupes positional + --ids
so the same id can't be promoted twice in one call. 5 new CLI-level
tests cover bulk happy path, partial-failure exit code, JSON shapes,
and dedup.

Also adds the thedavidmurray noreply-email -> github-login mapping in
scripts/release.py so the salvage cherry-pick passes the AUTHOR_MAP
contributor-credit check.
…prompt hint (NousResearch#31290)

* fix(profiles): cross-profile soft guard on file-write tools + system-prompt hint

Adds a soft guard so an agent running under one Hermes profile cannot
silently edit a different profile's skills/plugins/cron/memories.
Three layers:

A. agent/file_safety.classify_cross_profile_target
   Classifies a write target against the active HERMES_HOME. Returns
   a {active_profile, target_profile, area, target_path} dict when the
   path lands in another profile's scoped area. PROFILE_SCOPED_AREAS =
   (skills, plugins, cron, memories). get_cross_profile_warning()
   wraps it into a model-facing error string that names both profiles,
   names the area, and points at the cross_profile=True bypass.

   Defense-in-depth, NOT a security boundary — the terminal tool runs
   as the same OS user and can write any of these paths directly. The
   guard exists to prevent confused-agent corruption, not to stop a
   determined attacker. SECURITY.md §3.2 (terminal-bypass posture)
   still applies.

   Wired into tools/file_tools.write_file_tool and patch_tool with a
   cross_profile=False kwarg. WRITE_FILE_SCHEMA and PATCH_SCHEMA both
   advertise cross_profile so the model can pass it after explicit
   user direction. patch_tool extracts target paths from V4A patch
   bodies before checking (same shape as the existing sensitive-path
   check).

   skill_manage is already scoped to the active profile's SKILLS_DIR
   by construction, so no extra guard wiring is needed there. The
   D-side error message (below) still names other profiles when the
   skill exists elsewhere.

B. agent/system_prompt
   One deterministic line near the environment-hints block names the
   active profile and tells the model not to modify another profile's
   skills/plugins/cron/memories without explicit direction. Profile
   name is stable for the lifetime of the AIAgent, so the line is
   prompt-cache-safe.

D. tools/skill_manager_tool._skill_not_found_error
   Replaces the bare "Skill 'X' not found." with a message that:
     - names the active profile,
     - searches OTHER profiles' skills dirs for the same name,
     - names the profile(s) where the skill exists and the path,
     - suggests `hermes -p <name>` to switch profiles, or
       cross_profile=True for an explicit edit.

   All 5 "not found" sites in skill_manager_tool (edit, patch, delete,
   write_file, remove_file) now go through the helper.

Reference incident (May 2026): a hermes-security profile session
edited skills under both ~/.hermes/profiles/hermes-security/skills/
AND ~/.hermes/skills/ (the default profile's skills) without
realizing the second path belonged to a different profile. Three of
the four skill files needed manual restoration afterward.

What this PR does NOT do:

  * No hard block. The terminal tool can still touch any of these
    paths with no guard — same posture as the dangerous-command
    approval flow. SECURITY.md §3.2 applies.
  * No regex sweep on terminal commands for cross-profile paths.
    That direction is a Skills-Guard-style arms race (cd + relative
    paths, base64, etc.) and would false-positive on legitimate
    cross-profile reads. Filed as a follow-up.
  * No on-disk path migration. ~/.hermes/skills/ remains the
    default profile's skills dir; this PR is about telling the
    agent about that boundary, not changing the layout.

Tests:
  tests/agent/test_file_safety_cross_profile.py (16 tests)
    - _resolve_active_profile_name covers default/named/failure paths
    - classify_cross_profile_target covers all four scoped areas,
      both directions (default → named, named → default, named → named),
      non-Hermes paths, and root-level config files
    - get_cross_profile_warning covers in-profile no-op, cross-profile
      message shape, and the defense-in-depth self-documentation

  tests/tools/test_cross_profile_guard.py (12 tests)
    - write_file: in-profile allow, cross-profile block, cross_profile=True
      bypass, non-Hermes pass-through
    - patch: replace-mode block, cross_profile=True bypass, V4A patch
      path extraction
    - skill_manage: error names the other profile (single + multiple),
      missing-everywhere falls back to skills_list hint
    - system prompt: contract-level checks (both branches present,
      cross_profile=True mentioned, ~/.hermes/profiles/ referenced)

All 207 existing tests in file_safety/file_operations/skill_manager
still pass. 10 system-prompt tests still pass.

E2E verified: the exact incident scenario (security profile editing
default's hermes-agent-dev skill) is now blocked with the warning
message; cross_profile=True unblocks.

* fix(code_execution): add cross_profile to write_file/patch stubs

The cross_profile kwarg added to write_file_tool/patch_tool needs to
flow through the execute_code sandbox stubs in _TOOL_STUBS so the
test_stubs_cover_all_schema_params drift test passes. Without this,
scripts running inside execute_code couldn't pass cross_profile=True
through hermes_tools.write_file().

Caught by CI on PR NousResearch#31290.
When WeCom returns errcode=40001 (invalid credential) or 42001 (token
expired), send() was returning a failure without evicting the bad token
from _access_tokens. All subsequent sends then kept using the same
invalid cached token until its TTL naturally expired (~7200s).

Fix: on the first token-rejection errcode, evict the cache entry and
retry once with a freshly fetched token. Non-token errcodes fail
immediately as before. If the refreshed token also fails, the error
is returned without looping further.

Adds four regression tests covering: successful retry on 40001,
successful retry on 42001, no retry on unrelated errcode, and clean
failure when the refresh does not help.
CodeQL py/clear-text-logging-sensitive-data flagged the candidate-accept
debug log including event.text[:60]. Log text_len instead — sufficient for
debugging burst behavior without surfacing message contents.

Co-authored-by: Paulo Nascimento <pnascimento9596@gmail.com>
…message loss

When asyncio.sleep() fires just before Task.cancel() is called, CPython
sets _must_cancel=True but cannot cancel the already-completed sleep
future, so CancelledError is delivered at the next await (handle_message)
rather than at the sleep.  By that point the superseded task has already
popped the merged event from _pending_text_batches, so the superseding
task sees an empty batch and silently drops the message.

Fix: add a synchronous task-registry check between the sleep and the pop.
No await between the check and the pop means no other coroutine can
interleave, so the guard is race-free.
…ousResearch#31379)

PR #6a1aa420e coupled `display.tool_progress: verbose` (a per-tool display
toggle for full args / results / think blocks) to `self.verbose` — which
controls root-logger DEBUG level. Result: setting tool_progress: verbose
in config silently flipped every module in the process to DEBUG and
flooded the terminal with internal logging, far beyond just full tool
calls.

The two concepts are separate:
- `tool_progress_mode == 'verbose'` → display behavior (tool rendering)
- `self.verbose` → logging behavior (root logger → DEBUG, line 9795)

This change keeps PR #6a1aa420e's argparse.SUPPRESS / config-fallback
plumbing but severs the verbose-display → debug-logging link.

Changes:
- cli.py:2868 — `self.verbose` only follows explicit `verbose=` arg; no
  longer auto-True when tool_progress_mode == 'verbose'.
- cli.py:_toggle_verbose — slash-cycle through tool progress modes no
  longer flips `self.verbose` / `agent.verbose_logging` / `agent.quiet_mode`.
- cli.py:9355 — fix misleading label (drop 'and debug logs').
- tui_gateway/server.py:_make_agent — same decoupling on the TUI side
  (verbose_logging no longer derived from tool_progress_mode).
- tests/cli/test_tool_progress_scrollback.py — invert the test that
  asserted the broken coupling; add coverage for explicit `--verbose`
  still enabling DEBUG independent of tool_progress.

Live verified:
- tool_progress: verbose, no --verbose flag → 0 DEBUG/INFO log lines
- --verbose flag explicit → 32 DEBUG/INFO log lines (as expected)
…ousResearch#31378)

Closes NousResearch#31370.

bws defaults to the US identity endpoint, so EU Cloud and self-hosted
machine-account tokens fail with [400 Bad Request] {"error":"invalid_client"}
during 'hermes secrets bitwarden setup'. The token is valid — it's just
being checked against the wrong region.

Add a Bitwarden region step to the wizard between the access-token and
project-list steps:

  Step 1  Install bws
  Step 2  Provide access token
  Step 3  Pick region   <-- new (US / EU / self-hosted-custom-URL)
  Step 4  Pick project  (now talks to the right endpoint)
  Step 5  Test fetch

Region is stored in config.yaml as secrets.bitwarden.server_url and
plumbed into every bws subprocess as BWS_SERVER_URL (project list,
secret list, test fetch, and the env_loader startup pull).

Also:
- Non-interactive: 'hermes secrets bitwarden setup --server-url ...'
- Pre-existing BWS_SERVER_URL in the shell is detected and reused
- Cache key includes server_url so EU/US fetches don't collide
- 'hermes secrets bitwarden status' shows the configured region
- 'invalid_client' / '400 Bad Request' from bws now triggers a hint
  pointing at the region setting instead of looking like a bad token
…ut hook now visible

When streaming is active, streamed_message=True skipped the final_response
update, causing plugin hooks like transform_llm_output to be silently
invisible. Remove the `not streamed_message` guard so the final response
(possibly transformed by plugins) is always delivered to the ACP client.
…s streaming suppression

When a transform_llm_output hook modifies final_response after streaming,
the gateway was silently discarding the transformed content because
streamed=True / content_delivered=True triggered the final-send
suppression. Three changes:

1. conversation_loop: set `_response_transformed=True` when a
   transform_llm_output hook returns a non-empty string, and expose it
   as `response_transformed` in the result dict.

2. gateway/run: skip the final-send suppression when
   `response_transformed` is True — the transformed response must
   reach the client even if streaming already sent the original text.

3. acp_adapter/server: remove `not streamed_message` guard so
   final_response is always delivered (ACP path fixed separately).
…turn dict

run_sync() cherry-picks fields from the run_conversation result dict into
a new response dict for the gateway. response_transformed was missing from
the cherry-pick list, so the gateway always saw it as False and suppressed
the final send even though a transform_llm_output hook had modified the content.
… response_transformed

When a transform_llm_output hook appends content after streaming, the previous
fix skipped the final-send suppression which caused the full response to be
sent as a NEW message (duplicate). Instead, edit the existing streamed message
in-place to append the transformed content, then set already_sent=True.

Added stream_consumer.message_id and .accumulated_text public properties.
…ming

Adds a test that fails without the gateway fix, exercising the
response_transformed=True branch in _finalize_response: a streamed
response whose final text was modified by a transform_llm_output
plugin hook must be edit_message'd in place (not duplicate-sent),
with already_sent=True so the normal final-send is skipped.

Also drops two minor leftovers from the salvaged PR NousResearch#29119:

  * accumulated_text property on GatewayStreamConsumer (unused)
  * duplicate _response_transformed=False inside the hook try block
PR NousResearch#29119 dropped the 'not streamed_message' guard unconditionally so
that plugin-transformed responses (transform_llm_output hook) would
reach ACP clients. That regressed test_prompt_does_not_duplicate_streamed_final_message:
when no transform happened, the streamed text was re-sent as a duplicate
final delivery.

Tighten the condition to mirror the gateway side: deliver after streaming
only when response_transformed=True. Otherwise keep the old guard.

Adds test_prompt_delivers_transformed_response_after_streaming so the
transformed path stays covered.
… stub

When the API connection drops mid-stream after text deltas have already
been delivered, chat_completion_helpers returned a stub response with
finish_reason=stop. The conversation loop then classified the stub as a
clean text completion (text_response(finish_reason=stop)) and exited
with iteration budget remaining — even when the goal-judge verdict
came back as "continue" milliseconds later (issue NousResearch#30963).

Switch the text-only partial-stream stub to finish_reason=length. The
existing length-continuation path (length_continue_retries up to 3,
"continue exactly where you left off" prompt, partial parts merged
into final_response) then fires automatically: the partial assistant
content is persisted, the model is asked to continue from the cut
point, and the loop keeps making progress against the goal.

The mid-tool-call branch keeps finish_reason=stop on purpose — its
user-facing warning ("Ask me to retry if you want to continue") asks
the user to drive the retry rather than auto-replaying a tool call
with possible side effects.

NousResearch#5544's "no duplicate message" contract is preserved verbatim: the
partial content is reused, never re-emitted as a fresh API call, so
the user never sees two copies of the same delta.

Refs: NousResearch#30963
teknium1 and others added 20 commits May 25, 2026 14:54
Salvage follow-up. The transient thread-not-found retry test was
exercising chat_id='123' (positive, looks-like-private) which now
hits the new private-DM-topic fail-closed contract. The test's
intent is the transient-flake retry on real forum topics in groups,
so use -100123 to make the scenario unambiguous.
…quired guard

Salvage follow-up. The new private-DM-topic fail-loud contract from
PR NousResearch#27107 hits 'requires a reply anchor' when reply_to_mode='off' is
configured, even though commit 21a15b6 (PR NousResearch#23994) verified that
message_thread_id alone routes correctly on python-telegram-bot's
reference client when the user has explicitly opted out of quote
bubbles. Carve out the explicit opt-in path so users on reply_to_mode
'off' aren't regressed — the new guard now only applies to callers
that didn't ask for the anchor to be suppressed.
…influencing env vars (NousResearch#32277)

Two posture fixes surfaced by the web-pentest skill self-test against
the dashboard (issue NousResearch#32267).

1. /dashboard-plugins/<name>/<path> previously returned 200 for any
   file inside the plugin's dashboard directory — including
   plugin_api.py and __pycache__/*.pyc. The path is unauthenticated by
   architecture (SPA loads JS via <script src> and CSS via <link href>,
   neither of which can attach a custom auth header), so the fix is
   not "require token" — it's "restrict to browser-fetchable suffixes."
   Allowlist now: .js .mjs .css .json .html .svg .png .jpg .jpeg .gif
   .webp .ico .woff .woff2 .ttf .otf .map. Everything else → 404.

   This stops a private user-installed plugin's Python source from
   being readable by anyone reachable on the dashboard's loopback port
   (other local users on a shared box, sidecar containers sharing the
   host netns).

2. save_env_value() now refuses to persist env-var names that
   influence how the next subprocess executes: LD_PRELOAD,
   LD_LIBRARY_PATH, LD_AUDIT, DYLD_*, PYTHONPATH, PYTHONHOME,
   PYTHONSTARTUP, NODE_OPTIONS, NODE_PATH, PATH, SHELL, EDITOR,
   VISUAL, PAGER, BROWSER, GIT_SSH_COMMAND, GIT_EXEC_PATH; plus
   HERMES_HOME / HERMES_PROFILE / HERMES_CONFIG / HERMES_ENV.

   PUT /api/env is authed but the session token lives in the SPA HTML
   where any future plugin XSS or local process can read it. Without
   this gate, a token-holder could plant LD_PRELOAD in .env and the
   next hermes process start would load attacker code via the dotenv
   to os.environ chain. This is enforced on write only — pre-existing
   .env values are left alone (the gate is in save_env_value, not in
   load_env). PUT /api/env now returns 400 with the explanatory
   message instead of an opaque 500.

   IMPORTANT: HERMES_* overall is NOT blocked — only the four runtime
   location names. Integration credentials following the HERMES_*
   convention (HERMES_GEMINI_*, HERMES_LANGFUSE_*, HERMES_SPOTIFY_*,
   HERMES_QWEN_BASE_URL, ...) keep working.

Regression tests cover both fixes (30 new test cases). No existing
tests changed; 257 passing in tests/hermes_cli/.

Closes NousResearch#32267.
…search#32264)

The outer 'except Exception' guard in run_conversation() captures
exceptions raised inside the agent loop (during streaming, tool
dispatch, message construction, etc.) and prints a one-line summary
to the screen.  The traceback was only logged at DEBUG, so it never
landed in errors.log (WARNING+) and was lost.

For intermittent failures — the most important kind to debug — users
saw 'Error during OpenAI-compatible API call #N: <message>' on
screen with no way to recover the call site.  Switching to
logger.exception() emits the full traceback at ERROR so it goes to
both agent.log and errors.log automatically.

This is a pure logging change; control flow is unchanged.
…ilure escalation (NousResearch#507) (NousResearch#32273)

Three granular patch-tool refinements from the Roo Code deep-dive (NousResearch#507).

## Indentation preservation (fuzzy_match.py)

When fuzzy_find_and_replace matches via a non-exact strategy, the file's
indentation may differ from what the LLM sent in old_string/new_string
(common case: model sends zero-indent old/new for a method body that
lives inside an 8-space-indented class). Before this commit the
replacement was spliced in verbatim, producing a file with a broken
indent level that may still parse but is logically wrong.

The fix computes the indent delta between old_string's first meaningful
line and the matched region's first meaningful line, then re-indents
every line of new_string by that delta. Exact-strategy matches are
untouched (passthrough). Same approach as Roo Code's
multi-search-replace.ts:466-500.

## CRLF preservation (file_operations.py)

Models nearly always send tool args with bare LF endings (JSON-encoded),
but the file on disk may have CRLF (Windows-line-ending configs, .bat,
.cmd, .ini files). Before this commit:

- write_file silently normalized CRLF to LF on every overwrite
- patch produced mixed-ending files: the substituted region had LF,
  the surrounding context kept CRLF

The fix detects the file's existing line endings (via pre_content if
already read for lint/LSP, otherwise a tiny head -c 4096 probe), and
normalizes the entire write to that ending. New files are written
verbatim (no detection possible).

## Per-file failure escalation (file_tools.py)

When the agent fails to patch the same file 3+ times in a row, the
existing 'old_string not found' hint isn't strong enough — the model
keeps retrying with variations against a stale view of the file.

The fix tracks consecutive failures per (task_id, resolved_path) and
injects an escalating hint after 3 failures: 'This is failure #N
patching X. Stop retrying. Either re-read fresh, use longer context,
or fall back to write_file.' Counter resets on a successful patch to
the same path.

## Validation

- 22 new tests across tests/tools/test_fuzzy_match.py (5),
  test_line_ending_preservation.py (12), test_patch_failure_tracking.py (5)
- All existing tests pass (165/165 in the touched files)
- E2E verified with real _handle_patch / _handle_write_file calls
  against real CRLF files and real failure loops

Closes part of NousResearch#507. The remaining open items in NousResearch#507 (2b start_line
hint, behavioral rules) were declined after audit:
- 2b adds schema bloat for a problem the existing 'multiple matches'
  contract already handles
- Behavioral rules conflict with the personality system

Items 1, 2d, 2e, 3, 4 of NousResearch#507 were already landed in earlier work.
…ocess (NousResearch#32271)

`load_hermes_dotenv()` is called at module-import time from cli.py,
hermes_cli/main.py, run_agent.py, trajectory_compressor.py, gateway/run.py,
tui_gateway/server.py, acp_adapter/entry.py, and a few others. Each call
triggered `_apply_external_secret_sources()`, which re-parsed config,
re-fetched from Bitwarden Secrets Manager (its own 300s cache mostly absorbed
this), re-ran the ASCII sanitization sweep, and reprinted

  Bitwarden Secrets Manager: applied N secret(s) (...)

to stderr. Users saw the status line 3-5x per CLI startup.

Guard the function with a process-level set of HERMES_HOME paths that have
already had external secrets applied. Subsequent calls for the same home_path
are no-ops. `reset_secret_source_cache()` lets tests (and any future
long-running consumer that wants to refresh after a config change) force a
re-pull.
…ersist (NousResearch#32272)

Reported via AskClaw. When config.yaml has `model: <name>` (flat string)
instead of the nested `model: {default: ..., provider: ...}` form, every
gateway `/model X --global` crashed silently with

    TypeError: 'str' object does not support item assignment

The persist block did:

    model_cfg = cfg.setdefault("model", {})
    model_cfg["default"] = result.new_model

`setdefault` returns the existing scalar, and the next assignment blows
up. The 'switch failed' warning was logged at WARNING level and the user
never saw why their persist didn't stick.

Coerce scalar/None `model:` into a dict before mutation, in both the
gateway path (`gateway/run.py`) and the sister site in
`hermes_cli/doctor.py --fix` (same setdefault-on-string flaw). The CLI
`/model` path is unaffected because it goes through `_set_nested` which
already replaces scalar leaves with dicts.

Regression test `tests/gateway/test_model_command_flat_string_config.py`
covers the flat-string, missing, and proper-dict cases. Without the fix,
the flat-string case fails with the exact original TypeError.
… entries

When the user picks 'Anthropic API key' at `hermes setup` (vs 'Claude
Pro/Max subscription'), `save_anthropic_api_key()` writes ANTHROPIC_API_KEY
to ~/.hermes/.env and zeros ANTHROPIC_TOKEN.  That env-var pattern is the
user's explicit choice of auth method — API key, not OAuth.

But the anthropic credential pool's autodiscovery (_seed_from_singletons)
unconditionally read ~/.claude/.credentials.json from the Claude Code CLI
and any saved hermes_pkce creds, and added them to the SAME anthropic
pool as the user's API key.  Two problems:

  1. Even with the API key at higher priority, a 401/429 on the API key
     would rotate the session onto an autodiscovered OAuth credential,
     silently flipping the agent into the Claude Code masquerade
     mid-conversation: 'You are Claude Code' system block, every tool
     renamed to mcp_*, claude-cli User-Agent header.

  2. Switching OAuth → API key at `hermes setup` cleared the env vars
     but left previously-seeded OAuth entries dormant in auth.json,
     where rotation could revive them.

The user picking the API-key path is explicitly opting OUT of the
masquerade.  Mixing OAuth credentials into their pool defeats that
choice.

Fix: in `_seed_from_singletons` for provider='anthropic', detect the
API-key path (ANTHROPIC_API_KEY set in env, no OAuth env var set) and:
  - Skip calling read_claude_code_credentials() and
    read_hermes_oauth_credentials() entirely
  - Prune any stale hermes_pkce / claude_code entries that may already
    be in the on-disk pool

OAuth-path users (ANTHROPIC_TOKEN set) are unaffected — autodiscovery
continues to fire as before.

Tests: 3 new regression tests (api-key skips autodiscovery, api-key
prunes stale entries, oauth path still autodiscovers).  Full file 70/70.
…sitiving (NousResearch#32339)

The runtime cron prompt scanner (added in NousResearch#3968 to plug the
"malicious skill carrying an injection payload" gap) reuses the same
critical-severity patterns as the create-time user-prompt scan against
the *assembled* prompt — which includes loaded skill markdown.

That works fine for narrow patterns like "ignore previous instructions"
which never legitimately appear in prose. It catastrophically false-
positives on command-shape patterns like `cat ~/.hermes/.env`,
`authorized_keys`, `/etc/sudoers`, and `rm -rf /`, which routinely
appear in security postmortems and runbooks as **descriptive prose**
about attacks, not as actual commands.

Concrete failure: the bundled `hermes-agent-dev` skill contains a
security postmortem section saying "the attacker could just
`cat ~/.hermes/.env`". Every PR-scout cron job that loaded this skill
was silently blocked with `Blocked: prompt matches threat pattern
'read_secrets'`. All 11 scout jobs failed for weeks.

Fix: split the scanner into two tiers and route by context:

  - `_scan_cron_prompt` (strict, unchanged behavior) runs against
    the small user-authored cron prompt at create/update and as a
    runtime defense-in-depth when no skills are attached. A legit
    user prompt has no business saying `cat .env`, so the strict
    patterns still apply there.

  - `_scan_cron_skill_assembled` (new, looser) runs against the
    assembled prompt when skills are attached. It only catches
    unambiguous prompt-injection directives ("ignore previous
    instructions", "disregard your rules", "system prompt override",
    "do not tell the user") plus invisible-unicode markers. Command-
    shape patterns are dropped because they false-positive on prose.

This is defense-in-depth, not the only line of defense. Skill bodies
are already scanned at install time by `skills_guard.py`; the runtime
cron scan exists purely as a tripwire for an obvious injection
directive surviving a malicious install. Catching prose mentions of
commands was never the goal of NousResearch#3968 — the test that planted a skill
containing `cat ~/.hermes/.env` was the wrong shape of test for the
threat model.

Tests:
- `_scan_cron_prompt` strict behavior preserved (56 existing tests
  unchanged: bare `cat .env`, `rm -rf /`, etc. still block).
- New `TestScanCronSkillAssembled` class verifies the looser scanner:
  injection / disregard / system-override / do-not-tell-the-user /
  invisible-unicode still block; descriptive prose about attack
  commands is allowed; GitHub auth-header allowlist still works.
- `test_skill_with_env_exfil_payload_raises` (planted `cat .env`
  in skill body) replaced with `test_skill_with_env_exfil_command
  _in_prose_is_allowed` documenting the new correct behavior with
  the real-world postmortem-style example that triggered the bug.
- All 11 originally-failing PR-scout jobs validated end-to-end via
  `_build_job_prompt` — assembled prompts now build successfully
  with the `hermes-agent-dev` skill attached.

Total: 75/75 tests in cron + cronjob_tools + threat scanner pass;
544/544 across the wider cron / memory / threat-pattern surface.
Pre-salvage prep for the must-have security cluster (NousResearch#32103, NousResearch#32155).
NousResearch#32103 author commit uses dearmayo@localhost; PR opener is ffr31mr —
same pattern as the existing holynn-q localhost mapping.
…, ClawHub, browse.sh, OpenAI, …) (NousResearch#32336)

The Skills Hub page was stuck on a stale Feb 25 snapshot, showing only Built-in
+ Optional + Anthropic + LobeHub. The unified index already has 2078 skills
from skills.sh / ClawHub / LobeHub / GitHub taps / Claude Marketplace, and
BrowseShSource adds another ~330 — none of it was reaching the page.

Changes:

- website/scripts/extract-skills.py: read website/static/api/skills-index.json
  (the unified multi-source catalog, rebuilt twice daily) as the canonical
  external source. Keep the legacy skills/index-cache/ fallback for offline
  builds. Add friendly per-source labels (skills.sh, ClawHub, browse.sh,
  OpenAI, HuggingFace, Anthropic, LobeHub, etc.) and per-entry installCmd.
- website/src/pages/skills/index.tsx: add source pills + ordering for the 11
  new sources; render installCmd from the index entry.
- website/scripts/prebuild.mjs: when no local skills-index.json exists, fetch
  the live one from hermes-agent.nousresearch.com so local 'npm run build'
  matches production without burning GitHub API quota.
- scripts/build_skills_index.py: crawl BrowseShSource so browse.sh entries
  land in the unified index. Adjust source_order.
- tools/skills_hub.py: GitHubSource.DEFAULT_TAPS — openai/skills moved its
  skills into skills/.curated/ and skills/.system/, so add both as explicit
  taps (the listing code skips dotted dirs by design). Drop
  VoltAgent/awesome-agent-skills (README-only, no SKILL.md files) and
  MiniMax-AI/cli (singular skill, not a tap directory). Net effect: github
  source jumps from 83 → 143 skills, with OpenAI properly included.
- .github/workflows/deploy-site.yml: build the unified index BEFORE running
  extract-skills.py — previous order meant extract-skills always fell back
  to the legacy cache. Drop the 'skip if file exists' guard; the file is
  gitignored and must be rebuilt every deploy.
- .github/workflows/skills-index.yml: drop the broken 'deploy-with-index'
  job (it cp'd 'landingpage/\*' which no longer exists, failing every cron
  run since the landingpage move). Replace it with a workflow_dispatch
  trigger of deploy-site.yml so the index refresh still reaches production
  on schedule.
- website/docs/user-guide/features/skills.md: drop VoltAgent from the
  default-taps doc list to match the code.

Before: 695 skills (Built-in 90, Optional 84, Anthropic 16, LobeHub 505).
After:  2168 skills across 9 source pills, including the 1212 skills.sh
        entries the user expected to see.
…NousResearch#32345)

Layered safety so the Skills Hub at /docs/skills stays in sync without
silent rot. Three pieces:

1. build_skills_index.py — refuses to ship a degenerate index.
   EXPECTED_FLOORS per source (skills.sh ≥100, lobehub ≥100, clawhub ≥50,
   official ≥50, github ≥30, browse-sh ≥50) and MIN_TOTAL=1500. Any source
   collapsing to zero (the silent OpenAI breakage that hid for weeks) now
   fails the workflow loud — broken index never reaches the live site.

2. extract-skills.py + the React page — visible freshness signal.
   Sidecar website/src/data/skills-meta.json carries the index's
   generated_at timestamp, plus per-source counts. Skills Hub renders a
   'Catalog refreshed N hours ago · auto-rebuilt twice daily' line under
   the hero copy. If the cron stalls, users see the staleness immediately.

3. .github/workflows/skills-index-freshness.yml — watchdog cron.
   Every 4 hours, fetches the live /docs/api/skills-index.json, validates
   shape, checks age (>26h is stale), checks the same per-source floors,
   and opens (or appends to) a GitHub issue when anything is off. The
   issue is title-prefixed [skills-index-watchdog] so subsequent failures
   append a comment instead of spamming new issues.

Net effect:
- A silent regression like 'OpenAI tap moved its skills' now fails the
  build instead of shipping a quietly broken catalog.
- A stuck cron (like the landingpage breakage that ran red for weeks) now
  files an issue within 4 hours.
- Users see how fresh the catalog is on the page itself.

Test plan:
- Local: built skills-meta.json from the live index → 'Catalog refreshed
  N minutes ago' rendered correctly in the static HTML.
- Probe logic dry-run against the live index: total=2456, all 6 sources
  above floor, age 0.1h — issues=NONE.
- Triggered skills-index.yml manually; both jobs green, deploy-site.yml
  dispatch fired.
…st bullet

The GFM → Telegram-row-group rewriter previously joined every line in
every row with a blank line ("\n\n".join(rendered_rows)), which made
multi-column tables explode into one-bullet-per-paragraph walls on
mobile.  It also emitted the row heading twice when the table had no
row-label column: once as the standalone bold heading and once again
as the first labeled bullet (heading == headers[0] == data_cells[0]).

This commit:

* Uses single newlines between the heading and its bullets within a
  row-group, and a blank line only BETWEEN row-groups.
* Skips any bullet whose value duplicates the heading text when the
  table has no row-label column (the heading already carries that
  information).  Tables WITH a row-label column are unaffected since
  the heading comes from the label cell and never duplicates a header.

Updated existing test assertions accordingly and added two regression
tests: one that reproduces the screenshot bug (wide five-column "Plays"
comparison table) and one that pins the row-label-column behavior so
the dedup logic doesn't accidentally swallow real data.

tests/gateway/test_telegram_format.py: 101 passed
SubdirectoryHintTracker was scanning directories outside the active
working directory, allowing files like ~/.codex/AGENTS.md or
~/.claude/CLAUDE.md to be loaded and injected into the agent context.
This causes cross-agent context contamination and instruction mixup.

Add _is_ancestor_or_same() helper and a path boundary check in
_is_valid_subdir(): only directories within the working directory tree
(i.e. path.is_relative_to(working_dir)) are allowed.

Also add exist_ok=True to mkdir() calls in new tests to prevent
pytest-xdist race conditions when workers share the same tmp_path parent.

Tests added:
- test_outside_working_dir_rejected: verifies sibling dirs are blocked
- test_outside_working_dir_absolute_path_rejected: verifies ~/.codex paths blocked
- test_inside_workspace_subdir_allowed: verifies normal subdir access unaffected
- test_sibling_repo_not_loaded_via_ancestor_walk: ancestor walk stays within workspace
…sedxml

Two small defensive-hardening changes:

- web/src/components/Markdown.tsx: render links only for http(s)/mailto
  schemes; other schemes (javascript:, data:, vbscript:) are dropped to
  plain text so a crafted link in rendered content can't execute on click.

- gateway/platforms/wecom_callback.py: parse the untrusted, pre-auth WeCom
  callback request body with defusedxml instead of xml.etree, blocking
  entity-expansion / billion-laughs (and XXE) on the parse path. defusedxml
  is already a dependency (uv.lock); response-building XML in
  wecom_crypto.py is unchanged (it is not parsed from untrusted input).

Verified: dashboard typechecks and builds; defusedxml blocks an
entity-expansion payload while valid WeCom envelopes still parse.
Follow-up on top of @TheOnlyMika's NousResearch#32155 cherry-pick. The defusedxml
hardening import was unconditional, which would break the gateway for
anyone running a WeComCallback adapter without the (transitive-only)
defusedxml present.

- Wrap the import in the same try/except pattern as aiohttp/httpx in
  the same file. Sets DEFUSEDXML_AVAILABLE flag.
- Extend check_wecom_callback_requirements() to gate on the flag, so
  the gateway logs the actual missing dep and skips the adapter
  instead of crashing.
- Add [wecom] extra to pyproject.toml with defusedxml==0.7.1.
- Register platform.wecom_callback in tools/lazy_deps.py so users get
  prompted to install it on first WeComCallback configuration, same
  pattern as discord/slack/matrix.

defusedxml is still the right call for pre-auth XML parsing — this
commit just makes the dep declarative and recoverable instead of a
hard import-time crash.
…astes (NousResearch#32447)

Follow-up to NousResearch#32087 after community report from @ethernet that 8000-char
single-line pastes get dumped raw into the input box.

A) Fallback regression revert
   paste_collapse_threshold_fallback default: 0 -> 5
   NousResearch#32087 disabled the fallback handler by default. The fallback path
   has been always-on with line_count >= 5 since NousResearch#3065 (March 2026);
   the previous shape was the salvaged contributor's design and didn't
   match pre-existing behavior for terminals without bracketed paste
   support (Windows terminals, some SSH setups). Restoring the original
   on-by-default.

B) Long single-line paste guard
   New config key: paste_collapse_char_threshold (default 2000)
   Bracketed-paste handler and fallback handler now BOTH collapse when
   line count >= line threshold OR total char length >= char threshold.
   Catches the case ethernet hit: ~8000 chars of minified JSON / log
   output on a single line dumped raw into the buffer.
   TUI mirrors the same config via uiStore.pasteCollapseChars.
   Set 0 to disable.

Defaults verified:
  paste_collapse_threshold: 5
  paste_collapse_threshold_fallback: 5
  paste_collapse_char_threshold: 2000

Tests:
  tests/hermes_cli/test_config.py: 87/87 pass
  ui-tui useConfigSync.test.ts: 34/34 pass
  ui-tui useComposerState.test.ts: 9/9 pass
  tsc: 0 new errors in touched files
@github-actions

github-actions Bot commented May 26, 2026

Copy link
Copy Markdown

🔎 Lint report: feat/roadmap-tier2-2605 vs origin/main

ruff

Total: 0 on HEAD, 0 on base (➖ 0)

🆕 New issues: none

✅ Fixed issues: none

Unchanged: 0 pre-existing issues carried over.

ty (type checker)

Total: 9360 on HEAD, 9061 on base (🆕 +299)

🆕 New issues (282):

Rule Count
unresolved-attribute 86
unresolved-import 79
invalid-argument-type 60
unsupported-operator 17
invalid-assignment 15
invalid-method-override 7
unresolved-reference 5
unresolved-global 5
not-subscriptable 4
unused-type-ignore-comment 1
invalid-parameter-default 1
not-iterable 1
no-matching-overload 1
First entries
tests/gateway/test_discord_component_auth.py:23: [unresolved-import] unresolved-import: Module `plugins.platforms.discord.adapter` has no member `ModelPickerView`
plugins/platforms/discord/adapter.py:340: [unresolved-import] unresolved-import: Cannot resolve imported module `nacl.secret`
tests/agent/test_non_stream_stale_timeout.py:38: [invalid-argument-type] invalid-argument-type: Argument to `AIAgent.__init__` is incorrect: Expected `bool`, found `str | bool`
plugins/platforms/discord/adapter.py:4063: [unresolved-reference] unresolved-reference: Name `ExecApprovalView` used when not defined
tests/run_agent/test_tool_call_guardrail_runtime.py:335: [unresolved-attribute] unresolved-attribute: Unresolved attribute `_disable_streaming` on type `AIAgent`
plugins/platforms/discord/adapter.py:5377: [unresolved-attribute] unresolved-attribute: Module `discord` has no member `SelectOption`
plugins/platforms/discord/adapter.py:3806: [unresolved-attribute] unresolved-attribute: Module `discord` has no member `Forbidden`
tests/gateway/test_telegram_status_update.py:34: [unresolved-attribute] unresolved-attribute: Unresolved attribute `BadRequest` on type `ModuleType`
plugins/platforms/discord/adapter.py:1980: [unresolved-attribute] unresolved-attribute: Module `discord` has no member `FFmpegPCMAudio`
tests/run_agent/test_partial_stream_finish_reason.py:204: [unresolved-attribute] unresolved-attribute: Unresolved attribute `save_trajectories` on type `AIAgent`
tests/agent/test_non_stream_stale_timeout.py:38: [invalid-argument-type] invalid-argument-type: Argument to `AIAgent.__init__` is incorrect: Expected `int | float`, found `str | bool`
tests/cli/test_slash_confirm_windows.py:23: [unresolved-import] unresolved-import: Cannot resolve imported module `pytest`
tests/agent/test_tts_registry.py:147: [invalid-argument-type] invalid-argument-type: Argument to function `get_provider` is incorrect: Expected `str`, found `Literal[123]`
tests/tools/test_transcription_plugin_dispatch.py:21: [unresolved-import] unresolved-import: Cannot resolve imported module `pytest`
tests/tools/test_pr_6656_regressions.py:27: [unresolved-import] unresolved-import: Cannot resolve imported module `pytest`
tests/gateway/test_telegram_send_path_health.py:65: [unresolved-attribute] unresolved-attribute: Attribute `send_message` is not defined on `None` in union `Unknown | None`
tests/hermes_cli/test_nous_inference_url_validation.py:97: [invalid-argument-type] invalid-argument-type: Argument to function `_validate_nous_inference_url_from_network` is incorrect: Expected `str | None`, found `Literal[12345]`
tests/gateway/test_telegram_send_path_health.py:12: [unresolved-import] unresolved-import: Cannot resolve imported module `pytest`
plugins/platforms/discord/adapter.py:976: [invalid-argument-type] invalid-argument-type: Argument expression after ** must be a mapping type: Found `Unknown | None | dict[Unknown, Unknown]`
gateway/platforms/matrix.py:233: [unresolved-import] unresolved-import: Cannot resolve imported module `aiosqlite`
tests/gateway/test_active_session_text_merge.py:283: [invalid-assignment] invalid-assignment: Object of type `def _fake_start(event, session_key, *, interrupt_event=None) -> Unknown` is not assignable to attribute `_start_session_processing` of type `def _start_session_processing(self, event: MessageEvent, session_key: str, *, interrupt_event: Event | None = None) -> bool`
tests/gateway/test_wecom.py:911: [invalid-assignment] invalid-assignment: Object of type `def fake_handle(evt) -> CoroutineType[Any, Any, Unknown]` is not assignable to attribute `handle_message` of type `def handle_message(self, event: MessageEvent) -> CoroutineType[Any, Any, None]`
plugins/platforms/discord/adapter.py:5628: [unused-type-ignore-comment] unused-type-ignore-comment: Unused blanket `type: ignore` directive
gateway/platforms/api_server.py:2582: [invalid-argument-type] invalid-argument-type: Argument to function `create_job` is incorrect: Expected `bool`, found `Unknown | LiteralString | dict[str, str]`
gateway/platforms/matrix.py:272: [invalid-assignment] invalid-assignment: Object of type `None` is not assignable to `def ensure_and_bind(feature: str, importer: () -> dict[str, Any], target_globals: dict[Unknown, Unknown], *, prompt: bool = False) -> bool`
... and 257 more

✅ Fixed issues (126):

Rule Count
unresolved-attribute 48
invalid-argument-type 31
unresolved-import 15
invalid-assignment 12
invalid-method-override 7
unresolved-global 5
unresolved-reference 5
invalid-parameter-default 1
unused-type-ignore-comment 1
unsupported-operator 1
First entries
gateway/platforms/discord.py:1816: [unresolved-attribute] unresolved-attribute: Attribute `fetch_channel` is not defined on `None` in union `Unknown | None`
gateway/platforms/discord.py:383: [unresolved-import] unresolved-import: Cannot resolve imported module `davey`
cli.py:9369: [invalid-argument-type] invalid-argument-type: Argument to bound method `AIAgent._flush_messages_to_session_db` is incorrect: Expected `list[dict[Unknown, Unknown]]`, found `None`
gateway/platforms/discord.py:1978: [unresolved-attribute] unresolved-attribute: Attribute `FFmpegPCMAudio` is not defined on `None` in union `Unknown | None`
gateway/platforms/mattermost.py:352: [invalid-method-override] invalid-method-override: Invalid override of method `send_image_file`: Definition is incompatible with `BasePlatformAdapter.send_image_file`
cli.py:8351: [invalid-argument-type] invalid-argument-type: Argument to `AIAgent.__init__` is incorrect: Expected `str`, found `str | None`
gateway/platforms/discord.py:4205: [invalid-argument-type] invalid-argument-type: Argument to constructor `int.__new__` is incorrect: Expected `str | Buffer | SupportsInt | SupportsIndex | SupportsTrunc`, found `Any | None | str`
gateway/platforms/discord.py:5002: [unresolved-global] unresolved-global: Invalid global declaration of `SlashConfirmView`: `SlashConfirmView` has no declarations or bindings in the global scope
gateway/platforms/discord.py:2686: [invalid-method-override] invalid-method-override: Invalid override of method `send_document`: Definition is incompatible with `BasePlatformAdapter.send_document`
cli.py:9843: [unresolved-attribute] unresolved-attribute: Object of type `AIAgent` has no attribute `tools`
gateway/platforms/discord.py:3804: [unresolved-attribute] unresolved-attribute: Attribute `Forbidden` is not defined on `None` in union `Unknown | None`
cli.py:14497: [invalid-argument-type] invalid-argument-type: Argument to bound method `AIAgent.run_conversation` is incorrect: Expected `str`, found `Any | list[dict[str, Any]] | str`
cli.py:14490: [invalid-assignment] invalid-assignment: Object of type `Literal[True]` is not assignable to attribute `suppress_status_output` on type `AIAgent | None`
gateway/platforms/mattermost.py:867: [invalid-argument-type] invalid-argument-type: Argument is incorrect: Expected `list[str]`, found `(list[str] & ~AlwaysFalsy) | None`
gateway/platforms/discord.py:926: [invalid-argument-type] invalid-argument-type: Argument to function `atomic_json_write` is incorrect: Expected `int`, found `None`
cli.py:13954: [unresolved-attribute] unresolved-attribute: Attribute `interrupt` is not defined on `None` in union `AIAgent | None`
cli.py:8344: [invalid-argument-type] invalid-argument-type: Argument to `AIAgent.__init__` is incorrect: Expected `list[str]`, found `list[str] | set[str]`
cli.py:9873: [unresolved-attribute] unresolved-attribute: Object of type `AIAgent & ~AlwaysFalsy` has no attribute `tools`
tests/run_agent/test_plugin_context_engine_init.py:86: [unresolved-attribute] unresolved-attribute: Object of type `bound method _StubEngine.update_model(model: str, context_length: int, base_url: str = "", api_key: str = "", provider: str = "") -> None` has no attribute `call_args`
gateway/platforms/discord.py:1814: [unresolved-attribute] unresolved-attribute: Attribute `get_channel` is not defined on `None` in union `Unknown | None`
cli.py:9306: [unresolved-attribute] unresolved-attribute: Object of type `AIAgent & ~AlwaysFalsy` has no attribute `compression_enabled`
gateway/platforms/discord.py:5002: [unresolved-global] unresolved-global: Invalid global declaration of `ClarifyChoiceView`: `ClarifyChoiceView` has no declarations or bindings in the global scope
tests/run_agent/test_plugin_context_engine_init.py:62: [invalid-assignment] invalid-assignment: Object of type `MagicMock` is not assignable to attribute `update_model` of type `def update_model(self, model: str, context_length: int, base_url: str = "", api_key: str = "", provider: str = "") -> None`
gateway/platforms/discord.py:167: [invalid-parameter-default] invalid-parameter-default: Default value of type `None` is not assignable to annotated parameter type `set[Unknown]`
cli.py:4586: [invalid-argument-type] invalid-argument-type: Argument to `AIAgent.__init__` is incorrect: Expected `list[str] | None`, found `Unknown | None | str | list[str | Unknown]`
... and 101 more

Unchanged: 4675 pre-existing issues carried over.

Diagnostics are surfaced as warnings — this check never fails the build.

@xbrrr xbrrr merged commit fe6b7c9 into main May 26, 2026
26 checks passed
@xbrrr xbrrr deleted the feat/roadmap-tier2-2605 branch May 26, 2026 20:49
xbrrr pushed a commit that referenced this pull request Jun 12, 2026
…ookies

Mission-control style deploys reverse-proxy the dashboard at a path
prefix (e.g. mission-control.tilos.com/hermes/* -> :9119) and inject
X-Forwarded-Prefix: /hermes on every request. The SPA mount already
honoured this for asset URLs and the bootstrap __HERMES_BASE_PATH__,
but the OAuth gate didn't:

  1. The gate's Location: header to /login and the 401 envelope's
     login_url were built bare ("/login?next=..."). Under a /hermes
     prefix the browser follows that to mission-control.tilos.com/login
     which the proxy doesn't route to the dashboard.
  2. _redirect_uri (the OAuth callback URL handed to the IDP) used
     request.url_for() which doesn't honour X-Forwarded-Prefix
     (Starlette/uvicorn only proxy_headers Host + Proto + For). The
     IDP redirects back to /auth/callback instead of /hermes/auth/
     callback → 404 in the user's browser.
  3. Cookies were set with Path=/ which leaks them to other apps on
     the same origin and won't be sent back on requests under the
     prefix in the first place.

Fix threads the normalised prefix through every boundary:

  * New hermes_cli/dashboard_auth/prefix.py — single source of truth
    for X-Forwarded-Prefix parsing. web_server._normalise_prefix
    becomes a re-export so the SPA mount, the gate, and the cookies
    helper all agree.
  * middleware._unauth_response builds login_url = f"{prefix}/login".
  * routes._redirect_uri splices the prefix into the path component
    of the IDP-bound URL (with full validation of the header).
  * cookies.{set,clear}_{session,pkce}_cookie now take prefix="".
    Path attribute switches to /hermes when set; cookie name switches
    name variant (see below). Every caller passes the request's
    normalised prefix.

Cookie hardening (Teknium's lesser-note #1 in the PR review): adopt
the __Host- / __Secure- cookie name prefixes per draft-west-cookie-
prefixes. The variant is selected from (use_https, prefix):

  * Loopback HTTP → bare "hermes_session_at" (both prefixes require
    Secure, incompatible with HTTP).
  * HTTPS, direct deploy (Path=/) → "__Host-hermes_session_at".
    Strongest spec: bound to exact origin, no Domain attribute, Secure
    required.
  * HTTPS, behind a proxy prefix (Path=/hermes) →
    "__Secure-hermes_session_at". __Host- forbids Path != "/"; the
    explicit Path=/hermes covers same-origin app isolation.

Setter and reader BOTH consult the prefix because the cookie *name*
changes — a reader that looked up the bare name when the setter wrote
__Secure- would never find the value. The reader falls back across
all three variants so a request whose shape changed mid-session (e.g.
post-deploy from no-prefix to /hermes) still picks up the existing
cookie until it expires.

Test coverage:

  - tests/hermes_cli/test_dashboard_auth_prefix.py — new file. 11 tests
    pinning:
      • Location: /hermes/login on the gate's HTML redirect
      • 401 envelope login_url carries the prefix
      • Malformed X-Forwarded-Prefix is ignored (header-injection
        defence; the script-tag value is normalised to empty string)
      • _redirect_uri splices /hermes into the path (the property
        that prevents the IDP-returns-to-404 failure)
      • PKCE cookie uses Path=/hermes + __Secure- when proxied
      • Session cookies use __Host- when direct, __Secure- when
        proxied, bare on loopback HTTP
      • End-to-end round trip with hand-managed PKCE cookie carriage
        (TestClient can't simulate a Path=/hermes cookie automatically)
  - tests/hermes_cli/test_dashboard_auth_cookies.py — rewritten to pin
    each (use_https, prefix) shape produces its expected cookie name,
    plus reader-side coverage that __Host- and __Secure- variants are
    both recognised.
  - Existing tests across middleware / 401-reauth / etc. updated to
    match the new cookie names (substring contains instead of
    startswith).

Mutation-tested: reverting _unauth_response to build the bare
"/login" URL trips exactly the two tests that pin the prefix
carriage, confirming the suite discriminates the regression.
xbrrr pushed a commit that referenced this pull request Jun 12, 2026
Two CI flakes surfaced on PR NousResearch#34572 (both in files this PR doesn't touch;
pre-existing host-dependent flakes):

1. test_process_registry::TestPopenLeakOnSetupFailure — the failure-cleanup
   tests use a fake proc.pid (8888/9999) and assert proc.kill() runs. But
   spawn_local's primary cleanup is os.killpg(os.getpgid(pid), SIGKILL),
   falling back to proc.kill() only on ProcessLookupError/PermissionError/
   OSError. When the fake PID happens to exist on a busy host, os.getpgid
   succeeds, os.killpg fires against an UNRELATED real process group, and
   proc.kill() is never reached -> flaky AssertionError (and a real risk of
   SIGKILLing an innocent process group from a unit test). Patch os.getpgid
   to raise ProcessLookupError so the fallback path runs deterministically
   and no real killpg is ever issued.

2. test_web_server::test_resize_escape_is_forwarded — the receive loop calls
   the blocking conn.receive_bytes() with no exception guard. Once the child
   prints its winsize and exits, the PTY closes; on a missed-marker run the
   next recv blocks until the 30s pytest-timeout instead of failing fast.
   Add a try/except break (matching the working sibling tests) and bump the
   child's pre-read sleep 0.15s -> 0.5s so the resize reliably lands first.

Verified: 4/4 pass across 3 consecutive runs; root cause for #1 reproduced
(os.getpgid(1) succeeds -> old code skips proc.kill).
xbrrr pushed a commit that referenced this pull request Jun 12, 2026
Seven Copilot inline review comments on NousResearch#37679, four worth landing
in a polish pass before merge:

1. _dispose_unused_adapter signature: 'BasePlatformAdapter' ->
   'BasePlatformAdapter | None'. The function explicitly handles
   None and the reconnect watcher calls it with None in the
   except arm, so the annotation now matches the actual contract.

2. (duplicate of #1 on a different line) — same fix.

3. except Exception in _dispose_unused_adapter — the reviewer
   asked about asyncio.CancelledError swallowing. On Python 3.8+
   (Hermes requires 3.13, see pyproject.toml), CancelledError
   inherits from BaseException, NOT Exception, so the existing
   'except Exception' does NOT swallow task cancellation. Added
   an explicit comment explaining the contract so future readers
   don't repeat the analysis. We don't re-raise because the
   watcher loop intentionally treats dispose failures as
   best-effort: a failed dispose on an unowned adapter should not
   take down the watcher that's keeping the gateway alive.

4. _response_store = None after close in api_server.py — the
   reviewer flagged this for idempotency. Decided to keep the
   non-None state intentionally: setting it to None cascades
   to ~9 callers that access self._response_store without a
   None check, and 'close() is idempotent on a closed sqlite3
   Connection' means the current code is already safe. The
   type stays stable; LSP doesn't flag a cascade of
   reportOptionalMemberAccess errors. (This matches the
   pre-existing pattern in the codebase — e.g.
   _mark_disconnected doesn't reset state to None either.)

5. _build_adapter_with_store: reviewer worried about
   disconnect() failing on the self.name property if
   __init__ wasn't called. Already handled: we set
   'adapter.platform = Platform.API_SERVER' so the
   'self.platform.value.title()' property returns
   'Api_Server' without raising. The exception-swallowing
   branch in disconnect() does call self.name via the
   logger.debug format, so this is a real path that needs
   the platform attribute, and we have it.

6. test_disconnect_closes_response_store: bare 'pytest.raises(Exception)'
   -> 'pytest.raises(sqlite3.ProgrammingError)'. The bare
   Exception matcher would silently accept AttributeError,
   OperationalError, env-related issues, etc. The specific
   exception type ('Cannot operate on a closed database') is
   the actual signal we want — proves the SQLite conn is
   closed, not just that *something* raised.

7. test_nonretryable_failure_disposes_unowned_adapter:
   assertion tightened from '>= 1' to '== 1' on
   adapter._disconnect_calls. The docstring said 'exactly once',
   the assertion now matches. Catches the hypothetical
   'watcher disposes the same adapter twice' regression that
   '>=' would have missed.
xbrrr pushed a commit that referenced this pull request Jun 12, 2026
…ch#37677)

Anthropic enforces two independent ceilings per image:
1. 5 MB encoded byte size
2. 8000 px longest side

Hermes only guarded #1. A tall screenshot (e.g. 1200x12000 at 0.06 MB)
passes every byte check but fails the pixel check, returning a
non-retryable HTTP 400 that permanently bricks the conversation thread.

Fixes:
- error_classifier: add 'image dimensions exceed' pattern to
  _IMAGE_TOO_LARGE_PATTERNS so the 400 is classified as image_too_large
  and triggers the shrink/retry path instead of falling through to
  non-retryable error.
- conversation_compression: check pixel dimensions (via Pillow) even
  when byte size is under the 4 MB target. If max(dims) > 8000, force
  shrink.
- vision_tools._resize_image_for_vision: add optional max_dimension param.
  When set, images exceeding the pixel cap are downscaled even if they're
  under the byte budget. The resize loop now checks both byte AND pixel
  limits before accepting a candidate.

Closes NousResearch#37677
xbrrr pushed a commit that referenced this pull request Jun 12, 2026
…bes + test-leak fix (NousResearch#40909)

* fix(gateway,windows): reliability — supervisor task, JOB breakaway, status --deep

Three coordinated fixes for the Windows gateway reliability story:

1. CREATE_BREAKAWAY_FROM_JOB on every detached spawn

   The 'hermes update' triggered from the Electron Desktop GUI ran inside
   Electron's job object. Without breakaway, the post-update gateway
   watcher spawned by update — already DETACHED_PROCESS — was still
   reaped when Electron's job tore down, so the gateway never came back
   after a GUI-initiated update. Adds CREATE_BREAKAWAY_FROM_JOB (0x01000000)
   to:
     - hermes_cli/_subprocess_compat.py::windows_detach_flags() — used by
       every helper that calls windows_detach_popen_kwargs(), including
       launch_detached_profile_gateway_restart()
     - The watcher subprocess's own respawn snippet in
       hermes_cli/gateway.py (inlined flags so the watcher's child
       respawn also breaks away)

   _spawn_detached() in gateway_windows.py already had the flag; this
   change brings the rest of the codebase to parity.

2. Per-minute supervisor Scheduled Task — Windows equivalent of
   systemd Restart=always

   Introduces hermes_cli/gateway_supervisor.py and registers it as a
   second Scheduled Task ('Hermes_Gateway_Supervisor', SC MINUTE /MO 1,
   LIMITED rights) alongside the existing ONLOGON task. Every minute,
   the supervisor uses the same gateway.status.get_running_pid() probe
   as 'hermes gateway status' and, if no gateway is alive, calls
   gateway_windows._spawn_detached() (which now includes BREAKAWAY) to
   bring one back.

   Covers every crash mode, not just 'machine rebooted': taskkill,
   OOM, GUI update SIGTERM, parent job teardown. Cheap — one pythonw
   startup per minute when down, one PID-existence check per minute
   when up.

   Wired into both the schtasks-success and Startup-folder-fallback
   install paths via _install_supervisor_best_effort(), and removed in
   uninstall(). Best-effort: a failing supervisor install logs a
   warning but doesn't roll back the primary install.

3. 'hermes gateway status --deep' shows per-probe PASS/FAIL

   Replaces the existing terse '--deep' output (which only printed
   paths) with an actual diagnostic table:
     [1] PID file present
     [2] Lock file held by a live process
     [3] get_running_pid() result
     [4] _pid_exists(pid) — OS-level liveness
     [5] gateway_state.json (state + age)
     [6] Last lifecycle event from gateway-exit-diag.log

   When the high-level summary disagrees with reality, the user can
   see exactly which signal is lying.

Test-leak fix
-------------

tests/hermes_cli/test_gateway_wsl.py::TestGatewayCommandWSLMessages
monkey-patched is_linux/is_wsl/supports_systemd_services to simulate
WSL but did NOT stub is_windows(). On a Windows host, the dispatcher
in _gateway_command_inner takes the is_windows() branch BEFORE the
WSL guidance branch, so the test invoked gateway_windows.install()
for real. install() writes to %APPDATA%\...\Startup\Hermes_Gateway.cmd
— the REAL user Startup folder, never sandboxed by tmp_path — pointing
at the test's pytest-of-<user>/pytest-<N>/.../gateway-service/ wrapper.
When pytest tore down the tmp_path, every subsequent Windows login
flashed a cmd.exe window that failed to find the missing target.

Stubs is_windows=False on all four affected tests:
  test_install_wsl_no_systemd
  test_start_wsl_no_systemd
  test_status_wsl_running_manual
  test_status_wsl_not_running

Defense-in-depth: _build_startup_launcher() now prefixes the launcher
with 'if not exist <target> exit /b 0', so any future stale Startup
entry silently no-ops instead of flashing a console window.

Status enhancements
-------------------

- status() now reports supervisor task presence alongside the existing
  schtasks/Startup info, and nudges the user to reinstall if the
  supervisor isn't registered.
- Deep mode dumps both the supervisor task name + script path.

* fix(gateway,windows): drop the per-minute supervisor task — keep breakaway + deep probes

Earlier in this branch we added a per-minute schtasks-based supervisor to
respawn the gateway after crashes / GUI-update SIGTERMs. The implementation
flashed a brief console window on every firing, which stole window focus.
We tried several variants:

  - cmd.exe wrapper invoking pythonw  -> flashes (cmd.exe is console-subsystem)
  - schtasks /TR pointing at pythonw  -> flashes (uv venv launcher pythonw is
    actually subsystem=Console, not GUI; it respawns the real pythonw)
  - schtasks /TR pointing at base uv  -> still flashes (Task Scheduler-side
    conhost preallocation; documented Windows quirk)
  - XML registration with <Hidden>true>  -> still flashes (<Hidden> only hides
    the task in the Task Scheduler UI, not the spawned window)

Researched what leading projects do:

  - Ollama: GUI-subsystem tray exe + Startup-folder shortcut. No supervisor.
  - Tailscale: real Windows Service via SCM. Session 0, no console possible.
  - Syncthing: --no-console flag inside the binary + Startup folder.
  - openclaw: VBS Run(..., 0, False) wrapper. Suppresses the *window* but
    Super User Q971162 confirms focus-steal still occurs in some cases.

None of these use a per-minute polling scheduled task. The 'auto-restart on
crash' responsibility belongs INSIDE the daemon (Tailscale's in-process
recovery / Ollama's monitor+worker pair) OR is delegated to the Windows
Service Control Manager — not Task Scheduler.

So this commit drops the supervisor entirely. The CREATE_BREAKAWAY_FROM_JOB
fix in _subprocess_compat.py (from commit c1e5fa4) survives — that is the
*real* fix for problem #2 (GUI-update kills gateway): the post-update
watcher in launch_detached_profile_gateway_restart() now breaks out of
Electron's job object, so the gateway respawn watcher survives the GUI
quit and successfully respawns the gateway.

Surviving from c1e5fa4:
  * CREATE_BREAKAWAY_FROM_JOB in hermes_cli/_subprocess_compat.py (fixes #2)
  * Inlined breakaway flag in the watcher respawn snippet in gateway.py
  * hermes gateway status --deep PASS/FAIL probes (fixes #1 — visibility)
  * 'if not exist <target> exit /b 0' guard in _build_startup_launcher
    (fixes #3 — silent no-op for stale Startup entries)
  * tests/hermes_cli/test_gateway_wsl.py is_windows=False stubs (root cause
    of #3 — pytest WSL tests no longer leak Startup entries on Win hosts)

Removed in this commit:
  * hermes_cli/gateway_supervisor.py (entire file)
  * Supervisor section in hermes_cli/gateway_windows.py (~180 lines):
      get_supervisor_task_name, get_supervisor_script_path,
      _build_supervisor_cmd_script, _write_supervisor_script,
      _install_supervisor_task, is_supervisor_task_registered,
      _install_supervisor_best_effort
  * _install_supervisor_best_effort() calls in install() (3 spots)
  * supervisor cleanup block in uninstall()
  * supervisor display lines in status() / status(deep=True)

Future direction (out of scope for this PR): the right place for Windows
'Restart=always' semantics is a real Windows Service installed via
pywin32's win32serviceutil.ServiceFramework — session-0 isolation, SCM
auto-restart, no console window possible. That's a meaningful next-PR
project, not a band-aid.

Tests: 51 pass / 2 pre-existing failures in
tests/hermes_cli/test_gateway_{windows,wsl}.py (the 2 failures are
TestSupportsSystemdServicesWSL cases that fail on origin/main too —
unrelated to this PR).
xbrrr pushed a commit that referenced this pull request Jun 12, 2026
Add an official, production-grade WhatsApp integration via Meta's
Business Cloud API as a complement to the existing Baileys bridge.
No bridge subprocess, no QR codes, no account-ban risk — at the cost
of a Meta Business account and a public HTTPS webhook URL.

Setup is fully wizard-driven: 'hermes whatsapp-cloud' walks through
every credential with paste-time validation (catches the #1 trap of
pasting a phone number into the Phone Number ID field), generates a
verify token, and ends with copy-paste instructions for the
cloudflared / Meta-dashboard / Business Manager pieces that can't be
automated. The wizard also points users at Meta's Business Manager
for setting the bot's display name and profile picture.

Feature set:

- Inbound: text, images (with native-vision routing), voice notes
  (STT), documents (small text inlined, larger cached), reply context.
- Outbound: text with WhatsApp-flavored markdown conversion, images,
  videos, documents, opus voice notes via ffmpeg with MP3 fallback.
- Native interactive buttons for clarify, dangerous-command approval,
  and slash-command confirmation flows — matches the Telegram /
  Discord UX, graceful degrades to plain text.
- Read receipts (blue double-checkmarks) and typing indicator,
  using Meta's combined endpoint so they fire in a single API call.
- Webhook security: X-Hub-Signature-256 HMAC verification (raw body,
  constant-time), wamid deduplication, group-shaped-message refusal
  (groups deferred to v2 — Baileys still covers them).
- Full integration with the gateway's session, cron, display-tier,
  prompt-hint, and auth-allowlist systems. Cloud and Baileys can run
  side-by-side against different phone numbers.

Also wires STT (speech-to-text) through Nous's managed audio gateway
for Nous subscribers — previously the default stt.provider=local
required a separate faster-whisper install. New subscribers now get
voice-note transcription out of the box.

Docs: 418-line user guide at website/docs/user-guide/messaging/
whatsapp-cloud.md, sidebar entry, environment-variables reference,
ADDING_A_PLATFORM.md updated with the optional interactive-UX
contract for future adapter authors.

Tests: 100 dedicated tests for the adapter, 32 for the setup wizard,
20 for the Nous subscription STT wiring, plus regression coverage
across display_config, prompt_builder, and the cron scheduler.

Known limitations (deferred until clear demand signal):
- Group chats — use the Baileys bridge if you need them.
- Message templates for 24-hour-window outside-conversation sends —
  reactive chat is unaffected; cron / delegate_task with gaps > 24h
  will fail with a clear error. The agent's system prompt warns the
  model about this so it knows to mention it when scheduling delayed
  messages.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.