Skip to content

fix(gateway): stage safe local MEDIA files before delivery#38108

Closed
GodsBoy wants to merge 1 commit into
NousResearch:mainfrom
GodsBoy:fix/stage-local-media-files
Closed

fix(gateway): stage safe local MEDIA files before delivery#38108
GodsBoy wants to merge 1 commit into
NousResearch:mainfrom
GodsBoy:fix/stage-local-media-files

Conversation

@GodsBoy

@GodsBoy GodsBoy commented Jun 3, 2026

Copy link
Copy Markdown
Contributor

What does this PR do?

Agents routinely produce deliverables in a working directory (proposal.docx, report.pdf) instead of inside a Hermes cache. When the gateway runs as root, the whole /root home is on the MEDIA delivery denylist, so validate_media_delivery_path rejects the path and the attachment is silently dropped while the chat reply still claims a file is attached.

Live repro: a MEDIA:/root/clawd/...docx directive produced Skipping unsafe MEDIA directive path: /root/clawd/...docx in gateway.log, and the Telegram DM arrived with no attachment. Copying the same file to ~/.hermes/cache/documents/...docx and sending that cache path delivered fine.

This adds stage_media_delivery_path: when a path fails direct validation, a safe local deliverable is copied into the Hermes media cache that matches its extension, and the staged path (now under an allowlisted cache root) is delivered normally. filter_media_delivery_paths and filter_local_delivery_paths try direct validation first, then the staging fallback, then rejection, so every dispatch site benefits.

The denylist is preserved, and no arbitrary working directory is allowlisted:

  • Staging refuses credential/system directories via the existing _media_delivery_denied_paths (/etc, ~/.ssh, ~/.hermes/.env, ...).
  • It refuses credential filenames anywhere (.netrc, id_rsa, *.pem, auth.json, config.yaml, ...).
  • Symlinks are resolved before any check, so a link pointing at a denied target is rejected on the resolved path.
  • In strict mode it only stages freshly produced files, preserving a public-facing operator's stale-host-file protection.
  • The single relaxation is the running user's own home: that is what lets a root-run gateway deliver /root/work/proposal.docx while /root/.ssh stays blocked. It cannot un-block a credential sub-directory or another user's home.

This preserves the denylist protections and avoids allowlisting Path.home(), /root, /tmp, or arbitrary working dirs.

Related Issue

Fixes #38106

Type of Change

  • 🐛 Bug fix (non-breaking change that fixes an issue)
  • 🔒 Security fix
  • ✅ Tests (adding or improving test coverage)

Changes Made

  • gateway/platforms/base.py
    • Extract _resolve_media_candidate from validate_media_delivery_path (shared, behaviour-preserving resolution prologue).
    • Add _resolved_under_allowed_root, _stage_filename_is_credential, _stage_path_under_denied_dir, _stage_target_cache_dir, _copy_into_media_cache, and stage_media_delivery_path.
    • Wire the staging fallback into BasePlatformAdapter.filter_media_delivery_paths and filter_local_delivery_paths, and expose a stage_media_delivery_path static method.
    • import shutil.
  • tests/gateway/test_platform_base.py
    • Add TestMediaDeliveryStaging (21 tests).

How to Test

scripts/run_tests.sh tests/gateway/test_platform_base.py        # 160 passed
scripts/run_tests.sh tests/tools/test_send_message_tool.py      # 121 passed
scripts/run_tests.sh tests/gateway/test_media_extraction.py tests/gateway/test_tts_media_routing.py tests/gateway/test_send_image_file.py tests/gateway/test_signal.py   # all passed

Manual (root-run gateway, default non-strict mode):

  1. Create /root/work/proposal.docx.
  2. BasePlatformAdapter.validate_media_delivery_path("/root/work/proposal.docx") returns None (denied prefix).
  3. BasePlatformAdapter.filter_media_delivery_paths([("/root/work/proposal.docx", False)]) returns the staged ~/.hermes/cache/documents/proposal.docx, which then validates and delivers. The source file is untouched.
  4. ~/.ssh/id_rsa, ~/.hermes/.env, /etc/..., and a .netrc/*.pem in the workdir all remain rejected (not staged).

Checklist

Code

  • I've read the Contributing Guide
  • My commit messages follow Conventional Commits (fix(gateway):)
  • I searched for existing PRs to make sure this isn't a duplicate
  • My PR contains only changes related to this fix (staging + its tests)
  • I've run the gateway + tools media test suites and they pass (commands above)
  • I've added tests for my changes (21 new staging tests)
  • I've tested on my platform: Ubuntu 24.04 (Linux 6.8)

Documentation & Housekeeping

  • I've updated relevant documentation (docstrings) — code docstrings cover the new helpers
  • I've updated cli-config.yaml.example if I added/changed config keys — N/A (no new config keys)
  • I've updated CONTRIBUTING.md or AGENTS.md if I changed architecture or workflows — N/A
  • I've considered cross-platform impact (Windows, macOS) — uses pathlib/shutil, home-relative denylist resolution; macOS Keychains already denied
  • I've updated tool descriptions/schemas if I changed tool behavior — N/A (gateway internals, no tool schema change)

Screenshots / Logs

Before (gateway.log): Skipping unsafe MEDIA directive path: /root/clawd/...docx and no attachment.
After: Staged MEDIA file /root/.../...docx -> ~/.hermes/.../...docx for delivery and the file attaches.

Agents commonly produce deliverables in a working directory (proposal.docx,
report.pdf) rather than inside a Hermes cache. When the gateway runs as root,
the whole /root home is on the system denylist, so validate_media_delivery_path
drops the path and the user gets a chat reply that claims an attachment with
nothing attached. Copying the same file under ~/.hermes/cache/documents/ made
it deliver.

stage_media_delivery_path copies a safe local deliverable into the media cache
matching its extension, then returns the staged path so normal MEDIA delivery
proceeds. filter_media_delivery_paths and filter_local_delivery_paths now try
direct validation first, then the staging fallback, then rejection.

The denylist is preserved. Staging refuses credential/system directories via
the existing _media_delivery_denied_paths, refuses credential filenames
(.netrc, id_rsa, *.pem, ...), resolves symlinks before any check, and in strict
mode only stages freshly produced files. The running user's own home is the one
allowed exception to the broad denied-prefix, which is what lets a root-run
gateway deliver /root/work/proposal.docx while /root/.ssh stays blocked. No
arbitrary working directory is allowlisted.

Adds 21 tests covering staging, denylist preservation, collision-safe names,
strict-mode recency, and the filter integration.
@alt-glitch alt-glitch added type/bug Something isn't working comp/gateway Gateway runner, session dispatch, delivery P2 Medium — degraded but workaround exists labels Jun 3, 2026
teknium1 added a commit that referenced this pull request Jun 4, 2026
Root-run gateways have $HOME=/root, which is on the MEDIA system-path
denylist, so the gateway silently dropped agent-generated deliverables
under /root (e.g. /root/work/proposal.docx) — the user got a 'here is
your file' reply with nothing attached.

_path_under_denied_prefix now treats the running user's own home as
deliverable: the home tree itself is no longer denied, while the
more-specific denied paths inside it (~/.ssh, ~/.aws, ~/.hermes/.env,
auth.json, config.yaml) stay blocked because they are separate denylist
entries. The exception only matches when the denied prefix IS $HOME, so
a non-root gateway still can't deliver another user's home.

Diagnosis, reproduction, and the failing-case analysis are from
@GodsBoy (#38108 / #38106). Implemented here as the minimal denylist
fix rather than a staging/copy subsystem.

Co-authored-by: GodsBoy <dhuysamen@gmail.com>
@teknium1

teknium1 commented Jun 4, 2026

Copy link
Copy Markdown
Contributor

Thanks for the diagnosis and the live Telegram repro — that's what made this actionable.

We shipped the fix as #39063 (merged to main as 2982122b), using a minimal version of your approach. Rather than staging/copying the file into the media cache, we fixed the root cause directly in the delivery validator: a root-run gateway has $HOME=/root, which was on the system-path denylist, so _path_under_denied_prefix() was shadowing the operator's entire own home. It now treats the running user's own $HOME as deliverable while keeping the more-specific denied paths inside it (~/.ssh, ~/.aws, ~/.hermes/.env, auth.json, config.yaml) blocked. Platform adapters upload by path, so the file delivers in place — no cache duplication, no second denylist table to keep in sync (~27 LOC vs the staging subsystem).

Your authorship is preserved via Co-authored-by on the merge commit, and the bug + repro analysis are entirely yours. Closing this in favor of #39063 — much appreciated.

@teknium1 teknium1 closed this Jun 4, 2026
waym0reom3ga pushed a commit to waym0reom3ga/autolycus-agent that referenced this pull request Jun 4, 2026
Root-run gateways have $HOME=/root, which is on the MEDIA system-path
denylist, so the gateway silently dropped agent-generated deliverables
under /root (e.g. /root/work/proposal.docx) — the user got a 'here is
your file' reply with nothing attached.

_path_under_denied_prefix now treats the running user's own home as
deliverable: the home tree itself is no longer denied, while the
more-specific denied paths inside it (~/.ssh, ~/.aws, ~/.hermes/.env,
auth.json, config.yaml) stay blocked because they are separate denylist
entries. The exception only matches when the denied prefix IS $HOME, so
a non-root gateway still can't deliver another user's home.

Diagnosis, reproduction, and the failing-case analysis are from
@GodsBoy (NousResearch#38108 / NousResearch#38106). Implemented here as the minimal denylist
fix rather than a staging/copy subsystem.

Co-authored-by: GodsBoy <dhuysamen@gmail.com>
Yuki-14544869 pushed a commit to Yuki-14544869/hermes-agent that referenced this pull request Jun 4, 2026
Root-run gateways have $HOME=/root, which is on the MEDIA system-path
denylist, so the gateway silently dropped agent-generated deliverables
under /root (e.g. /root/work/proposal.docx) — the user got a 'here is
your file' reply with nothing attached.

_path_under_denied_prefix now treats the running user's own home as
deliverable: the home tree itself is no longer denied, while the
more-specific denied paths inside it (~/.ssh, ~/.aws, ~/.hermes/.env,
auth.json, config.yaml) stay blocked because they are separate denylist
entries. The exception only matches when the denied prefix IS $HOME, so
a non-root gateway still can't deliver another user's home.

Diagnosis, reproduction, and the failing-case analysis are from
@GodsBoy (NousResearch#38108 / NousResearch#38106). Implemented here as the minimal denylist
fix rather than a staging/copy subsystem.

Co-authored-by: GodsBoy <dhuysamen@gmail.com>
davidgut1982 pushed a commit to davidgut1982/hermes-agent that referenced this pull request Jun 5, 2026
Root-run gateways have $HOME=/root, which is on the MEDIA system-path
denylist, so the gateway silently dropped agent-generated deliverables
under /root (e.g. /root/work/proposal.docx) — the user got a 'here is
your file' reply with nothing attached.

_path_under_denied_prefix now treats the running user's own home as
deliverable: the home tree itself is no longer denied, while the
more-specific denied paths inside it (~/.ssh, ~/.aws, ~/.hermes/.env,
auth.json, config.yaml) stay blocked because they are separate denylist
entries. The exception only matches when the denied prefix IS $HOME, so
a non-root gateway still can't deliver another user's home.

Diagnosis, reproduction, and the failing-case analysis are from
@GodsBoy (NousResearch#38108 / NousResearch#38106). Implemented here as the minimal denylist
fix rather than a staging/copy subsystem.

Co-authored-by: GodsBoy <dhuysamen@gmail.com>
changman pushed a commit to changman/hermes-agent that referenced this pull request Jun 10, 2026
Root-run gateways have $HOME=/root, which is on the MEDIA system-path
denylist, so the gateway silently dropped agent-generated deliverables
under /root (e.g. /root/work/proposal.docx) — the user got a 'here is
your file' reply with nothing attached.

_path_under_denied_prefix now treats the running user's own home as
deliverable: the home tree itself is no longer denied, while the
more-specific denied paths inside it (~/.ssh, ~/.aws, ~/.hermes/.env,
auth.json, config.yaml) stay blocked because they are separate denylist
entries. The exception only matches when the denied prefix IS $HOME, so
a non-root gateway still can't deliver another user's home.

Diagnosis, reproduction, and the failing-case analysis are from
@GodsBoy (NousResearch#38108 / NousResearch#38106). Implemented here as the minimal denylist
fix rather than a staging/copy subsystem.

Co-authored-by: GodsBoy <dhuysamen@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/gateway Gateway runner, session dispatch, delivery P2 Medium — degraded but workaround exists type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: Gateway silently drops safe local MEDIA files from working dirs (e.g. /root) instead of staging them

3 participants