feat(file-sync): sync remote changes back to host on teardown (salvage of #8189 + hardening) by teknium1 · Pull Request #11291 · NousResearch/hermes-agent

teknium1 · 2026-04-17T00:11:54Z

Salvage of #8189 (by @kshitijk4poor) which salvaged #8018 (by @alt-glitch) onto current main, with follow-up hardening.

Phase 2 of the bulk file-sync spec: on SSH/Modal/Daytona sandbox teardown, pull back files that were modified remotely (or newly created by the agent) so skill/cache/credential writes inside the sandbox aren't lost.

Preserved from #8189

FileSyncManager.sync_back() with retry + exponential backoff
SIGINT deferral (main-thread only) so Ctrl-C during teardown doesn't leave partial writes
fcntl.flock serialization across concurrent gateway sandboxes (with Windows fallback)
SHA-256 hash diff to apply only changed files; unchanged files skipped
_infer_host_path prefix matching for new remote files
tarfile.extractall(filter="data") for path-traversal safety
Bulk download implementations for SSH (tar-over-SSH), Modal (exec + stdout), Daytona (exec + SDK download)
49 tests from the original salvage

Follow-up hardening added on top

tools/environments/daytona.py

PID-suffix the remote temp tar path: /tmp/.hermes_sync.<pid>.tar instead of the fixed /tmp/.hermes_sync.tar, so concurrent sync_back calls (e.g. retry after a partial failure) don't collide.
Move sync_back() inside the cleanup lock AND after the _sandbox is None guard, wrapped in its own try/except. Previously a no-op cleanup (sandbox already cleared) still fired sync_back → 3-attempt retry storm against a nil sandbox, ~6s of wasted sleep per teardown.

tools/environments/file_sync.py

_SYNC_BACK_MAX_BYTES (2 GiB) defensive cap: refuse to extract a tar larger than the limit. Protects against runaway sandboxes producing arbitrary-size archives.
"Nothing previously pushed" guard at the top of sync_back(): when both _pushed_hashes and _synced_files are empty, the manager was never initialized from the host side, so there is nothing coherent to sync back. Skips the retry/backoff machinery on uninitialized managers.

tests/tools/

Added TestSyncBackSizeCap with positive and negative cap coverage.
Updated _make_manager helper to seed _pushed_hashes so the default test path exercises the real sync_back flow; seed_pushed_state=False opt-out for noop-path tests.
Updated Daytona download test to assert the PID-suffixed pattern.

Test results

89 passed, 11 skipped in 7.56s

Ran: tests/tools/test_file_sync.py, test_file_sync_back.py, test_sync_back_backends.py, test_daytona_environment.py, test_ssh_environment.py.

Also: the pre-existing TestCleanup::test_persistent_cleanup_stops_sandbox and friends dropped from ~25s (retry-storm against mock sandbox) to 0.44s after the Daytona lifecycle fix and the "no prior push" guard.

E2E verification

Ran FileSyncManager.sync_back() with real file I/O (tar generation, extraction, diff, apply) — no mocks. Confirmed:

Modified remote file → host updated
Unchanged file → skipped by hash match (host untouched)
New remote file → pulled back via _infer_host_path prefix matching
Size cap → blocks extraction when tar exceeds the limit
Empty _pushed_hashes → guard skips sync_back without running retries

Credits

@alt-glitch — original implementation (feat(file-sync): sync remote changes back to host on teardown #8018)
@kshitijk4poor — initial salvage with first round of review fixes (feat(file-sync): sync remote changes back to host on teardown #8189)
Current PR — cherry-picks @kshitijk4poor's commit onto current main with authorship preserved, adds follow-up hardening

Closes #8189 and #8018.

@alt-glitch

Salvage of PR #8018 by @alt-glitch onto current main. On sandbox teardown, FileSyncManager now downloads the remote .hermes/ directory, diffs against SHA-256 hashes of what was originally pushed, and applies only changed files back to the host. Core (tools/environments/file_sync.py): - sync_back(): orchestrates download -> unpack -> diff -> apply with: - Retry with exponential backoff (3 attempts, 2s/4s/8s) - SIGINT trap + defer (prevents partial writes on Ctrl-C) - fcntl.flock serialization (concurrent gateway sandboxes) - Last-write-wins conflict resolution with warning - New remote files pulled back via _infer_host_path prefix matching Backends: - SSH: _ssh_bulk_download — tar cf - piped over SSH - Modal: _modal_bulk_download — exec tar cf - -> proc.stdout.read - Daytona: _daytona_bulk_download — exec tar cf -> SDK download_file - All three call sync_back() at the top of cleanup() Fixes applied during salvage (vs original PR #8018): | # | Issue | Fix | |---|-------|-----| | C1 | import fcntl unconditional — crashes Windows | try/except with fallback; _sync_back_locked skips locking when fcntl=None | | W1 | assert for runtime guard (stripped by -O) | Replaced with proper if/raise RuntimeError | | W2 | O(n*m) from _get_files_fn() called per file | Cache mapping once at start of _sync_back_impl, pass to resolve/infer | | W3 | Dead BulkDownloadFn imports in 3 backends | Removed unused imports | | W4 | Modal hardcodes root/.hermes, no explanation | Added docstring comment explaining Modal always runs as root | | S1 | SHA-256 computed for new files where pushed_hash=None | Skip hashing when pushed_hash is None (comparison always False) | | S2 | Daytona /tmp/.hermes_sync.tar never cleaned up | Added rm -f after download (best-effort) | Tests: 49 passing (17 new: _infer_host_path edge cases, SIGINT main/worker thread, Windows fcntl=None fallback, Daytona tar cleanup). Based on #8018 by @alt-glitch.

Follow-ups on top of kshitijk4poor's cherry-picked salvage of PR #8018: tools/environments/daytona.py - PID-suffix /tmp/.hermes_sync.<pid>.tar so concurrent sync_back calls against the same sandbox don't collide on the remote temp path - Move sync_back() inside the cleanup lock and after the _sandbox-None guard, with its own try/except. Previously a no-op cleanup (sandbox already cleared) still fired sync_back → 3-attempt retry storm against a nil sandbox (~6s of sleep). Now short-circuits cleanly. tools/environments/file_sync.py - Add _SYNC_BACK_MAX_BYTES (2 GiB) defensive cap: refuse to extract a tar larger than the limit. Protects against runaway sandboxes producing arbitrary-size archives. - Add 'nothing previously pushed' guard at the top of sync_back(). If _pushed_hashes and _synced_files are both empty, the FileSyncManager was never initialized from the host side — there is nothing coherent to sync back. Skips the retry/backoff machinery on uninitialized managers and eliminates test-suite slowdown from pre-existing cleanup tests that don't mock the sync layer. tests/tools/test_file_sync_back.py - Update _make_manager helper to seed a _pushed_hashes entry by default so sync_back() exercises its real path. A seed_pushed_state=False opt-out is available for noop-path tests. - Add TestSyncBackSizeCap with positive and negative coverage of the new cap. tests/tools/test_sync_back_backends.py - Update Daytona bulk download test to assert the PID-suffixed path pattern instead of the fixed /tmp/.hermes_sync.tar.

github-actions · 2026-04-17T00:12:08Z

⚠️ Supply Chain Risk Detected

This PR contains patterns commonly associated with supply chain attacks. This does not mean the PR is malicious — but these patterns require careful human review before merging.

⚠️ WARNING: exec() or eval() usage

Dynamic code execution can hide malicious behavior, especially when combined with base64 or network fetches.

Matches (first 20):

1011:+        self._sandbox.process.exec(
1017:+            self._sandbox.process.exec(f"rm -f {shlex.quote(remote_tar)}")

Automated scan triggered by supply-chain-audit. If this is a false positive, a maintainer can approve after manual review.

mxnstrexgl

🤖 Hermes Agent Security Review

Verdict: ✅ APPROVE — Strong security engineering

Security Controls Verified

✅ Path traversal: tarfile.extractall(filter="data") strips dangerous paths/symlinks
✅ Size cap: 2 GiB limit prevents DoS via oversized archives
✅ Race conditions: fcntl.flock(LOCK_EX) with proper cleanup in finally
✅ Atomic staging: tempfile.TemporaryDirectory() isolation before host writes
✅ Signal safety: SIGINT deferred during critical sync, restored after
✅ Cleanup guarantees: Context managers + finally blocks throughout
✅ Retry limiting: Max 3 retries with exponential backoff

Minor Observations

💡 filter="data" requires Python 3.12+ — verify min version compatibility
💡 _infer_host_path() uses prefix matching — tests confirm no false positives

Looks Good

Defense-in-depth: tar filter + staging + path validation + size cap + locking
Comprehensive test coverage (473+ lines)
Clear code structure with proper error handling

Reviewed by Hermes Agent (automated)

…ers (#17727) Covers ~60 merged PRs from Apr 15–29 that shipped user-visible behavior without docs coverage. No functional code changes; docs + static manifest regeneration only. Highlights: Stale / incorrect: - configuration.md: auxiliary auto-routing line was wrong since #11900; now correctly states auto routes to the main model, with a note on the cost trade-off and per-task override pattern. - integrations/providers.md + configuration.md compression intro: removed stale 'Gemini Flash via OpenRouter' claim. - website/static/api/model-catalog.json: rebuilt from hermes_cli/models.py so the live manifest picks up tencent/hy3-preview (and remains in sync for future model-catalog PRs). Platform messaging (#17417 #16997 #16193 #14315 #13151 #11794 #10610 #10283 #10246 #11564 #13178): - Signal: native formatting (bodyRanges), reply quotes, reactions. - Telegram: table rendering (bullets + code-block fallback), disable_link_previews, group_allowed_chats. - Slack: strict_mention config. - Discord: slash_commands disable, send_animation GIF, send_message native media attachments. - DingTalk: require_mention + allowed_users. CLI (#16052 #16539 #16566 #15841 #14798 #10043): - New 'hermes fallback' interactive manager. - New 'hermes update --check', '--backup' flag, and pre-update pairing snapshot behavior. - 'hermes gateway start/restart --all' multi-profile flag. - cron.md: 'hermes tools' as a platform, per-job enabled_toolsets, wakeAgent gate, context_from chaining. Config keys / env vars (#17305 #17026 #17000 #15077 #14557 #14227 #14166 #14730 #17008): - terminal.docker_run_as_host_user, display.runtime_metadata_footer, compression.hygiene_hard_message_limit, HINDSIGHT_TIMEOUT, skills.guard_agent_created, TAVILY_BASE_URL, security.allow_private_urls, agent.api_max_retries, gateway hot-reload of compression/context_length config edits. TUI / CLI UX (#17130 #17113 #17175 #17150 #16707 #12312 #12305 #12934 #14810 #14045 #17286 #17126): - HERMES_TUI_RESUME, HERMES_TUI_THEME, LaTeX rendering, busy-indicator styles, ctrl-x queued-message delete, git branch in status bar, per- prompt elapsed stopwatch, external-editor keybind, markdown stripping, TUI voice-mode parity, /agents overlay, /reload + /mouse. Gateway features (#16506 #15027 #13428 #12116): - Native multimodal image routing based on vision capability. - /usage account-limits section. - /steer slash command (added to reference + explanation in CLI). Plugins / hooks (#12929 #12972 #10763 #16364): - transform_tool_result, transform_terminal_output plugin hooks. - PluginContext.dispatch_tool() documented with slash-command example. - google_meet bundled plugin entry under built-in-plugins.md. Other (#16576 #16572 #16383 #15878 #15608 #15606 #14809 #14767 #14231 #14232 #14307 #13683 #12373 #11891 #11291 #10066): - hermes backup exclusions (WAL/SHM/journal + checkpoints/). - security.md hardline blocklist (floor below --yolo). - FHS install layout for root installs. - openssh-client + docker-cli baked into the Docker image. - MEDIA: tag supported extensions table (docs/office/archives/pdf). - Remote-to-host file sync on SSH/Modal/Daytona teardown. - 'hermes model' -> Configure Auxiliary Models interactive picker. - Podman support via HERMES_DOCKER_BINARY. Providers / STT / one-shot (#15045 #14473 #15704): - alibaba-coding-plan first-class provider entry. - xAI Grok STT as a 6th transcription option. - 'hermes -z' scripted one-shot mode + HERMES_INFERENCE_MODEL. Build: 'docusaurus build' succeeds. No new broken links/anchors; pre-existing warnings unchanged.

…ers (NousResearch#17727) Covers ~60 merged PRs from Apr 15–29 that shipped user-visible behavior without docs coverage. No functional code changes; docs + static manifest regeneration only. Highlights: Stale / incorrect: - configuration.md: auxiliary auto-routing line was wrong since NousResearch#11900; now correctly states auto routes to the main model, with a note on the cost trade-off and per-task override pattern. - integrations/providers.md + configuration.md compression intro: removed stale 'Gemini Flash via OpenRouter' claim. - website/static/api/model-catalog.json: rebuilt from hermes_cli/models.py so the live manifest picks up tencent/hy3-preview (and remains in sync for future model-catalog PRs). Platform messaging (NousResearch#17417 NousResearch#16997 NousResearch#16193 NousResearch#14315 NousResearch#13151 NousResearch#11794 NousResearch#10610 NousResearch#10283 NousResearch#10246 NousResearch#11564 NousResearch#13178): - Signal: native formatting (bodyRanges), reply quotes, reactions. - Telegram: table rendering (bullets + code-block fallback), disable_link_previews, group_allowed_chats. - Slack: strict_mention config. - Discord: slash_commands disable, send_animation GIF, send_message native media attachments. - DingTalk: require_mention + allowed_users. CLI (NousResearch#16052 NousResearch#16539 NousResearch#16566 NousResearch#15841 NousResearch#14798 NousResearch#10043): - New 'hermes fallback' interactive manager. - New 'hermes update --check', '--backup' flag, and pre-update pairing snapshot behavior. - 'hermes gateway start/restart --all' multi-profile flag. - cron.md: 'hermes tools' as a platform, per-job enabled_toolsets, wakeAgent gate, context_from chaining. Config keys / env vars (NousResearch#17305 NousResearch#17026 NousResearch#17000 NousResearch#15077 NousResearch#14557 NousResearch#14227 NousResearch#14166 NousResearch#14730 NousResearch#17008): - terminal.docker_run_as_host_user, display.runtime_metadata_footer, compression.hygiene_hard_message_limit, HINDSIGHT_TIMEOUT, skills.guard_agent_created, TAVILY_BASE_URL, security.allow_private_urls, agent.api_max_retries, gateway hot-reload of compression/context_length config edits. TUI / CLI UX (NousResearch#17130 NousResearch#17113 NousResearch#17175 NousResearch#17150 NousResearch#16707 NousResearch#12312 NousResearch#12305 NousResearch#12934 NousResearch#14810 NousResearch#14045 NousResearch#17286 NousResearch#17126): - HERMES_TUI_RESUME, HERMES_TUI_THEME, LaTeX rendering, busy-indicator styles, ctrl-x queued-message delete, git branch in status bar, per- prompt elapsed stopwatch, external-editor keybind, markdown stripping, TUI voice-mode parity, /agents overlay, /reload + /mouse. Gateway features (NousResearch#16506 NousResearch#15027 NousResearch#13428 NousResearch#12116): - Native multimodal image routing based on vision capability. - /usage account-limits section. - /steer slash command (added to reference + explanation in CLI). Plugins / hooks (NousResearch#12929 NousResearch#12972 NousResearch#10763 NousResearch#16364): - transform_tool_result, transform_terminal_output plugin hooks. - PluginContext.dispatch_tool() documented with slash-command example. - google_meet bundled plugin entry under built-in-plugins.md. Other (NousResearch#16576 NousResearch#16572 NousResearch#16383 NousResearch#15878 NousResearch#15608 NousResearch#15606 NousResearch#14809 NousResearch#14767 NousResearch#14231 NousResearch#14232 NousResearch#14307 NousResearch#13683 NousResearch#12373 NousResearch#11891 NousResearch#11291 NousResearch#10066): - hermes backup exclusions (WAL/SHM/journal + checkpoints/). - security.md hardline blocklist (floor below --yolo). - FHS install layout for root installs. - openssh-client + docker-cli baked into the Docker image. - MEDIA: tag supported extensions table (docs/office/archives/pdf). - Remote-to-host file sync on SSH/Modal/Daytona teardown. - 'hermes model' -> Configure Auxiliary Models interactive picker. - Podman support via HERMES_DOCKER_BINARY. Providers / STT / one-shot (NousResearch#15045 NousResearch#14473 NousResearch#15704): - alibaba-coding-plan first-class provider entry. - xAI Grok STT as a 6th transcription option. - 'hermes -z' scripted one-shot mode + HERMES_INFERENCE_MODEL. Build: 'docusaurus build' succeeds. No new broken links/anchors; pre-existing warnings unchanged.

kshitijk4poor and others added 2 commits April 16, 2026 17:10

mxnstrexgl approved these changes Apr 17, 2026

View reviewed changes

teknium1 merged commit 7fd5089 into main Apr 17, 2026
4 of 5 checks passed

teknium1 deleted the hermes/hermes-37ea1ad1 branch April 17, 2026 02:39

This was referenced Apr 17, 2026

feat(file-sync): sync remote changes back to host on teardown #8018

Closed

feat(file-sync): sync remote changes back to host on teardown #8189

Closed

github-actions Bot mentioned this pull request Apr 24, 2026

chore: bump NousResearch/hermes-agent version from v2026.4.16 to v2026.4.23 Docker-Hub-sirmark/docker-hermes-agent#3

Merged

teknium1 mentioned this pull request Apr 30, 2026

docs: two-week gap sweep — platforms, CLI, config, TUI, hooks, providers #17727

Merged

alt-glitch mentioned this pull request May 16, 2026

feat(file-sync): sync remote changes back to host on teardown (+2 bugfixes) #27091

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(file-sync): sync remote changes back to host on teardown (salvage of #8189 + hardening)#11291

feat(file-sync): sync remote changes back to host on teardown (salvage of #8189 + hardening)#11291
teknium1 merged 2 commits into
mainfrom
hermes/hermes-37ea1ad1

teknium1 commented Apr 17, 2026

Uh oh!

github-actions Bot commented Apr 17, 2026

Uh oh!

mxnstrexgl left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

teknium1 commented Apr 17, 2026

Preserved from #8189

Follow-up hardening added on top

Test results

E2E verification

Credits

Uh oh!

github-actions Bot commented Apr 17, 2026

⚠️ Supply Chain Risk Detected

⚠️ WARNING: exec() or eval() usage

Uh oh!

mxnstrexgl left a comment

Choose a reason for hiding this comment

🤖 Hermes Agent Security Review

Security Controls Verified

Minor Observations

Looks Good

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants