Skip to content

[Bug]: New Daytona implementation requires hundreds of file uploads at every session start #6035

@kentimsit

Description

@kentimsit

Bug Description

DaytonaEnvironment.__init__() calls _sync_skills_and_credentials() which uploads every file in ~/.hermes/skills/ into the sandbox one at a time via individual process.exec("mkdir -p ...") + fs.upload_file() calls. With a standard Hermes install (445 skill files), this takes ~275 seconds on every new agent session — two SDK round-trips per file, ~890 calls total.

The synced files are never read inside the sandbox. All skill access goes through host-side skill_view(), build_skills_system_prompt(), and _load_skill_payload(). The files are write-only.

The same method is also called before every execute() command (line 297). Subsequent calls skip unchanged files via an in-memory _synced_files cache, but the cache is per-instance and lost between sessions, so every new conversation pays the full 275s cost.

Steps to Reproduce

  1. Install Hermes with the default bundled skills (~445 files)
  2. Configure terminal.backend: daytona in config.yaml
  3. Run any skill that uses the terminal tool
  4. Observe that the first tool call takes ~275 seconds

Expected Behavior

First tool call completes in under 5 seconds (sandbox creation + first command is ~3.5s measured).

Actual Behavior

First tool call takes ~275 seconds. All time is spent in _sync_skills_and_credentials() at tools/environments/daytona.py:156-171, specifically the iter_skills_files loop (lines 167-169).

Measured with standalone debug scripts using the Daytona SDK directly:

Step Time
Daytona() client construction 1.26s
daytona.get() + start() or create() 1.33s
First command (echo $HOME) 0.71s
Upload 50/445 skill files (sample) 31.74s
Extrapolated full sync (445 files) 282.5s
Average per file (mkdir -p + upload_file) 0.635s

Affected Component

Tools (terminal, file ops, web, code execution, etc.)

Messaging Platform (if gateway-related)

No response

Operating System

Ubuntu 24.04

Python Version

3.11

Hermes Version

0.7 (commit 99ff375)

Relevant Logs / Traceback

# Timing from debug script (50-file sample of 445 total):
Upload 50 skill file(s) — sample of 445
  Uploaded 50/50 files in 31.74s
  Avg per file: 0.635s
  Min: 0.537s  Max: 0.971s
  EXTRAPOLATED total for all 445 skill files: 282.5s

Root Cause Analysis (optional)

_upload_if_changed() (daytona.py:136-154) issues two SDK calls per file:

  1. self._sandbox.process.exec(f"mkdir -p {parent}") — bash command via SDK
  2. self._sandbox.fs.upload_file(host_path, remote_path) — singular file upload

For 445 files, this is ~890 sequential SDK round-trips.

No code inside the Daytona sandbox reads from the synced ~/.hermes/skills/ directory. Skills are loaded on the host side by skill_view(), build_skills_system_prompt(), and _load_skill_payload().

The same pattern exists in Docker (docker.py:370-379), Modal (modal.py:209-218), and SSH (ssh.py:140-153) backends.

Proposed Fix (optional)

Remove the iter_skills_files loop in _sync_skills_and_credentials() (daytona.py:167-169). Keep the credential file sync loop (lines 162-165) which is correct but currently has no files to sync.

As a longer-term enhancement, skills that need specific files inside the sandbox could declare them in their SKILL.md frontmatter and have a staging mechanism upload only those files using the batch fs.upload_files() API, rather than syncing the entire skills tree.

Are you willing to submit a PR for this?

  • I'd like to fix this myself and submit a PR

Metadata

Metadata

Assignees

No one assigned

    Labels

    type/bugSomething isn't working

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions