fix(desktop): stage dropped files into the remote session workspace by OutThisLife · Pull Request #43109 · NousResearch/hermes-agent

OutThisLife · 2026-06-09T21:50:40Z

Summary

Finder/OS drops became @file:/Users/... refs that only resolve when the gateway shares the local disk. On a remote gateway that path doesn't exist, so non-image files (PDF, CSV, Markdown, spreadsheets…) never reached the agent — the exact symptom users hit dragging a PDF into a VPS-backed session.

The byte-upload plumbing (file.attach / image.attach_bytes) already lives in the gateway + submit-time sync; the gap was at the drop layer, which short-circuited every drop into an inline local-path ref. This routes by origin:

In-app drags (project tree, gutter line refs) are workspace-relative and resolve on the gateway as-is → stay inline @file: / @line: refs.
OS/Finder drops are absolute paths on this machine (and images need their bytes for vision) → go through the upload pipeline, which stages them into the session workspace and hands back a gateway-side ref.

Applied across every drop surface that previously leaked the local path: conversation area, composer form, the contenteditable input, and the message-edit composer.

Also in here

Eager upload + loader — a dropped file stages as soon as it lands (when a session exists), so the card shows a spinner instead of stalling the send; submit reuses the already-staged ref. Images stay submit-time on purpose (eager upload raced attachImagePath's async thumbnail write, clobbering previewUrl + the local path).
Attachment card polish — rounded, no monospace, plus a spinner / error overlay driven by a new uploadState.
Screenshot preview fix — image previews now render from the base64 we already hold, so a pasted/dropped screenshot shows its thumbnail and previews even when its only on-disk copy is a transient path the renderer can't re-read. The data URL is stripped before persisting the preview registry (it would blow the localStorage quota).

Supersedes #38615 and #41203 (both closed); preserves their authors via Co-authored-by.

Test plan

npm run type-check clean
eslint clean on touched files (no new warnings)
vitest — new partitionDroppedFiles unit tests + eager-upload / file.attach sync tests pass (pre-existing unrelated interrupted test still red on main)
Manual E2E against a Dockerized remote gateway: drag a PDF from Finder → uploads via file.attach, shows a proper chip, agent reads it (previously sent a dead local path)
Reviewer: drop a screenshot (and cmd-shift-4 thumbnail drag) → thumbnail on the card + renders in the preview pane
Reviewer: edit a sent message and drop an external file → inserts a gateway-side ref, not the local path

Finder/OS drops became `@file:/Users/...` refs that only resolve when the gateway shares the local disk, so on a remote gateway non-image files (PDF/CSV/Markdown/...) never reached the agent. Route OS drops through the file.attach / image.attach_bytes upload pipeline — in-app project-tree and gutter drags stay inline workspace-relative refs — across every drop surface: the conversation area, the composer form, the contenteditable input, and the message-edit composer (which still reproduced the bug). Also: - upload dropped files eagerly when a session exists, so the card shows a spinner instead of stalling the send (images stay submit-time to avoid racing their thumbnail write); - round the attachment card and drop the monospace detail; - render image previews from the bytes we already hold, so a pasted/dropped screenshot shows its thumbnail and previews even when its only on-disk copy is a transient path (the data URL is not persisted to localStorage). Supersedes #38615, #41203. Co-authored-by: LeonSGP <154585401+LeonSGP43@users.noreply.github.com> Co-authored-by: Teknium <127238744+teknium1@users.noreply.github.com>

github-actions · 2026-06-09T21:51:24Z

🔎 Lint report: `fix/desktop-remote-attach-drops` vs `origin/main`

ruff

Total: 0 on HEAD, 0 on base (➖ 0)

🆕 New issues: none

✅ Fixed issues: none

Unchanged: 0 pre-existing issues carried over.

ty (type checker)

Total: 10620 on HEAD, 10620 on base (➖ 0)

🆕 New issues: none

✅ Fixed issues: none

Unchanged: 5564 pre-existing issues carried over.

Diagnostics are surfaced as warnings — this check never fails the build.

The in-flight user bubble seeded image attachment refs as `@image:<localpath>`. In remote-gateway mode that path lives on the desktop, not the gateway, so the inline thumbnail fetch hit /api/media and 403'd ("Path outside media roots"), flashing a fallback chip until submit uploaded the bytes. Seed (and keep) image refs as the raw base64 preview data URL instead. It renders inline via extractEmbeddedImages with zero network, and survives the post-sync rewrite (the agent gets the bytes through the attached-image pipeline, not this display ref) so the thumbnail no longer remounts/flashes. Non-image refs are unchanged. Adds optimisticAttachmentRef + unit coverage.

helix4u · 2026-06-09T22:30:04Z

Nice, this is hitting the right layer. #42634 gave the backend/submit path a file.attach staging route, but the remaining failure is that OS drops were getting turned into path-only inline @file:/Users/... refs before submit sync ever had a path-bearing attachment to upload. Splitting File-bearing OS drops from path-only in-app drags is the right shape, and I like that it covers the conversation area, composer, contenteditable input, and edit composer instead of fixing only one surface.

Two races I think are worth tightening before merge:

Removed chip can resurrect after eager upload. eagerlyUploadAttachment eventually calls addComposerAttachment(await uploadComposerAttachment(...)). Since upsertAttachment appends when the id no longer exists, this sequence can re-add a file the user removed while upload was in flight:

drop file
eager upload starts
user removes chip
upload resolves
addComposerAttachment(uploaded) appends it again

The error path already behaves better because setComposerAttachmentUploadState is a no-op if the chip is gone. Success probably needs the same update-only behavior, or an existence check before re-adding.

Fast submit can duplicate the upload. Submit-time sync skips !attachment.path and attachedSessionId === sessionId, but it does not join/await the eager upload in flight. If the user drops a PDF and immediately hits enter, eager upload and submit sync can both call file.attach; the gateway will stage duplicate files via _unique_attachment_path, and the final ref depends on which path wins. Not data loss, but it leaves stray duplicates under .hermes/desktop-attachments/ and makes the UI state racy. Either join the in-flight eager upload from submit or make submit wait/retry while uploadState === 'uploading'.

Smaller, non-blocking notes:

Edit-composer drops are async with no visible staging state. If the user confirms the edit before upload finishes, the inserted gateway refs can miss the edit. Probably okay for now, but worth consciously accepting or adding a follow-up.
Remote non-image upload still goes through readFileDataUrl, so files above the 16 MB IPC data-url cap will fail with the underlying read error. Fine as a limit, but eventually that wants a friendlier "file too large to upload to remote gateway" style message.

Overall: fix direction looks right and matches the reported Desktop remote-gateway bug. I would address at least the remove-during-upload and fast-submit duplicate-upload races before merge.

@helix4u

Two races in the drop-time eager upload: - Resurrected chip: the success path used addComposerAttachment, which re-appends when the id is gone, so a file removed mid-upload reappeared once the upload resolved. Add updateComposerAttachment (update-only; no-op when the chip was removed) and use it on both the eager success path and submit-time sync. - Duplicate upload: submit-time sync didn't join an eager upload still in flight, so drop-then-Enter could fire file.attach twice and leave a duplicate under .hermes/desktop-attachments/. Track in-flight eager uploads by id and await the pending one before deciding to re-upload, reusing its gateway ref. Tests: composer-store no-resurrect unit tests + a join-on-submit integration test asserting a single file.attach. Addresses @helix4u review on #43109.

…ops upload The message-edit composer staged dropped OS files asynchronously with no visible state, so confirming the edit before the upload resolved could send the message without the gateway-side ref (helix4u review note on #43109). Add a staging flag: while uploadOsDropRefs is in flight, show a small spinner pill in the bubble and block submit (disabled send button + submitEdit guard) so the edit can't outrace the ref insertion. New `attachingFile` i18n string across en/zh/zh-hant/ja.

…6MB cap Remote attachments read their bytes through the readFileDataUrl IPC, which is hard-capped at 16MB and rejects with a raw "file is too large (N bytes; limit M bytes)" string straight into the failure toast (helix4u review note on #43109). Translate that into "<file> is too large to upload to the remote gateway (max 16 MB)", parsing the limit out of the message so it tracks the real cap. Applies to both the image and non-image remote read paths; non-cap errors pass through unchanged. Adds unit coverage for both.

OutThisLife · 2026-06-09T23:35:21Z

Thanks @helix4u — addressed all four points.

Blocking races (891c9a6):

Removed chip resurrecting: the eager success path now goes through an update-only updateComposerAttachment (no-op when the id is gone) instead of addComposerAttachment, and submit-time sync uses it too.
Fast-submit duplicate upload: in-flight eager uploads are tracked by id and submit awaits the pending one before deciding, so drop-then-Enter fires a single file.attach.

Non-blocking notes (b021497, 29147af):

Edit composer now shows a staging spinner and blocks submit while a dropped file uploads, so confirming the edit can't outrace the ref insertion.
The 16MB IPC cap now surfaces as " is too large to upload to the remote gateway (max 16 MB)" instead of the raw read error.

Store/integration tests cover the no-resurrect and single-file.attach cases plus the cap-error translation.

…ad end A binary @file: ref (PDF, docx, spreadsheet, …) expanded to a bare "binary files are not supported" warning with no content. The model saw a failure and gave up — e.g. a dropped PDF came back as a text note claiming the type was unsupported, even though the file was staged on disk right next to it. Inject an actionable content block instead: the path, mime type, size, and a nudge to use its tools to read/convert/view the file (and explicitly not to tell the user the type is unsupported). General across every binary type — not PDF-specific. The file already resolves where the agent's tools run (local cwd or the staged copy in a remote session workspace), so it can act on it directly.

OutThisLife · 2026-06-10T00:17:39Z

Follow-up from manual testing (Windows → Mac remote): a dropped PDF uploaded fine but the agent then refused it with binary files are not supported and wrote a "not supported" note — even though the file was staged on disk right next to it.

Root cause wasn't the attach path, it was the context-reference layer. A binary @file: ref expanded to a bare warning with no content, so the model saw a failure and gave up. Fixed generally (not PDF-specific) in 7ffc216: a binary ref now injects an actionable block — path, mime type, size, and a nudge to read/convert/view it with tools (and explicitly not to tell the user the type is unsupported). It resolves wherever the agent's tools run (local cwd or the staged copy in a remote workspace), so it can act on the file directly. Covers any binary — PDF, docx, spreadsheets, etc.

Test swaps the old "binary → warning" assertion for one asserting the ref yields no dead-end warning, names the file + on-disk path, and never says "not supported".

@helix4u

Two races in the drop-time eager upload: - Resurrected chip: the success path used addComposerAttachment, which re-appends when the id is gone, so a file removed mid-upload reappeared once the upload resolved. Add updateComposerAttachment (update-only; no-op when the chip was removed) and use it on both the eager success path and submit-time sync. - Duplicate upload: submit-time sync didn't join an eager upload still in flight, so drop-then-Enter could fire file.attach twice and leave a duplicate under .hermes/desktop-attachments/. Track in-flight eager uploads by id and await the pending one before deciding to re-upload, reusing its gateway ref. Tests: composer-store no-resurrect unit tests + a join-on-submit integration test asserting a single file.attach. Addresses @helix4u review on NousResearch#43109.

…ops upload The message-edit composer staged dropped OS files asynchronously with no visible state, so confirming the edit before the upload resolved could send the message without the gateway-side ref (helix4u review note on NousResearch#43109). Add a staging flag: while uploadOsDropRefs is in flight, show a small spinner pill in the bubble and block submit (disabled send button + submitEdit guard) so the edit can't outrace the ref insertion. New `attachingFile` i18n string across en/zh/zh-hant/ja.

…6MB cap Remote attachments read their bytes through the readFileDataUrl IPC, which is hard-capped at 16MB and rejects with a raw "file is too large (N bytes; limit M bytes)" string straight into the failure toast (helix4u review note on NousResearch#43109). Translate that into "<file> is too large to upload to the remote gateway (max 16 MB)", parsing the limit out of the message so it tracks the real cap. Applies to both the image and non-image remote read paths; non-cap errors pass through unchanged. Adds unit coverage for both.

@helix4u

Two races in the drop-time eager upload: - Resurrected chip: the success path used addComposerAttachment, which re-appends when the id is gone, so a file removed mid-upload reappeared once the upload resolved. Add updateComposerAttachment (update-only; no-op when the chip was removed) and use it on both the eager success path and submit-time sync. - Duplicate upload: submit-time sync didn't join an eager upload still in flight, so drop-then-Enter could fire file.attach twice and leave a duplicate under .hermes/desktop-attachments/. Track in-flight eager uploads by id and await the pending one before deciding to re-upload, reusing its gateway ref. Tests: composer-store no-resurrect unit tests + a join-on-submit integration test asserting a single file.attach. Addresses @helix4u review on NousResearch#43109.

…ops upload The message-edit composer staged dropped OS files asynchronously with no visible state, so confirming the edit before the upload resolved could send the message without the gateway-side ref (helix4u review note on NousResearch#43109). Add a staging flag: while uploadOsDropRefs is in flight, show a small spinner pill in the bubble and block submit (disabled send button + submitEdit guard) so the edit can't outrace the ref insertion. New `attachingFile` i18n string across en/zh/zh-hant/ja.

…6MB cap Remote attachments read their bytes through the readFileDataUrl IPC, which is hard-capped at 16MB and rejects with a raw "file is too large (N bytes; limit M bytes)" string straight into the failure toast (helix4u review note on NousResearch#43109). Translate that into "<file> is too large to upload to the remote gateway (max 16 MB)", parsing the limit out of the message so it tracks the real cap. Applies to both the image and non-image remote read paths; non-cap errors pass through unchanged. Adds unit coverage for both.

@helix4u

Two races in the drop-time eager upload: - Resurrected chip: the success path used addComposerAttachment, which re-appends when the id is gone, so a file removed mid-upload reappeared once the upload resolved. Add updateComposerAttachment (update-only; no-op when the chip was removed) and use it on both the eager success path and submit-time sync. - Duplicate upload: submit-time sync didn't join an eager upload still in flight, so drop-then-Enter could fire file.attach twice and leave a duplicate under .hermes/desktop-attachments/. Track in-flight eager uploads by id and await the pending one before deciding to re-upload, reusing its gateway ref. Tests: composer-store no-resurrect unit tests + a join-on-submit integration test asserting a single file.attach. Addresses @helix4u review on #43109.

…ops upload The message-edit composer staged dropped OS files asynchronously with no visible state, so confirming the edit before the upload resolved could send the message without the gateway-side ref (helix4u review note on #43109). Add a staging flag: while uploadOsDropRefs is in flight, show a small spinner pill in the bubble and block submit (disabled send button + submitEdit guard) so the edit can't outrace the ref insertion. New `attachingFile` i18n string across en/zh/zh-hant/ja.

…6MB cap Remote attachments read their bytes through the readFileDataUrl IPC, which is hard-capped at 16MB and rejects with a raw "file is too large (N bytes; limit M bytes)" string straight into the failure toast (helix4u review note on #43109). Translate that into "<file> is too large to upload to the remote gateway (max 16 MB)", parsing the limit out of the message so it tracks the real cap. Applies to both the image and non-image remote read paths; non-cap errors pass through unchanged. Adds unit coverage for both.

…-drops fix(desktop): stage dropped files into the remote session workspace

alt-glitch added type/bug Something isn't working P3 Low — cosmetic, nice to have labels Jun 9, 2026

OutThisLife added 3 commits June 9, 2026 18:21

OutThisLife enabled auto-merge June 10, 2026 00:18

OutThisLife merged commit aecdacb into main Jun 10, 2026
22 checks passed

OutThisLife deleted the fix/desktop-remote-attach-drops branch June 10, 2026 00:22

alt-glitch pushed a commit that referenced this pull request Jun 14, 2026

Merge pull request #43109 from NousResearch/fix/desktop-remote-attach…

47c177f

…-drops fix(desktop): stage dropped files into the remote session workspace

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix(desktop): stage dropped files into the remote session workspace#43109

fix(desktop): stage dropped files into the remote session workspace#43109
OutThisLife merged 6 commits into
mainfrom
fix/desktop-remote-attach-drops

OutThisLife commented Jun 9, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Jun 9, 2026 •

edited

Loading

Uh oh!

helix4u commented Jun 9, 2026

Uh oh!

OutThisLife commented Jun 9, 2026

Uh oh!

OutThisLife commented Jun 10, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

OutThisLife commented Jun 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Also in here

Test plan

Uh oh!

github-actions Bot commented Jun 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔎 Lint report: fix/desktop-remote-attach-drops vs origin/main

ruff

ty (type checker)

Uh oh!

helix4u commented Jun 9, 2026

Uh oh!

OutThisLife commented Jun 9, 2026

Uh oh!

OutThisLife commented Jun 10, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

OutThisLife commented Jun 9, 2026 •

edited

Loading

github-actions Bot commented Jun 9, 2026 •

edited

Loading

🔎 Lint report: `fix/desktop-remote-attach-drops` vs `origin/main`