feat(hub): add copyFiles API for remote file copy between buckets and repositories#2121
Conversation
… repositories Implements the 'copy files remotely' API in @huggingface/hub, porting the Python huggingface_hub.HfApi.copy_files functionality to TypeScript/JS. Supports: - Bucket-to-bucket copy (server-side, no data transfer) - Repo (model/dataset/space) to bucket copy - xet-backed files: server-side copy by hash - non-xet files: download + re-upload via commit - Single file and recursive folder copy - hf:// handle parsing with revision support Reference: huggingface/huggingface_hub#3874 Co-authored-by: Lucain <Wauplin@users.noreply.github.com>
|
closing in favor of @julien-c 's PR to come |
|
hmm I'll keep the |
|
split between |
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 2 potential issues.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit f8cc5a9. Configure here.
| state: "error", | ||
| }; | ||
| } | ||
| } |
There was a problem hiding this comment.
Copy failures silently swallowed when batch partially succeeds
Medium Severity
When the copy batch API returns HTTP 422, individual copy failures are yielded as fileProgress error events but processing continues to the delete-operations phase. The commit() wrapper at line 1017 collects these errors and throws, but the error message says "Failed to upload N file(s)" which is misleading for copy operations. More importantly, the addFile batch (line 888) uses the same pattern — on partial failure, both the successfully added files AND successfully copied files persist in the bucket while the caller receives an error. This means a partial copy state is left in the destination bucket with no rollback, and retrying the entire copyFiles call would re-process all files rather than just the failed ones.
Additional Locations (1)
Reviewed by Cursor Bugbot for commit f8cc5a9. Configure here.


Summary
Implements the "copy files remotely" API in
@huggingface/hub, porting the PythonHfApi.copy_filesfunctionality to TypeScript/JS.This enables the "Copy to Bucket" feature on the Hub UI, allowing instant server-side file copy between buckets and from repositories to buckets.
Supported operations
commit()Not supported (yet)
Features
copyFiles()— main function, exported from@huggingface/hubparseHfCopyHandle()— parseshf://handles (buckets, models, datasets, spaces, with@revisionsupport)POST /api/buckets/{id}/batchwith NDJSONcopyFileoperationscommit()infrastructureUsage
Reference
Python implementation: huggingface/huggingface_hub#3874
How to test locally
Unit tests (handle parsing)
The
parseHfCopyHandleunit tests run without any network access.Integration tests (require CI Hub access)
The integration tests (
copyFilesdescribe block) run against the CI Hub athttps://hub-ci.huggingface.co. They:copyFilesTo run them:
Manual testing
You can also test manually against the production Hub:
Slack Thread