Skip to content

[Buckets] Support rsync-style trailing slash in copy_files#4187

Merged
Wauplin merged 1 commit into
mainfrom
fix/cp-trailing-slash-rsync-semantics
May 5, 2026
Merged

[Buckets] Support rsync-style trailing slash in copy_files#4187
Wauplin merged 1 commit into
mainfrom
fix/cp-trailing-slash-rsync-semantics

Conversation

@Wauplin

@Wauplin Wauplin commented May 5, 2026

Copy link
Copy Markdown
Collaborator

Context: @Pierrci 's report in private slack. Currently it's not possible to copy the content of a folder from a bucket, to the root of another bucket.

Solution suggested by Claude:

  • "copy src dst/" => copy src files under dst/src
  • "copy src/ dst/" => copy src files under dst/

This follows rsync style, not UNIX cp style so it's more of a design question rather than implementation problem. What's your take on this @hanouticelina @Pierrci @julien-c ?

Before (no way to avoid nesting):

# Both with and without trailing slash would nest nvidia/ inside target
hf buckets cp hf://buckets/src/nvidia/ hf://buckets/dst/
# => dst/nvidia/file1.txt, dst/nvidia/file2.txt

After (rsync-style trailing slash):

# Without trailing slash: nests (cp -r behavior)
hf buckets cp hf://buckets/src/nvidia hf://buckets/dst/
# => dst/nvidia/file1.txt, dst/nvidia/file2.txt

# With trailing slash: copies contents only (rsync behavior)
hf buckets cp hf://buckets/src/nvidia/ hf://buckets/dst/
# => dst/file1.txt, dst/file2.txt

The implementation is a one-line condition change in _resolve_target_path — when source_is_contents_only (trailing / detected before parsing), the nesting step is skipped.

Updated docs in CLI guide, buckets guide, CLI reference, and copy_files docstring.

🤖 Generated with Claude Code


Note

Medium Risk
Changes folder-copy path resolution in HfApi.copy_files based on a trailing /, which could alter destination layouts for existing callers that relied on the previous nesting behavior. Covered by a new regression test, but impacts data placement in buckets.

Overview
Folder copy semantics changed for buckets/repo-to-bucket copies. HfApi.copy_files now treats a trailing / on the source as “copy contents only” (rsync-style), skipping the usual nesting of the source directory when the destination exists.

Docs + CLI examples updated, and a new test asserts the new trailing-slash behavior when copying a folder into an existing destination directory.

Reviewed by Cursor Bugbot for commit 3562867. Bugbot is set up for automated code reviews on this repo. Configure here.

…lder contents

When copying folders between buckets, a trailing `/` on the source path now
uses rsync semantics: copy the *contents* of the folder into the destination
without nesting the source folder itself. Without the trailing slash, the
existing `cp -r` behavior is preserved (source folder nested inside destination).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@bot-ci-comment

bot-ci-comment Bot commented May 5, 2026

Copy link
Copy Markdown

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@Wauplin Wauplin left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✔️

@julien-c julien-c left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, actually I think it's the behavior I want as well.

@Pierrci

Pierrci commented May 5, 2026

Copy link
Copy Markdown
Member

^same here, thanks!

@Wauplin Wauplin merged commit 51adb8f into main May 5, 2026
21 checks passed
@Wauplin Wauplin deleted the fix/cp-trailing-slash-rsync-semantics branch May 5, 2026 15:15
@huggingface-hub-bot

Copy link
Copy Markdown
Contributor

This PR has been shipped as part of the v1.14.0 release.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants