fix: Fall back to mkdir when clonefile fails on non-APFS filesystems by kilian-hu · Pull Request #2260 · conda/rattler

kilian-hu · 2026-03-19T12:42:38Z

Description

I think the macOS directory clonefile optimization (using reflinks) implemented in #995 overlooked the case for filesystems which do not support reflinks. I noticed an error like this

Error:   × failed to link libuv-1.51.0-h58003a5_1.conda
  ├─▶ failed to create directory '/some/path/.pixi/envs/some-env/include'
  ╰─▶ Operation not supported (os error 45)

and in my case this is on a HFS+ filesystem, but I think the bug would also occur on NFS, SMB, exFAT and so on.

In the case of single files, there is some fallback logic if reflinks are not supported, see the following

rattler/crates/rattler/src/install/link.rs

Lines 426 to 443 in 6462049

    
           Err(e) if e.kind() == ErrorKind::AlreadyExists => { 
        
               fs::remove_file(destination_path).map_err(|err| { 
        
                   LinkFileError::IoError(String::from("removing clobbered file"), err) 
        
               })?; 
        
           } 
        
           Err(e) if e.kind() == ErrorKind::Unsupported && allow_hard_links => { 
        
               return hardlink_to_destination(source_path, destination_path); 
        
           } 
        
           Err(e) if e.kind() == ErrorKind::Unsupported && !allow_hard_links => { 
        
               return copy_to_destination(source_path, destination_path); 
        
           } 
        
           Err(_) => { 
        
               return if allow_hard_links { 
        
                   hardlink_to_destination(source_path, destination_path) 
        
               } else { 
        
                   copy_to_destination(source_path, destination_path) 
        
               }; 
        
           }

but for the directory case there is no such fallback logic and it just errors, see the following

rattler/crates/rattler/src/install/mod.rs

Lines 792 to 811 in 6462049

    
               // reflink the whole directory if possible 
        
               // currently this does not handle noarch packages 
        
               match reflink_copy::reflink(package_dir.join(&directory), &full_path) { 
        
                   Ok(_) => { 
        
                       created_directories.insert(directory.clone()); 
        
                       // remove paths that we just reflinked (everything that starts with the directory) 
        
                       let (matching, non_matching): (HashMap<_, _>, HashMap<_, _>) = 
        
                           paths_by_directory 
        
                               .drain() 
        
                               .partition(|(k, _)| k.starts_with(&directory)); 
        
                       // Store matching paths in reflinked_files 
        
                       reflinked_files.extend(matching); 
        
                       // Keep non-matching paths in paths_by_directory 
        
                       paths_by_directory = non_matching; 
        
                   } 
        
                   Err(e) if e.kind() == ErrorKind::AlreadyExists => (), 
        
                   Err(e) => return Err(InstallError::FailedToCreateDirectory(full_path, e)), 
        
               } 
        
           } else {

The fix adds a catch-all fallback that creates the directory and disables further clonefile attempts for the remaining directories in the package. Individual files are then linked via hardlink (or copy).

This mirrors the approach already used by the single file reflink code and restores installation on non-APFS macOS volumes.

Open question: Currently I'm changing allow_ref_links to be mutable such that after the first clonefile failure we set it to false to no spam further failures. But I'm not sure if this is a clean solution.

How Has This Been Tested?

All existing tests pass
Manually tested on an HFS+ disk image:
1. Create and mount: hdiutil create -size 2g -fs HFS+ -volname testhfs /tmp/testhfs.dmg && hdiutil attach /tmp/testhfs.dmg
2. Init a pixi project on the volume: cd /Volumes/testhfs/project && pixi init && pixi add python
3. First install to populate the cache on HFS+: PIXI_CACHE_DIR=/Volumes/testhfs/cache pixi install
4. Remove the environment and reinstall (cache and target now both on HFS+, so allow_ref_links = true): rm -rf .pixi && PIXI_CACHE_DIR=/Volumes/testhfs/cache pixi install
- Before fix: step 4 fails with failed to create directory ... Operation not supported (os error 45)
- After fix: step 4 completes successfully, packages installed via hardlinks

AI Disclosure

This PR contains AI-generated content.
- I have tested any AI-generated content in my PR.
- I take responsibility for any AI-generated content in my PR.
  Tools: Claude Code with Opus 4.6

Checklist:

I have performed a self-review of my own code
I have commented my code, particularly in hard-to-understand areas
I have made corresponding changes to the documentation
I have added sufficient tests to cover my changes.

wolfv

Looks good to me!

fix: Fall back to mkdir when clonefile fails on non-APFS filesystems

9d0352f

wolfv reviewed Mar 19, 2026

View reviewed changes

baszalmstra approved these changes Mar 19, 2026

View reviewed changes

baszalmstra merged commit 4aaefe9 into conda:main Mar 19, 2026
20 checks passed

github-actions Bot mentioned this pull request Mar 19, 2026

chore: release #2256

Merged

kilian-hu deleted the fix/clonefile-fallback-non-apfs branch March 19, 2026 19:18

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix: Fall back to mkdir when clonefile fails on non-APFS filesystems#2260

fix: Fall back to mkdir when clonefile fails on non-APFS filesystems#2260
baszalmstra merged 1 commit intoconda:mainfrom
kilian-hu:fix/clonefile-fallback-non-apfs

kilian-hu commented Mar 19, 2026 •

edited

Loading

Uh oh!

wolfv left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

	Err(e) if e.kind() == ErrorKind::AlreadyExists => {
	fs::remove_file(destination_path).map_err(\|err\| {
	LinkFileError::IoError(String::from("removing clobbered file"), err)
	})?;
	}
	Err(e) if e.kind() == ErrorKind::Unsupported && allow_hard_links => {
	return hardlink_to_destination(source_path, destination_path);
	}
	Err(e) if e.kind() == ErrorKind::Unsupported && !allow_hard_links => {
	return copy_to_destination(source_path, destination_path);
	}
	Err(_) => {
	return if allow_hard_links {
	hardlink_to_destination(source_path, destination_path)
	} else {
	copy_to_destination(source_path, destination_path)
	};
	}

	// reflink the whole directory if possible
	// currently this does not handle noarch packages
	match reflink_copy::reflink(package_dir.join(&directory), &full_path) {
	Ok(_) => {
	created_directories.insert(directory.clone());
	// remove paths that we just reflinked (everything that starts with the directory)
	let (matching, non_matching): (HashMap<_, _>, HashMap<_, _>) =
	paths_by_directory
	.drain()
	.partition(\|(k, _)\| k.starts_with(&directory));

	// Store matching paths in reflinked_files
	reflinked_files.extend(matching);
	// Keep non-matching paths in paths_by_directory
	paths_by_directory = non_matching;
	}
	Err(e) if e.kind() == ErrorKind::AlreadyExists => (),
	Err(e) => return Err(InstallError::FailedToCreateDirectory(full_path, e)),
	}
	} else {

Conversation

kilian-hu commented Mar 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

How Has This Been Tested?

AI Disclosure

Checklist:

Uh oh!

wolfv left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

kilian-hu commented Mar 19, 2026 •

edited

Loading