Add ability to overwrite regular files and symbolic links in NIO FileSystem by stepan-ulyanin · Pull Request #3470 · apple/swift-nio

stepan-ulyanin · 2025-12-30T06:08:28Z

Adds a new parameter to the NIOFS FileSystem's copyItem that allows to atomically overwrite regular files and symbolic links.

Note that I opted in to not implement the overwrite for directories in this PR as it seems that there is no atomic way to overwrite directory trees on Posix systems - do we want to do it non-atomically?

Motivation:

Per #3403 it is not uncommon to want to copy an item and not care if the copied item overwrites an existing one.

Modifications:

Adds a new overwriting: Bool parameter to NIOFS FileSystem's copyItem.
Adds new syscall wrappers for TOCTOU-protected atomic operations:
1. system_renameatx_np / Syscall.rename(from:relativeTo:to:relativeTo:options:) - Darwin directory-relative rename
2. system_symlinkat / Syscall.symlinkat - create symlink relative to directory FD
3. system_unlinkat / Syscall.unlinkat - unlink file relative to directory FD
Implements atomic overwrite for both regular files and symbolic links.
1. For regular files:
  1. On Darwin: a COPYFILE_UNLINK flag is added to copyfile(2)
  2. On Linux: an atomic rename is used - the file is copied to a temporary destination first and then atomically renamed, using renameat2(2) which overwrites the existing file.
2. For symlinks:
  1. On Darwin: an atomic rename is used - the symlink is copied to a temporary destination first and then atomically renamed, using renameatx_np(2) which overwrites the existing symlink.
  2. On Linux: an atomic rename is used - the symlink is copied to a temporary destination first and then atomically renamed, using renameat2(2) which overwrites the existing symlink.
Adds test cases to the integration test suite.
Updates the _NIOFileSystem FileSystem's copyItem for backward compatibility.

Tests

Syscall tests:

test_symlinkat
test_unlinkat
test_renameatx_np

Integration tests:

testCopyFileOverwritingExistingFile
testCopyFileOverwritingNonExistingFile
testCopyFileOverwritingCleansUpTempFile
testCopyFileOverwritingPreservesPermissions
testCopySymlinkOverwritingExistingSymlink
testCopySymlinkOverwritingNonExistingSymlink
testCopySymlinkOverwritingCleansUpTempLink
testCopyFileOverwritingExistingSymlink
testCopySymlinkOverwritingExistingFile

Result:

Users will be able to copy a regular file or a symbolic link, overwriting an existing regular file or existing symbolic link.

…py with overwrite

…lementation

…files

…opyFileOverwritingExistentDestination` test

…th a file and file with a symlink

…pan-ulyanin/swift-nio into su/add-overwriting-file-copy

glbrntt

Thanks for opening this! This is quite a large set of changes, can I suggest that you split this into (at least) two PRs. Maybe (1) the syscall changes, and (2) the file system changes?

One notable omission from a quick skim is the detailed error descriptions for the syscalls (take look at FileSystemError+Syscall.swift).

It might actually be easier to do this on _NIOFileSystem first and then bring over the changes to NIOFS as a 3rd PR but I'll leave that decision to you.

stepan-ulyanin · 2026-02-03T17:36:16Z

Thanks @glbrntt, I think that is a good idea, let open smaller PR in the coming weeks

stepan-ulyanin · 2026-02-07T01:51:44Z

Hey @glbrntt, here is the first PR, adding the system calls -> #3505

…3505) Adds three new system call wrappers for the `symlinkat`, `renameatx_np`, and `unlinkat` system calls. ### Motivation: Related to #3403 and #3470. This PR adds syscall wrappers needed to atomically overwrite existing files or symlinks at the destination during copy operations. On Linux, atomic overwrites require a "copy to temp file, then rename" strategy. We use the `*at` family of syscalls (which operate relative to directory file descriptors) to avoid TOCTOU race conditions. ### Modifications: 1. Adds three system call wrappers for the `symlinkat`, `renameatx_np`, and `unlinkat` system calls. 2. Adds related tests 3. Updates the `FileSystemError` for `symlink` and `unlink` to take in the system call name to allow for the `*at` names to be passed.

stepan-ulyanin · 2026-02-11T07:18:04Z

@glbrntt here is the PR for the overwriting: logic in NIOFS -> #3508. CI/CD is failing but it looks like the issue is unrelated to the changes. When we get this one in, I will add the same changes to the _NIOFileSystem for backward compatibility in a separate PR.

…3508) Adds capability to NIOFS to copy regular files and symlink, allowing to overwrite the destination. ### Motivation: Per #3403 and #3470 we want to add `replaceExisting: bool` to `FileSystem.copyItem`. ### Modifications: 1. Adds `replaceExisting: bool` parameter to `FileSystemProtocol.copyItem`. 2. Adds `replaceExisting: bool` parameter to `FileSystem.copyItem` and implementation for regular files and symbolic links. 3. Adds tests. --------- Co-authored-by: George Barnett <gbarnett@apple.com>

stepan-ulyanin · 2026-02-27T01:54:28Z

@glbrntt I ported the syscall changes to _NIOFileSystem in #3524.

…to the `_NIOFileSystem` module (#3524) Ports the added system calls from `NIOFS` (from #3505) to `_NIOFileSystem`: > Adds three new system call wrappers for the `symlinkat`, `renameatx_np`, and `unlinkat` system calls. ### Motivation: `_NIOFileSystem` module is around for backward compatibility - hence, we are porting the new feature to it. ### Modifications: Exactly same as in #3505 but applied to the `_NIOFileSystem`: > Related to #3403 and #3470. This PR adds syscall wrappers needed to atomically overwrite existing files or symlinks at the destination during copy operations. On Linux, atomic overwrites require a "copy to temp file, then rename" strategy. We use the `*at` family of syscalls (which operate relative to directory file descriptors) to avoid TOCTOU race conditions.

stepan-ulyanin · 2026-03-09T22:56:30Z

Ok, we have added all of these changes in separate PRs, closing this one.

stepan-ulyanin and others added 19 commits December 28, 2025 23:45

Introduce overwrite parameter, update the Darwin platform atomic co…

979f8fe

…py with overwrite

Add implementation to NIOFS, add tests for file copy with overwrite

2b8f11a

Add implementations for Glibc, Musl, and Bionic

10942dc

Update the _NIOFileSystem implementation to be in sync with NIOFS imp…

098e2ac

…lementation

Update some variable names in _copyRegularFile

cf3e311

Remove the test involving testing the sendfile(2) chunking for large …

9434fc3

…files

Remove an incorrect comment inlcuding Darwin platform from the `testC…

512c183

…opyFileOverwritingExistentDestination` test

Add some comments and docstrings

3246eb8

Add the symlink copy, using the temp link and remame(2)

52b70de

Update the protocol docstring

0c6b04f

Merge branch 'main' into su/add-overwriting-file-copy

66b7338

Revert back the error messages when destinations can't be opened

5a52992

Update the overwriting docstring in the protocol

596cfb0

Lean out the overwriting description even more

8ba010a

Update the test testCopyFileOverwritingExistingDestination naming

c3ef217

Update the testCopyFileOverwritingNonExistingDestination test name

7d36c81

Clean up the tests and comments

2bb758d

Remove trailing commas

b8d2a7d

Apply formatting

1cc569a

stepan-ulyanin marked this pull request as ready for review December 30, 2025 06:51

Remove more trailing commas

365ad86

stepan-ulyanin marked this pull request as draft December 30, 2025 06:59

stepan-ulyanin added 5 commits December 29, 2025 23:06

Move overwriting to the last position in the paramter list

84a1f71

Remove trailing commas

8c26bcc

Fix the position of the parameter in the test

42705fc

Update the Linux tests to have a discardable result

2dd6ad5

Fix the removingLastComponent call in Linux tests

aad6784

stepan-ulyanin marked this pull request as ready for review December 30, 2025 08:05

stepan-ulyanin mentioned this pull request Dec 30, 2025

NIOFileSystem: provide a function that copies or replaces a file/dir #3403

Open

Add tests for the cases where we would like to overwrite a symlink wi…

4392131

…th a file and file with a symlink

stepan-ulyanin and others added 13 commits January 22, 2026 00:09

Apply formatting

61bdc9e

Line split the deferred file handle closes

1a39b0c

Remove a redundant COPYFILE_UNLINK comments

2903e7b

Use functions instead of closures

c695450

Update the _NIOFileSystem to match NIOFS

2bac123

Unnest some error definitions

d6f0fe8

Update the SystemFileHandle logic

2b6355e

Update error handling

306daae

Remove the test that requires root permissions

7c318bf

Merge branch 'main' into su/add-overwriting-file-copy

cf670bf

Merge branch 'main' into su/add-overwriting-file-copy

4618f35

Update the expectations for the renameatx_np sys call test

672cee6

Merge branch 'su/add-overwriting-file-copy' of https://github.com/ste…

de4253a

…pan-ulyanin/swift-nio into su/add-overwriting-file-copy

stepan-ulyanin marked this pull request as ready for review January 24, 2026 01:46

stepan-ulyanin requested a review from Lukasa January 24, 2026 01:46

Merge branch 'main' into su/add-overwriting-file-copy

efe0c4d

glbrntt requested changes Feb 3, 2026

View reviewed changes

stepan-ulyanin mentioned this pull request Feb 7, 2026

Add symlinkat, renameatx_np, and unlinkat system call wrappers #3505

Merged

stepan-ulyanin mentioned this pull request Feb 11, 2026

Allow to copy files and symlinks while overwriting the destination #3508

Merged

stepan-ulyanin marked this pull request as draft February 20, 2026 02:48

stepan-ulyanin mentioned this pull request Feb 26, 2026

Add symlinkat, renameatx_np, and unlinkat system call wrappers to the _NIOFileSystem module #3524

Merged

stepan-ulyanin closed this Mar 9, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add ability to overwrite regular files and symbolic links in NIO FileSystem#3470

Add ability to overwrite regular files and symbolic links in NIO FileSystem#3470
stepan-ulyanin wants to merge 58 commits intoapple:mainfrom
stepan-ulyanin:su/add-overwriting-file-copy

stepan-ulyanin commented Dec 30, 2025 •

edited

Loading

Uh oh!

glbrntt left a comment •

edited

Loading

Uh oh!

stepan-ulyanin commented Feb 3, 2026

Uh oh!

stepan-ulyanin commented Feb 7, 2026

Uh oh!

stepan-ulyanin commented Feb 11, 2026 •

edited

Loading

Uh oh!

stepan-ulyanin commented Feb 27, 2026

Uh oh!

stepan-ulyanin commented Mar 9, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

stepan-ulyanin commented Dec 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Motivation:

Modifications:

Tests

Result:

Uh oh!

glbrntt left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

stepan-ulyanin commented Feb 3, 2026

Uh oh!

stepan-ulyanin commented Feb 7, 2026

Uh oh!

stepan-ulyanin commented Feb 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

stepan-ulyanin commented Feb 27, 2026

Uh oh!

stepan-ulyanin commented Mar 9, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

stepan-ulyanin commented Dec 30, 2025 •

edited

Loading

glbrntt left a comment •

edited

Loading

stepan-ulyanin commented Feb 11, 2026 •

edited

Loading