Issue made from #8647 by @jeremy
What is the problem you are having with rclone?
When every source object already exists at the destination and --no-traverse is set, march stalled/deadlocked after 100 entries because dstChan lagged srcChan. Introduced in v1.70.0 with callback-based syncing.
I couldn't get a test case working reliably, but here's a script to reproduce:
#!/usr/bin/env bash
# Reproduces rclone v1.70+ deadlock with --no-traverse when all files exist
# Usage: ./repro.sh [rclone-binary-path]
# Use provided rclone binary or default to system rclone
RCLONE="${1:-rclone}"
echo "Using rclone: $RCLONE"
echo "Version: $($RCLONE version | head -1)"
echo
# Create test directories
src=$(mktemp -d)
dst=$(mktemp -d)
# Create 200 test files (enough to trigger the ~100 file deadlock)
echo "Creating 200 test files..."
for i in {1..200}; do
echo "content$i" > "$src/file$i.txt"
done
# Initial copy to establish destination
echo "Initial copy to destination..."
$RCLONE copy "$src/" "$dst/" --quiet
# Create file list
filelist=$(mktemp)
ls "$src" > "$filelist"
# This command deadlocks in v1.70+ when all files exist at destination
echo "Testing --no-traverse with all files existing (10s timeout)..."
timeout 10s $RCLONE copy "$src/" "$dst/" \
--no-traverse \
--files-from "$filelist" \
--ignore-existing \
--progress
if [ $? -eq 124 ]; then
echo -e "\n❌ FAIL: Command timed out (deadlock detected)"
exit 1
else
echo -e "\n✅ PASS: Command completed successfully"
fi
# Cleanup
rm -rf "$src" "$dst" "$filelist"
Results on 1.69.3, 1.70.2, and with this patch:
Using rclone: rclone
Version: rclone v1.69.3
Creating 200 test files...
Initial copy to destination...
Testing --no-traverse with all files existing (10s timeout)...
Transferred: 0 B / 0 B, -, 0 B/s, ETA -
Checks: 200 / 200, 100%
Elapsed time: 0.0s
✅ PASS: Command completed successfully
Using rclone: rclone
Version: rclone v1.70.2
Creating 200 test files...
Initial copy to destination...
Testing --no-traverse with all files existing (10s timeout)...
Transferred: 0 B / 0 B, -, 0 B/s, ETA -
Checks: 0 / 0, -, Listed 200
Elapsed time: 9.5s
❌ FAIL: Command timed out (deadlock detected)
Using rclone: ./rclone
Version: rclone v1.71.0-DEV
Creating 200 test files...
Initial copy to destination...
Testing --no-traverse with all files existing (10s timeout)...
Transferred: 0 B / 0 B, -, 0 B/s, ETA -
Checks: 200 / 200, 100%, Listed 200
Elapsed time: 0.0s
✅ PASS: Command completed successfully
What is your rclone version (output from rclone version)
v1.70.2
The associated forum post URL from https://forum.rclone.org
How to use GitHub
- Please use the 👍 reaction to show that you are affected by the same issue.
- Please don't comment if you have no relevant information to add. It's just extra noise for everyone subscribed to this issue.
- Subscribe to receive notifications on status change and new comments.
Issue made from #8647 by @jeremy
What is the problem you are having with rclone?
When every source object already exists at the destination and
--no-traverseis set, march stalled/deadlocked after 100 entries because dstChan lagged srcChan. Introduced in v1.70.0 with callback-based syncing.I couldn't get a test case working reliably, but here's a script to reproduce:
Results on 1.69.3, 1.70.2, and with this patch:
Using rclone: rclone Version: rclone v1.69.3 Creating 200 test files... Initial copy to destination... Testing --no-traverse with all files existing (10s timeout)... Transferred: 0 B / 0 B, -, 0 B/s, ETA - Checks: 200 / 200, 100% Elapsed time: 0.0s ✅ PASS: Command completed successfullyUsing rclone: rclone Version: rclone v1.70.2 Creating 200 test files... Initial copy to destination... Testing --no-traverse with all files existing (10s timeout)... Transferred: 0 B / 0 B, -, 0 B/s, ETA - Checks: 0 / 0, -, Listed 200 Elapsed time: 9.5s ❌ FAIL: Command timed out (deadlock detected)Using rclone: ./rclone Version: rclone v1.71.0-DEV Creating 200 test files... Initial copy to destination... Testing --no-traverse with all files existing (10s timeout)... Transferred: 0 B / 0 B, -, 0 B/s, ETA - Checks: 200 / 200, 100%, Listed 200 Elapsed time: 0.0s ✅ PASS: Command completed successfullyWhat is your rclone version (output from
rclone version)v1.70.2
The associated forum post URL from
https://forum.rclone.orgHow to use GitHub