Fix listFiles() to follow symbolic links#6808
Conversation
The listFiles() method was not following symbolic links because Files.walkFileTree() defaults to not following them. This caused listFiles() to fail to locate files when called on a symlinked directory. Added FileVisitOption.FOLLOW_LINKS to the walkFileTree call to ensure symlinks are properly resolved. Fixes #6807 Generated by Claude Code Signed-off-by: adamrtalbot <12817534+adamrtalbot@users.noreply.github.com>
✅ Deploy Preview for nextflow-docs-staging canceled.
|
|
Testing: #!/usr/bin/env nextflow
params.dir = "$projectDir/test_data"
workflow {
channel.of(file(params.dir))
.map { dir -> dir.listFiles() }
.flatten()
.view { "Found: ${it.name}" }
}mkdir -p test_data_real
echo "hello" > test_data_real/file.txt
ln -s test_data_real test_data
echo "hello" > file_real.txt
ln -s file_real.txt test_data/file_link.txtNextflow 25.10.3: > nextflow run test-symlinks.nf
N E X T F L O W ~ version 25.10.3
Launching `test-symlinks.nf` [romantic_poitras] DSL2 - revision: dc961e3873
Found: test_dataThis branch: > make compile && ./launch.sh run test-symlinks.nf
./gradlew compile exportClasspath
<redacted>
DONE Thu Feb 5 10:39:28 GMT 2026
N E X T F L O W ~ version 25.12.0-edge
Launching `test-symlinks.nf` [pedantic_tuckerman] - revision: dc961e3873
Found: file.txt
Found: file_link.txt |
|
For completion, here's using the new workflow {
channel.of(file(params.dir))
.map { dir -> dir.listDirectory() }
.flatten()
.view { "Found: ${it.name}" }
}Current N E X T F L O W ~ version 25.12.0-edge
Launching `test-symlinks.nf` [romantic_boyd] - revision: a10ccfe3f8
Found: test_dataThis branch: N E X T F L O W ~ version 25.12.0-edge
Launching `test-symlinks.nf` [mighty_mcclintock] - revision: a10ccfe3f8
Found: file.txt
Found: file_link.txt |
…w-io#6808) Signed-off-by: adamrtalbot <12817534+adamrtalbot@users.noreply.github.com>
|
Folks, i've reverted this PR. The change looks harmless at first glance but has a wider blast radius than expected. Affected production callers
Task hash impact
This is a semantic difference that could introduce bugs both in internal code and in user pipelines calling |
|
Thanks @pditommaso, well spotted. |
|
That's fine, we can just apply this change to @adamrtalbot please re-submit this PR, making |
Summary
listFiles()to properly follow symbolic links by addingFileVisitOption.FOLLOW_LINKSto thewalkFileTreecallProblem
The
listFiles()method was not following symbolic links becauseFiles.walkFileTree()defaults to not following them. This causedlistFiles()to fail to locate files when:A workaround using
toRealPath()was suggested, but this doesn't work for cloud storage (S3 throwsUnsupportedOperationException, Azure just returnstoAbsolutePath()).Solution
Added
FileVisitOption.FOLLOW_LINKSto thewalkFileTreecall inlistFiles0(). This is safe for cloud filesystems since:FOLLOW_LINKSis effectively a no-opEnumSet.of(FileVisitOption.FOLLOW_LINKS)internally for downloadsFixes #6807
Test plan
should list files in symlinked directoryFilesExTesttests pass🤖 Generated with Claude Code