can't create parent directories when stageOutMode is rsync #6447
Description
Bug report
Expected behavior and actual behavior
When using storeDir and scratch at the same time, i would think that storeDir would be able to create any parent folders that don't already exist so that rsync doesn't fail.
Steps to reproduce the problem
i have the following test script:
process A {
storeDir "./folder1/folder2/"
stageOutMode 'rsync'
input:
val(x)
output:
path("*.txt")
script:
"""
touch ${x}.txt
"""
}
workflow {
A("input")
}
changing stageOutMode to copy or move will not result in this error. nxf_fs_rsync in the .command.run is probably missing some mkdir commands. in fact, nxf_fs_move and nxf_fs_copy do have mkdir -p commands, and nxf_fs_rsync is missing them.
Program output
ERROR ~ Error executing process > 'A'
Caused by:
Process `A` terminated with an error exit status (3)
Command executed:
touch input.txt
Command exit status:
3
Command output:
(empty)
Command error:
rsync: change_dir#3 "/data/noronhaa/forte_recreate/workspace/demo_rsync/folder1" failed: No such file or directory (2)
rsync error: errors selecting input/output files, dirs (code 3) at main.c(702) [Receiver=3.1.3]
cp: '.command.out' and '.command.out' are the same file
cp: '.command.err' and '.command.err' are the same file
Environment
- Nextflow version: 25.04.7
- Java version: 23
- Operating system: Linux
- Bash version: GNU bash, version 4.4.20(1)-release (x86_64-redhat-linux-gnu)