Conversation
✅ Deploy Preview for nextflow-docs-staging canceled.
|
|
The problem with the stdin params right now is that in the case of a 3-step chain, the second step blocks to wait for stdin and the third step skips the stdin because it is not "available". This leads to a lock error since the first and third run try to lock the history file. If I skip the I'll have to see how composable CLI tools normally handle this. I assume each tool has to initialize stdout before waiting for stdin, in order to signal to the next tool in the pipeline that stdin is coming |
Signed-off-by: Ben Sherman <bentshermann@gmail.com>
Signed-off-by: Ben Sherman <bentshermann@gmail.com>
Signed-off-by: Ben Sherman <bentshermann@gmail.com>
13b9af5 to
474cc1e
Compare
|
Notes:
TODO: trim PR scope to just the workflow outputs JSON with CLI option |
|
Closing in favor of #6875 . We can revisit if we decide to enable params via stdin |
This PR adds several features to facilitate pipeline chaining based on workflow inputs/outputs
Demo project: bentsherman/pipeline-chaining-demo
The JSON output is very similar to the lineage record (
lid://<workflow-run-hash>#output), so maybe we could use that instead. The lineage record has the advantage of using LIDs for files, but the JSON structure doesn't quite match the params file.With a bit more magic we could even make the pipeline chain as simple as a Bash pipeline:
But the main thing for now is the basic reading/writing of inputs/outputs, then enhanced validation with schemas and record types.