Skip to content

Strict DAG-building mode to help with debugging #3331

@bsmith89

Description

@bsmith89

Is your feature request related to a problem? Please describe.
I'm frequently frustrated by MissingInputExceptions and other challenges while trying to debug a complex workflow, especially when making extensive use of Callables for deriving input files based on a config.

I believe that this is often due to unintuitive behavior when an input function raises an error (my understanding is that this error is caught and not re-raised as long as a different source for the prerequisite file is found).

Hard-to-reason-about behaviors like these, while they may often result in the correct DAG and fewer errors for the user to sort through, are potentially harmful and allow workflow writers to be less disciplined in their design.

Describe the solution you'd like
I'd like to see a "strict mode" for DAG building, where errors like these are re-raised instead of being caught. I.e. a -Werror -Wall for Snakemake. I believe that in many (all?) cases, workflows can be designed that do not rely on the current lax/dynamic approach to dependency resolution, and that workflows that strive for this ideal are easier to debug.

Ideally, any workflow that runs successfully in strict mode will be resolved identically in the default mode. For some workflow writers, strict mode would serve as a complement to the linter.

Additional context
I'm optimistic that increasing the strictness of DAG resolution may enable other optimizations that speed up DAG building or allow for static analyses.

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Type

No type

Projects

Status

Done

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions