Skip to content

Run-order -n summary (rather than alphabetical) #1767

@emmahodcroft

Description

@emmahodcroft

Problem
I often run workflows that are really long, with loads of steps. Since they're big jobs, I like to check with -n before I start them, in particular to glance over what's going to run & the number of calls to each rule.
However, I've messed up a couple of times by not noticing that the starting point differs from what I expect (usually due to leftover old files). This is particularly easy to miss because the -n summary shows alphabetically - so it's really easy to miss if a step is missing or present when you aren't expecting it to be. Yes, prior to that every command is printed i order of execution -- but for me this is so long it's impossible to view in any reasonable amount of time.

Proposed solution
It would be fantastic if there was a switch on -n that allowed you to print the summary of execution either alphabetically (default & current behaviour) or in order of execution. For me this would allow a very simple way to check that the workflow is starting where I think it is & proceeding in the order I expect.

Alternatives
I'm work primarily remotely on a CL only server, so viewing the DAG diagrams is not very straightforward. I have tried this but my workflows are seemingly complex enough that rendering them is an issue for ssh Visual Studio Code (the only way I can view them without transferring them locally). Also, the DAG diagrams, while sometimes very useful, are a bit more complex than what I need here - I just want a quick check that things are starting where I think they are.

Additional context
I know that sometimes runs are not necessarily linear and they can get complicated, but at least for me, any approximation would be fine - for example, printing them in the order they show up in the 'every command print' that shows up before the summary, then just collapsing duplicate calls - or forcing you to pick one file endpoint to print one 'example' - these would also be fine.

What do I mean when I say '-n summary'?
This thing:

Job counts:
        count   jobs
        34      aa_muts_explicit
        34      add_branch_labels
        34      add_labels
        34      adjust_metadata_regions
        1       all
        34      ancestral
        34      annotate_metadata_with_index
        34      build_align
        1       cat_exclude_sites
        34      clade_files
        34      clades
        34      combine_samples
        34      diagnostic
        34      distances
        34      export
        34      extract_cluster
        34      filter
        34      filter_cluster
        34      finalize_swiss
        34      include_hcov19_prefix
        34      index
        1       index_sequences
        34      logistic_growth
        34      mask
....

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions