Some notes as I work on this:

Moving filter later in the workflow means we can no longer support origin-specific filter configuration parameters like this one from the example_multiple_inputs build:

# STAGE 1: Input-specific filtering parameters
filter:
  aus:
    min_length: 5000 # Allow shorter genomes. Parameter used to filter alignment.
    skip_diagnostics: True # skip diagnostics (which can remove genomes) for this input

These would need to be converted to top-level filter parameters like:

# STAGE 1: Input-specific filtering parameters
filter:
  min_length: 5000
  skip_diagnostics: True

Actually, the diagnostic rule doesn’t make sense at the beginning of the workflow anymore since it analyzes each input “origin” and we don’t have a mapping of origins to build names after subsampling. It would make more sense to do diagnostics after subsampling and alignment (with Nextclade), before filtering.

We need the sequence index at the beginning of the workflow, still, because priorities use the sequence index and the proximity calculation need to extract subsampled sequences for the focal set.

The priorities script only seems to use the number of Ns in each sequence for its calculations, so maybe we could pull this information from somewhere else (e.g., seqtk comp, etc.).

We could also extract subsampled sequences with samtools, seqkit, faSomeRecords, etc. depending on compatibility of those tools with our sequence compression format.

Fortunately, we only need to build the sequence index in the case with priorities. Here is the DAG without priorities (assuming unaligned input):

Here is the DAG with priorities (where we need to index sequences and extract sequences for focal subsampled set):

Move index/filter after subsampling #810

Description

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions