Skip to content

Support disabling the --clock-filter-iqd parameter for augur refine#884

Merged
huddlej merged 2 commits intonextstrain:masterfrom
chanzuckerberg:jgadling/augur-clock-disable
Mar 22, 2022
Merged

Support disabling the --clock-filter-iqd parameter for augur refine#884
huddlej merged 2 commits intonextstrain:masterfrom
chanzuckerberg:jgadling/augur-clock-disable

Conversation

@jgadling
Copy link
Copy Markdown
Contributor

@jgadling jgadling commented Mar 9, 2022 •
edited
Loading

Description of proposed changes

Support disabling the --clock-filter-iqd parameter for augur refine. Since --clock-filter-iqd 0 doesn't make sense, we can assume that setting the value to 0 (or even false) should disable it.

Related issue(s)

Fixes #852
Related to #

Testing

For a very simple test config:

inputs:
  - name: "root_ref"
    metadata: "data/references_metadata.tsv"
    sequences: "data/references_sequences.fasta"
  - name: "example_data"
    metadata: "data/example_metadata_aus.tsv.xz"
    sequences: "data/example_sequences_aus.fasta.xz"

builds:
  test:
    subsampling_scheme: test_subsampling
    title: test tree

refine:
  keep_polytomies: True
  clock_filter_iqd: 0
  
subsampling:
  test_subsampling:
    test:
      group_by: "division year month"
      max_sequences: 10
      exclude: "--exclude-where 'region=europe'"
~/ncov# grep clock_filter_iqd my_profiles/builds.yaml
  clock_filter_iqd: 5
~/ncov# rm results/test/tree.nwk && snakemake --printshellcmds auspice/ncov_test.json --profile /tmp/ncov/my_profiles/ --until refine 2>&1 | grep 'augur refine'
        augur refine             --tree results/test/tree_raw.nwk             --alignment results/test/filtered.fasta
             --metadata results/test/metadata_adjusted.tsv.xz             --output-tree results/test/tree.nwk
             --output-node-data results/test/branch_lengths.json             --root Wuhan/Hu-1/2019
             --timetree             --keep-polytomies             --clock-rate 0.0008
             --clock-std-dev 0.0004             --coalescent opt             --date-inference marginal 
            --divergence-unit mutations             --date-confidence
             --no-covariance             --clock-filter-iqd 5 2>&1 | tee logs/refine_test.txt

~/ncov# sed -i 's/filter_iqd:.*/filter_iqd: 0/' my_profiles/builds.yaml
~/ncov# grep clock_filter_iqd my_profiles/builds.yaml
  clock_filter_iqd: 0
~/ncov# rm results/test/tree.nwk && snakemake --printshellcmds auspice/ncov_test.json --profile /tmp/ncov/my_profiles/ --until refine 2>&1 | grep 'augur refine'
        augur refine             --tree results/test/tree_raw.nwk             --alignment results/test/filtered.fasta
             --metadata results/test/metadata_adjusted.tsv.xz             --output-tree results/test/tree.nwk
             --output-node-data results/test/branch_lengths.json             --root Wuhan/Hu-1/2019
             --timetree             --keep-polytomies             --clock-rate 0.0008
             --clock-std-dev 0.0004             --coalescent opt             --date-inference marginal
             --divergence-unit mutations             --date-confidence
             --no-covariance              2>&1 | tee logs/refine_test.txt

Release checklist

If this pull request introduces backward incompatible changes, complete the following steps for a new release of the workflow:

  • Determine the version number for the new release by incrementing the most recent release (e.g., "v2" from "v1").
  • Update docs/src/reference/change_log.md in this pull request to document these changes and the new version number.
  • After merging, create a new GitHub release with the new version number as the tag and release title.

If this pull request introduces new features, complete the following steps:

  • Update docs/src/reference/change_log.md in this pull request to document these changes by the date they were added.

@jgadling jgadling marked this pull request as ready for review March 9, 2022 17:33
@danrlu
Copy link
Copy Markdown
Contributor

danrlu commented Mar 9, 2022

Test runs from this branch. Spot the difference:
image

@huddlej huddlej self-requested a review March 9, 2022 22:18
huddlej
huddlej approved these changes Mar 22, 2022
View reviewed changes
Copy link
Copy Markdown
Contributor

@huddlej huddlej left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for putting this together, @jgadling, and I'm sorry for the late review!

The Snakemake implementation looks like exactly what we need to toggle the clock filter and @danrlu's screenshots confirm that this works as expected. Thank you also for updating the default value in the docs!

I was originally concerned about using 0 to disable the clock filter instead of a YAML value like null that indicates the lack of a value, but it turns out that augur refine internally treats the clock filter IQD value as false-y when it is 0 and skips the filter, so this implementation would work the same with an augur refine command on the terminal as in the workflow.

@huddlej huddlej merged commit 464041d into nextstrain:master Mar 22, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@huddlej huddlej huddlej approved these changes

Assignees

No one assigned

Labels

None yet

Projects

No open projects
Nextstrain planning (archived)
Archived in project

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Allow users to disable clock filter for augur refine

3 participants

@jgadling @danrlu @huddlej