Skip to content

Add stats calculations and results display#342

Merged
bamader merged 2 commits into
mainfrom
update-metrics-tests
May 1, 2025
Merged

Add stats calculations and results display#342
bamader merged 2 commits into
mainfrom
update-metrics-tests

Conversation

@bamader

@bamader bamader commented May 1, 2025

Copy link
Copy Markdown
Collaborator

Description

This PR is a fast-follow to a conversation had on 4/30 in which we established we wanted a standardized performance calculation script to be uploaded to our repo. This script integrates into the run_test.py script executed as part of algorithm tests, and gives us calculations for: raw counting stats (TP, FP, TN, FN), derived match quality stats (sens, spec, F1, PPV), and labels for matches graded possible by either system. This should standardize the way we calculate and report performance going forward.

NOTE: The change numbers look massive, but that's because we got the all-clear to upload our more robust test and seed data sets into the algorithm test configuration. There are only a couple changed code files, most of this is CSV uploading.

<--------------------- REMOVE THE LINES BELOW BEFORE MERGING --------------------->

Checklist

Please review and complete the following checklist before submitting your pull request:

  • I have ensured that the pull request is of a manageable size, allowing it to be reviewed within a single session.
  • I have reviewed my changes to ensure they are clear, concise, and well-documented.
  • I have updated the documentation, if applicable.
  • I have added or updated test cases to cover my changes, if applicable.
  • I have minimized the number of reviewers to include only those essential for the review.

Checklist for Reviewers

Please review and complete the following checklist during the review process:

  • The code follows best practices and conventions.
  • The changes implement the desired functionality or fix the reported issue.
  • The tests cover the new changes and pass successfully.
  • Any potential edge cases or error scenarios have been considered.

@codecov

codecov Bot commented May 1, 2025

Copy link
Copy Markdown

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 98.11%. Comparing base (952a4da) to head (9ff53a0).
Report is 3 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #342      +/-   ##
==========================================
- Coverage   98.23%   98.11%   -0.13%     
==========================================
  Files          33       33              
  Lines        1924     1958      +34     
==========================================
+ Hits         1890     1921      +31     
- Misses         34       37       +3     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Comment thread tests/algorithm/scripts/run_test.py
Comment thread tests/algorithm/scripts/calculate_metrics.py Outdated
@bamader bamader merged commit 2d055ce into main May 1, 2025
15 checks passed
@bamader bamader deleted the update-metrics-tests branch May 1, 2025 16:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants