Skip to content

Remove the remnants of dibbs basic#214

Merged
bamader merged 6 commits into
mainfrom
remove-dibbs-basic
Feb 13, 2025
Merged

Remove the remnants of dibbs basic#214
bamader merged 6 commits into
mainfrom
remove-dibbs-basic

Conversation

@bamader

@bamader bamader commented Feb 11, 2025

Copy link
Copy Markdown
Collaborator

Description

This PR removes the dibbs basic algorithm from the RecordLinker service, updating the documentation and unit tests to reflect this change. There's a lot of alterations, but tests are now stable and everything looks good across the board. Would probably prefer waiting to merge this until both other engineers get eyes on to make sure I didn't miss anything and to catch any outstanding nuances in test cases.

Related Issues

closes #205

Additional Notes

[Add any additional context or notes that reviewers should know about.]

<--------------------- REMOVE THE LINES BELOW BEFORE MERGING --------------------->

Checklist

Please review and complete the following checklist before submitting your pull request:

  • I have ensured that the pull request is of a manageable size, allowing it to be reviewed within a single session.
  • I have reviewed my changes to ensure they are clear, concise, and well-documented.
  • I have updated the documentation, if applicable.
  • I have added or updated test cases to cover my changes, if applicable.
  • I have minimized the number of reviewers to include only those essential for the review.

Checklist for Reviewers

Please review and complete the following checklist during the review process:

  • The code follows best practices and conventions.
  • The changes implement the desired functionality or fix the reported issue.
  • The tests cover the new changes and pass successfully.
  • Any potential edge cases or error scenarios have been considered.

@codecov

codecov Bot commented Feb 11, 2025

Copy link
Copy Markdown

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 97.65%. Comparing base (1af8368) to head (8220172).
Report is 3 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #214      +/-   ##
==========================================
- Coverage   97.66%   97.65%   -0.02%     
==========================================
  Files          32       32              
  Lines        1587     1618      +31     
==========================================
+ Hits         1550     1580      +30     
- Misses         37       38       +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@ericbuckley

Copy link
Copy Markdown
Collaborator

I believe we'll also need to make some changes to the tests/algorithm/configurations/algorithm_configuration.json sample file as some of the functions it references are no longer available. (Really that file should have been updated when we did the function rename, but it looks like we missed it)

Comment thread tests/unit/database/test_algorithm_service.py
Comment thread tests/unit/routes/test_link_router.py
@bamader

bamader commented Feb 11, 2025

Copy link
Copy Markdown
Collaborator Author

@ericbuckley What actually is the purpose of tests/algorithm/configurations/algorithm_configuration.json? I definitely missed it making my pass, but since the tests all passed without updating it, is it actually being invoked anywhere? It seems to me like we can just delete it but want to make sure I'm not missing something.

@ericbuckley

Copy link
Copy Markdown
Collaborator

@ericbuckley What actually is the purpose of tests/algorithm/configurations/algorithm_configuration.json? I definitely missed it making my pass, but since the tests all passed without updating it, is it actually being invoked anywhere? It seems to me like we can just delete it but want to make sure I'm not missing something.

  • tests/unit: these are unit tests (some might technically be integration, but its nuanced), we have a GH action workflow to run these in PRs
  • tests/algorithm: this is a set of scripts we developed to test an algorithm configuration with a known set of test cases. Over the fall, NBS asked us many times how does the code handle edge case X. Our answer was always, it depends on how you have it configured. This project was our way to respond to that question and create developer tooling that allows those types of questions to be answered. These are not automated, developers need to go through the steps in the README to run them.
  • tests/performance: another set of scripts developed to see how fast the API can process linkage requests using synthetic data. This is useful for verifying refactors are still performant and helping developers identify bottlenecks along the way. These are not automated, developers need to go through the steps in the README to run them.

I'm guessing some documentation like the above would be helpful in the top level README 😁

@bamader

bamader commented Feb 12, 2025

Copy link
Copy Markdown
Collaborator Author

Algo config tests have been updated and some new details added to the README to cover this explanation!

Comment thread README.md

@ericbuckley ericbuckley left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good to me.

Note that this is very much a breaking change, and if someone were to upgrade to this version with an existing database, it would likely break because of the algorithm configurations. We'll need to make these notes in our release docs for version v25.2.0.

@bamader

bamader commented Feb 13, 2025

Copy link
Copy Markdown
Collaborator Author

For sure. Would you actually rather we delayed merging this until after the release?

@ericbuckley

Copy link
Copy Markdown
Collaborator

For sure. Would you actually rather we delayed merging this until after the release?

If we're going to do it, I'd rather do it sooner than later. I'd only vote for holding off, if you're not 100% sure this is the right direction to go in.

@bamader

bamader commented Feb 13, 2025

Copy link
Copy Markdown
Collaborator Author

No hesitation on my end. I very strongly believe that pushing just THE DIBBs Algorithm will exponentially reduce confusion and will put our most significant contributions front and center.

@bamader bamader merged commit 83a1d35 into main Feb 13, 2025
@bamader bamader deleted the remove-dibbs-basic branch February 13, 2025 18:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

remove dibbs basic and non-probablistic compare functions

2 participants