Description

This PR adds a script that allow us to combinatorially expand and scramble the test cases NBS provided for us. We leave each of the original test cases unchanged, but we generate a random number of duplicates of each case to mess with. We then dropout some fields to simulate missingness, as well as apply random likelihoods of edit distances to string values in algorithm-relevant fields. All of this is controlled by parameters at the top of the script (which only needs to be run once to make the data-set, doesn't need to be run each time we run the algorithm tests). Of note, the script right now copies the match decisions of the original test case into each duplicate we generate from it, no matter how much we then scramble and mangle the duplicate case. This is the reason we have poor performance on this set, so that's expected. Before sharing this out, we should consider "re-grading" our expanded set to have better labels of when things should and shouldn't match. Also, since the script is customizable, we can also generate different test expansion cases if we want to simulate "pretty close" data (low randomness scrambling) vs "really bad data" (lots of scrambling). We'd need to grade each separately, but this could be useful for purposes of showing performance in different contexts.

Related Issues

Closes #354

<--------------------- REMOVE THE LINES BELOW BEFORE MERGING --------------------->

Checklist

Please review and complete the following checklist before submitting your pull request:

• I have ensured that the pull request is of a manageable size, allowing it to be reviewed within a single session.
• I have reviewed my changes to ensure they are clear, concise, and well-documented.
• I have updated the documentation, if applicable.
• I have added or updated test cases to cover my changes, if applicable.
• I have minimized the number of reviewers to include only those essential for the review.

Checklist for Reviewers

Please review and complete the following checklist during the review process:

• The code follows best practices and conventions.
• The changes implement the desired functionality or fix the reported issue.
• The tests cover the new changes and pass successfully.
• Any potential edge cases or error scenarios have been considered.