Although the expander randomizer script successfully generates new test cases based on permuted values from the existing NBS cases, because match decisions are re-used, the cases aren't necessarily properly graded. Instead of using the random permutations generated, we want to create a larger set of more robust human training examples. The scope of this ticket is the following: for each of the 83 test cases in the original NBS test file, generate 2-4 permutations in which different combinations of fields are dropped (simulating missingness) or altered (simulating typos and entry quality issues), and then assign a match-grade based on a reasoned judgment of how an end-user would classify the result.
Although the expander randomizer script successfully generates new test cases based on permuted values from the existing NBS cases, because match decisions are re-used, the cases aren't necessarily properly graded. Instead of using the random permutations generated, we want to create a larger set of more robust human training examples. The scope of this ticket is the following: for each of the 83 test cases in the original NBS test file, generate 2-4 permutations in which different combinations of fields are dropped (simulating missingness) or altered (simulating typos and entry quality issues), and then assign a match-grade based on a reasoned judgment of how an end-user would classify the result.