Skip to content

Auto initialize algorithms in DB#89

Merged
ericbuckley merged 59 commits into
mainfrom
feature/57-auto-init-db
Oct 21, 2024
Merged

Auto initialize algorithms in DB#89
ericbuckley merged 59 commits into
mainfrom
feature/57-auto-init-db

Conversation

@ericbuckley

@ericbuckley ericbuckley commented Oct 18, 2024

Copy link
Copy Markdown
Collaborator

Description

Configuring the code to optionally load in initial algorithms after the tables have been initialized. Doing so, removes our needs now for having to use a database migration tool for the near future.

Related Issues

closes #57

Additional Notes

  • This is contentious, but have removed alembic. Using alembic has required a lot of manual effort to get migrations correct, with little benefit this early in a project. Our schema hasn't stabilized yet, and we don't have any systems running the code in production, so it's of little value. Over time, we may need to bring this back to safely adjust production schemas, but now is not the time.
  • Created assets/initial_algorithms.json with the two existing DIBBs algorithms. We can use this for our test cases, and also initializing a new database with the data, if the INITIAL_ALGORITHMS env variable is set. This also gives customers another option for managing their algorithms, in they can write their own and set the INITIAL_ALGORITHMS env variable to load that json file.
  • Capitalized schemas.pii.Features. The blocking keys were capitalized, so making them both capitalized will hopefully reduce errors in writing the algorithm specs.
  • Consolidated some of the assets and testing assets, as there were duplicates and there was some testing files in the top level one.

<--------------------- REMOVE THE LINES BELOW BEFORE MERGING --------------------->

Checklist

Please review and complete the following checklist before submitting your pull request:

  • I have ensured that the pull request is of a manageable size, allowing it to be reviewed within a single session.
  • I have reviewed my changes to ensure they are clear, concise, and well-documented.
  • I have updated the documentation, if applicable.
  • I have added or updated test cases to cover my changes, if applicable.
  • I have minimized the number of reviewers to include only those essential for the review.

Checklist for Reviewers

Please review and complete the following checklist during the review process:

  • The code follows best practices and conventions.
  • The changes implement the desired functionality or fix the reported issue.
  • The tests cover the new changes and pass successfully.
  • Any potential edge cases or error scenarios have been considered.

@ericbuckley ericbuckley added the api New API feature label Oct 18, 2024
@ericbuckley ericbuckley self-assigned this Oct 18, 2024
@codecov

codecov Bot commented Oct 18, 2024

Copy link
Copy Markdown

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 93.51%. Comparing base (bc0d93f) to head (c0c55ea).
Report is 1 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main      #89      +/-   ##
==========================================
+ Coverage   93.26%   93.51%   +0.24%     
==========================================
  Files          19       19              
  Lines         936      971      +35     
==========================================
+ Hits          873      908      +35     
  Misses         63       63              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@ericbuckley ericbuckley marked this pull request as ready for review October 18, 2024 18:30

@alhayward alhayward left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work! Left a few quick questions & 1 suggestion. The justification for moving away from Alembic for now makes sense.

Comment thread assets/initial_algorithms.json Outdated
Comment thread assets/initial_algorithms.json
Comment thread src/recordlinker/linking/link.py
Comment thread src/recordlinker/main.py
Comment thread tests/unit/linking/test_link.py
Comment thread assets/initial_algorithms.json
update to the basic algorithm description

Co-authored-by: Alex Hayward <44384574+alhayward@users.noreply.github.com>

@alhayward alhayward left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@ericbuckley ericbuckley merged commit dbe3761 into main Oct 21, 2024
@ericbuckley ericbuckley deleted the feature/57-auto-init-db branch October 21, 2024 20:15
@ericbuckley ericbuckley mentioned this pull request Jun 26, 2025
2 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

api New API feature

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Auto initialize database with algorithms

3 participants