Add caching of RecordLinker image to smoke tests#261
Conversation
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #261 +/- ##
=======================================
Coverage 97.82% 97.82%
=======================================
Files 33 33
Lines 1748 1748
=======================================
Hits 1710 1710
Misses 38 38 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
bamader
left a comment
There was a problem hiding this comment.
This looks super painful despite being so short 😞 . Thanks for working through these difficulties. Looking at the draft of the other work, splitting definitely seems like the right call to me.
| with: | ||
| push: true | ||
| tags: | | ||
| ghcr.io/${{ env.REPO }}/rl-service:latest |
There was a problem hiding this comment.
I think there is a potential for a conflict here if we have more than one open PR. If both PR's are building, the last one win's out.
There was a problem hiding this comment.
@ericbuckley I was thinking of changing the image name to be based on the pr like rl-service:pr-$PR_NUMBER, but want to make sure the image gets removed after the PR is merged. I could do it through a workflow or let a retention policy take care of it, but I don't know what our retention policy is and but don't seem to have the right permissions to do so. Do we have a retention policy set or should I remove the image with a workflow?
There was a problem hiding this comment.
Will make any updates in the other PR to address this
There was a problem hiding this comment.
To my knowledge we don't have a retention policy set. I'm not even sure how to do that on a ghcr.io repo, but I'm sure there is a way. One thing that might be worth considering is using docker layer caching for this rather than an ghcr.io repo. Take a look at this article https://medium.com/@depot.dev/how-to-use-docker-layer-caching-in-github-actions-bee1f7b19093
There was a problem hiding this comment.
Author of ☝️ (co-founder of Depot, former PDX native), you can find a less annoying & updated version (Medium can be meh) of that post here: https://depot.dev/blog/docker-layer-caching-in-github-actions
Also, if you want to skip pushing to GHCR to use as the cache, you can specify load: true, and that should just get the image back to you in its tagged form to then do a docker run on it. You could then do the docker push after your tests to put it in GHCR.
If you also want an open source sponsorship of Depot to swap docker/build-push-action for depot/build-push-action, also happy to set that up.
There was a problem hiding this comment.
Thanks @kylegalbraith I appreciate the guidance.
Description
This PR addresses half of the caching issue outlined in ticket 224: ensuring that we use the cached the RecordLinker image when we run smoke tests to avoid hitting the limit on requests to DockerHub.
After much tinkering, I added a
build-push-runstep to our smoke tests so now when you run them you should see that we use the image from ghcr for each smoke test instead of hittingdocker.io/library/python:3.12-slimto build the service in each run.Related Issues
Fixes #224
Additional Notes
Getting the services to run from images in the GHCR has proven to be much more difficult, which is why I opted to split up the work on #244. At least we will have cut down the number of requests for RecordLinker by the 4/1 deadline.
<--------------------- REMOVE THE LINES BELOW BEFORE MERGING --------------------->
Checklist
Please review and complete the following checklist before submitting your pull request:
Checklist for Reviewers
Please review and complete the following checklist during the review process: