Test uv+PyPI Trusted Publishing on Gitlab#17443
Conversation
92d4fea to
c358d27
Compare
c358d27 to
6df5cd2
Compare
6df5cd2 to
1721b3d
Compare
1721b3d to
dfa9f86
Compare
dfa9f86 to
5e7d97c
Compare
5e7d97c to
63086ad
Compare
63086ad to
61532d7
Compare
Signed-off-by: William Woodruff <william@astral.sh> disable attestations for GitLab TP test Signed-off-by: William Woodruff <william@astral.sh>
61532d7 to
9b9bcdf
Compare
04bf039 to
e5315b7
Compare
Signed-off-by: William Woodruff <william@astral.sh>
e5315b7 to
9feb224
Compare
There was a problem hiding this comment.
Apologies for the churn in this file -- I realized that I was having trouble tracking the state through the different sub-tests, so what I've done is add a "plan" abstraction that pre-plans some test state. We can then pass that state around consistently via a Plan object rather than sharing it piecemeal.
| "pypi-trusted-publishing-gitlab": TargetConfiguration( | ||
| "astral-test-pypi-trusted-publishing-gitlab", | ||
| publish_url=TEST_PYPI_PUBLISH_URL, | ||
| index_url="https://test.pypi.org/simple/", | ||
| index=None, | ||
| # We're impersonating GitLab, so we can't easily test attestations here. | ||
| # TODO: In principle we could test this by having GitLab issue us an `aud:sigstore` | ||
| # OIDC token in addition to the `aud:testpypi` one. | ||
| attestations=False, | ||
| ), |
There was a problem hiding this comment.
This is the main operative change 🙂
| if plan.target in ("pypi-trusted-publishing-gitlab",): | ||
| return |
There was a problem hiding this comment.
Flagging: the nuance here is that GitLab, unlike GitHub and other TP providers, doesn't have an "active" OIDC source: the runner gets a single OIDC credential at startup. As a result of that PyPI (and pyx) consider the OIDC credential "spent" after its first use, meaning that our duplicate/conflict/etc. tests all fail for an unrelated reason (the index is rejecting our credential as reused).
There's no great way around this, it's seemingly an architectural limitation of GitLab. But the good news is that it won't affect typical user workflows, it only dings us here because we're intentionally re-using an OIDC credential across separate uv publish invocations 🙂
There was a problem hiding this comment.
(This is true for the other tests below as well.)
There was a problem hiding this comment.
Would this mean that two subsequent uv publish commands in the same job in a user's workflow fail the same way? If so, that's quite a severe limitation
There was a problem hiding this comment.
Yep, exactly. There's ways we could potentially work around that within uv itself (e.g. we could stash the minted credential for reuse between invocations), but the limitation on OIDC token reuse is architectural/inside PyPI.
In practice this doesn't generally bite users because all they do is a single uv publish invocation, plus must CI providers don't have this limitation (GitHub and all the others can acquire additional OIDC creds at runtime.)
| packages-dir: "astral-test-pypa-gh-action/dist" | ||
|
|
||
| - name: "Request GitLab OIDC token for impersonation" | ||
| uses: digital-blueprint/gitlab-pipeline-trigger-action@20e77989b24af658ba138a0aa5291bdc657f1505 # v1.3.0 |
There was a problem hiding this comment.
Flagging: this adds ~30s of sync runtime to the publishing tests, since this action needs to block while the Gitlab pipeline completes. That's not ideal, but the silver lining is that it'll be amortized as we add more Gitlab publishing tests (e.g. for pyx too), since we can use the same pipeline invocation to generate multiple ID tokens.
There was a problem hiding this comment.
(Also noting that I reviewed this action's source. I considered trying to open-code this with GitLab's glab CLI instead, but this seemed simpler. We could always switch to that instead, though.)
zsol
left a comment
There was a problem hiding this comment.
This looks ok to me, but maybe it would be slightly better if we would have a separate, parallel job for the non-github trusted publishing tests, so if they do fail at the point of getting the oidc token the rest of the tests would still run
Yeah, I was thinking about either sharding this across jobs with a matrix or changing it to use a more "traditional" pytest setup, so that we could run the entire suite without failing fast. I can look at that with a follow up. |
Summary
WIP.The basic idea here is to invoke a Gitlab Pipeline via GitHub. That pipeline generates an OIDC token which it saves as an artifact, which the GitHub workflow can then read. We then use that OIDC token to impersonate the identity of the Gitlab Pipeline for Trusted Publishing purposes.
Guinea pig project: https://test.pypi.org/project/astral-test-pypi-trusted-publishing-gitlab/
See #17438 for motivating context.
Test Plan
Implement the above within
test_publish.pyandci.yml.