Add Trusted Publishers documentation for keyless CI/CD#2506
Conversation
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
|
|
||
| ## Supported CI providers | ||
|
|
||
| The settings UI ships with presets for the providers below, but any OIDC-compliant provider works (AWS, GCP, Buildkite, your own IdP, …). |
There was a problem hiding this comment.
could HF Jobs be one, potentially?
There was a problem hiding this comment.
yes, if we're interested
There was a problem hiding this comment.
but jobs are already called with tokens already - so less interesting. Unless we have better ways to automate jobs
(still interesting if we want to allow an external job owned by someone else to publish to a repo)
|
|
||
| ### For CI/CD pipelines | ||
|
|
||
| If you only need access from a CI/CD workflow (GitHub Actions, GitLab CI, CircleCI, …), you can avoid storing an access token as a CI secret entirely. See [Trusted Publishers](./trusted-publishers), which exchanges your CI provider's OpenID Connect (OIDC) identity token for a short-lived Hub token at the start of each run — either repo-scoped (to publish models, datasets, Spaces or kernels) or user-scoped (to read your gated repos and get your account's rate limits). |
There was a problem hiding this comment.
maybe @gary149 let's do a super simple visual or thumbnail "Trusted Publishers" to insert here and more generally to do a little bit of comms about this in a security light
Co-authored-by: Julien Chaumond <julien@huggingface.co>
| HEADERS=$(mktemp); BODY=$(mktemp) | ||
| STATUS=$(curl -sS -D "$HEADERS" -o "$BODY" -w '%{http_code}' \ | ||
| -X POST "https://huggingface.co/oauth/token" \ | ||
| -H "Content-Type: application/json" \ | ||
| -d "$(jq -n --arg t "$ID_TOKEN" --arg r "acme/awesome-model" \ | ||
| '{ | ||
| grant_type: "urn:ietf:params:oauth:grant-type:token-exchange", | ||
| subject_token_type: "urn:ietf:params:oauth:token-type:id_token", | ||
| subject_token: $t, | ||
| resource: $r | ||
| }')") | ||
|
|
||
| if [ "$STATUS" -ne 200 ]; then | ||
| REQUEST_ID=$(grep -i '^x-request-id:' "$HEADERS" | sed 's/.*: *//' | tr -d '\r') | ||
| echo "::error::Token exchange failed with HTTP $STATUS (x-request-id: ${REQUEST_ID:-unknown})" | ||
| cat "$BODY" | ||
| exit 1 | ||
| fi | ||
|
|
||
| HF_TOKEN=$(jq -r .access_token "$BODY") |
There was a problem hiding this comment.
cc @Wauplin @hanouticelina maybe this part could be "automated" in hf cli eg if ID_TOKEN is set & its audience points to huggingface.co (and HF_TOKEN not set?)
anyway not urgent need, just thinking
5019d0d to
0dcfce1
Compare
|
cc @gary149 @julien-c ready to be merged companion PR: https://github.com/huggingface-internal/moon-landing/pull/18488 |
| | Flavor | Configured on | What you get | Use it to… | | ||
| | --- | --- | --- | --- | | ||
| | **Repo publisher** | A repo's **Settings → Trusted Publishers** | A token with **write access to that one repo** | Publish a model, dataset, Space, or kernel from CI | | ||
| | **User publisher** | Your account's [**Authentication settings → CI/CD Access**](https://huggingface.co/settings/authentication#ci-cd-access) | A read-only token with the `gated-repos` scope | Read **your** gated repos and use your rate limits from CI | |
There was a problem hiding this comment.
| | **User publisher** | Your account's [**Authentication settings → CI/CD Access**](https://huggingface.co/settings/authentication#ci-cd-access) | A read-only token with the `gated-repos` scope | Read **your** gated repos and use your rate limits from CI | | |
| | **User publisher** | Your account's [**Authentication settings → CI/CD Access**](https://huggingface.co/settings/authentication#ci-cd-access) | A read-only token with the `gated-repos` scope | Read [gated repos](https://huggingface.co/docs/hub/en/datasets-gated#access-gated-datasets-as-a-user) you have access to, and use your rate limits from CI | |
nit - since it's not "your" as in "gated repos you published", correct?
Internal PR: https://github.com/huggingface-internal/moon-landing/pull/17449
https://moon-ci-docs.huggingface.co/docs/hub/pr_2506/en/trusted-publishers
maybe
hfCLI should have a mechanism to automatically pick up the token (like npm publish)Note
Low Risk
Documentation-only changes with no application or auth implementation modified in this PR.
Overview
Adds Trusted Publishers documentation so CI/CD can publish to the Hub without storing an
HF_TOKENsecret, by exchanging a CI provider OIDC ID token for a short-lived Hub token viaPOST /oauth/token(RFC 8693).The new
trusted-publisherspage covers repo vs user publishers, claim matching, supported providers (GitHub Actions, GitLab, CircleCI, Bitbucket), a full GitHub Actions workflow example, API reference, and security notes. The Hub docs toctree lists it under Security, and oauth, security-tokens, and repositories-github-actions now point readers to Trusted Publishers where relevant (including contrasting Enterprise org token exchange with keyless CI).Reviewed by Cursor Bugbot for commit 093d9d4. Bugbot is set up for automated code reviews on this repo. Configure here.