test: add token pool integration for rate limit distribution#7397
Conversation
- Create .github/actions/fetch-token composite action to fetch tokens from mise-versions - Integrate token pool into coverage jobs in test.yml (8 parallel tranches) - Integrate token pool into test-tool jobs in registry.yml (8 parallel tranches) - Each parallel job gets a different token via round-robin from the pool - Falls back to existing GITHUB_TOKEN if pool is unavailable - Tokens are masked in logs with ::add-mask:: Requires MISE_VERSIONS_API_SECRET secret to be configured. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
This PR integrates a token pool system to distribute GitHub API rate limits across parallel CI jobs by fetching different tokens from the mise-versions API. Jobs that hit rate limits will now use pooled tokens instead of the default GITHUB_TOKEN, improving reliability for parallel test execution.
- Creates a reusable composite action to fetch tokens from the mise-versions token pool
- Integrates token fetching into 8 parallel coverage jobs in test.yml
- Integrates token fetching into 8 parallel test-tool jobs in registry.yml
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 5 comments.
| File | Description |
|---|---|
| .github/actions/fetch-token/action.yml | New composite action that fetches tokens from mise-versions API with fallback handling |
| .github/workflows/test.yml | Adds token pool integration to coverage jobs before installing dependencies |
| .github/workflows/registry.yml | Adds token pool integration to test-tool jobs before downloading artifacts |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| - id: fetch | ||
| shell: bash | ||
| run: | | ||
| if [ -z "${{ inputs.api-secret }}" ]; then |
There was a problem hiding this comment.
The api-secret is being checked for emptiness but is also directly interpolated into the shell script. If the secret contains special characters or shell metacharacters, it could lead to command injection. While GitHub Actions masks secrets in logs, the interpolation pattern could still be exploited. Consider using environment variables instead: set API_SECRET: ${{ inputs.api-secret }} in an env: block at the step level, then reference $API_SECRET in the script.
| echo "No API secret provided, skipping token fetch" | ||
| exit 0 | ||
| fi | ||
| response=$(curl -sf -H "Authorization: Bearer ${{ inputs.api-secret }}" \ |
There was a problem hiding this comment.
The api-secret is directly interpolated into the curl command. This creates a security risk if the secret contains special characters. Use an environment variable instead to avoid potential command injection and ensure proper shell escaping.
| token=$(echo "$response" | jq -r '.token') | ||
| echo "::add-mask::$token" |
There was a problem hiding this comment.
There's a timing window between extracting the token and masking it where the token could appear in logs if an error occurs. Consider masking the entire response first, or ensure error handling doesn't echo the response before the mask is applied.
| api-secret: ${{ secrets.MISE_VERSIONS_API_SECRET }} | ||
| - name: Set GITHUB_TOKEN from pool | ||
| if: steps.token.outputs.token | ||
| run: echo "GITHUB_TOKEN=${{ steps.token.outputs.token }}" >> "$GITHUB_ENV" |
There was a problem hiding this comment.
Writing the token directly to GITHUB_ENV could expose it in the environment. While GitHub Actions should handle this, consider whether this approach maintains the same security guarantees as using secrets context. The token should ideally be masked before being written to the environment file.
| api-secret: ${{ secrets.MISE_VERSIONS_API_SECRET }} | ||
| - name: Set GITHUB_TOKEN from pool | ||
| if: steps.token.outputs.token | ||
| run: echo "GITHUB_TOKEN=${{ steps.token.outputs.token }}" >> "$GITHUB_ENV" |
There was a problem hiding this comment.
Writing the token directly to GITHUB_ENV could expose it in the environment. While GitHub Actions should handle this, consider whether this approach maintains the same security guarantees as using secrets context. The token should ideally be masked before being written to the environment file.
If the API returns valid JSON but the token field is missing or null, jq outputs the literal string "null". Add validation to ensure the token matches GitHub token format (gh*_...) before outputting it. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
| if [ -n "$response" ]; then | ||
| token=$(echo "$response" | jq -r '.token') | ||
| # Validate token looks like a GitHub token (starts with gh and has reasonable length) | ||
| if [[ "$token" =~ ^gh[a-z]_[A-Za-z0-9_]+$ ]] && [ ${#token} -ge 20 ]; then |
There was a problem hiding this comment.
Bug: Token regex rejects fine-grained personal access tokens
The token validation regex ^gh[a-z]_[A-Za-z0-9_]+$ only matches classic GitHub tokens (like ghp_, ghs_, gho_) but rejects fine-grained personal access tokens which use the github_pat_ prefix format. If the token pool contains fine-grained PATs, they would be silently rejected and the workflow would fall back to the default GITHUB_TOKEN, defeating the rate limit distribution purpose.
Hyperfine Performance
|
| Command | Mean [ms] | Min [ms] | Max [ms] | Relative |
|---|---|---|---|---|
mise-2025.12.12 x -- echo |
20.0 ± 0.3 | 19.3 | 23.0 | 1.00 |
mise x -- echo |
20.5 ± 0.4 | 19.6 | 22.4 | 1.02 ± 0.02 |
mise env
| Command | Mean [ms] | Min [ms] | Max [ms] | Relative |
|---|---|---|---|---|
mise-2025.12.12 env |
19.6 ± 0.5 | 18.8 | 25.3 | 1.00 |
mise env |
20.3 ± 1.0 | 19.1 | 37.4 | 1.03 ± 0.06 |
mise hook-env
| Command | Mean [ms] | Min [ms] | Max [ms] | Relative |
|---|---|---|---|---|
mise-2025.12.12 hook-env |
19.8 ± 0.8 | 18.9 | 32.5 | 1.00 |
mise hook-env |
20.2 ± 0.5 | 19.2 | 23.3 | 1.02 ± 0.05 |
mise ls
| Command | Mean [ms] | Min [ms] | Max [ms] | Relative |
|---|---|---|---|---|
mise-2025.12.12 ls |
17.6 ± 0.5 | 16.6 | 18.8 | 1.00 |
mise ls |
17.8 ± 0.7 | 16.8 | 27.9 | 1.01 ± 0.05 |
xtasks/test/perf
| Command | mise-2025.12.12 | mise | Variance |
|---|---|---|---|
| install (cached) | 109ms | 111ms | -1% |
| ls (cached) | 65ms | 67ms | -2% |
| bin-paths (cached) | 72ms | 73ms | -1% |
| task-ls (cached) | 283ms | 282ms | +0% |
* upstream/renovate/lockfile-maintenance: chore(deps): lock file maintenance fix(ci): improve GHA cache efficiency and fix registry-ci bug (jdx#7404) feat(tera): add haiku() function for random name generation (jdx#7399) feat: implement independent versioning for subcrates (jdx#7402) docs: add comprehensive glossary (jdx#7401) docs: improve installation documentation (jdx#7403) test: add token pool integration for rate limit distribution (jdx#7397) docs: add link to COPR package page for Fedora/RHEL test: rename duplicate 'ci' job names for clarity (jdx#7398) registry: add github backend for swiftformat (jdx#7396) chore: rename mise-tools to mise-versions chore: release 2025.12.12 (jdx#7386) fix(github): use version_prefix when fetching release for SLSA verification (jdx#7391) refactor(vfox): remove submodules, embed plugins directly (jdx#7389) test(registry): add final ci job as merge gate (jdx#7390) test: split unit job to speed up macOS CI (jdx#7388) feat(backend): add security features to github backend (jdx#7387)
### 🚀 Features - **(tera)** add haiku() function for random name generation by @jdx in [#7399](#7399) - implement independent versioning for subcrates by @jdx in [#7402](#7402) ### 🐛 Bug Fixes - **(ci)** improve GHA cache efficiency and fix registry-ci bug by @jdx in [#7404](#7404) - **(ci)** use !cancelled() instead of always() for registry-ci by @jdx in [#7435](#7435) - **(test)** update backend_arg test to use clojure instead of poetry by @jdx in [#7436](#7436) ### 📚 Documentation - add link to COPR package page for Fedora/RHEL by @jdx in [bc8ac73](bc8ac73) - improve installation documentation by @jdx in [#7403](#7403) - add comprehensive glossary by @jdx in [#7401](#7401) ### 🧪 Testing - rename duplicate 'ci' job names for clarity by @jdx in [#7398](#7398) - add token pool integration for rate limit distribution by @jdx in [#7397](#7397) ### 📦 Registry - add github backend for swiftformat by @jdx in [#7396](#7396) - use pipx backend for azure-cli by @jdx in [#7406](#7406) - use pipx backend for dvc by @jdx in [#7413](#7413) - add github backend for zprint by @jdx in [#7410](#7410) - use gem backend for cocoapods by @jdx in [#7411](#7411) - use pipx backend for gallery-dl by @jdx in [#7409](#7409) - add aqua backends for HashiCorp tools by @jdx in [#7408](#7408) - use npm backend for danger-js by @jdx in [#7407](#7407) - use pipx backend for pipenv by @jdx in [#7415](#7415) - use pipx backend for poetry by @jdx in [#7416](#7416) - add github backend for xcodegen ([github:yonaskolb/XcodeGen](https://github.com/yonaskolb/XcodeGen)) by @jdx in [#7417](#7417) - use npm backend for heroku by @jdx in [#7418](#7418) - add aqua backend for setup-envtest by @jdx in [#7421](#7421) - add github backend for xcresultparser ([github:a7ex/xcresultparser](https://github.com/a7ex/xcresultparser)) by @jdx in [#7422](#7422) - add aqua backend for tomcat by @jdx in [#7423](#7423) - use npm backend for serverless by @jdx in [#7424](#7424) - add github backend for daytona ([github:daytonaio/daytona](https://github.com/daytonaio/daytona)) by @jdx in [#7412](#7412) - add github backend for flyway ([github:flyway/flyway](https://github.com/flyway/flyway)) by @jdx in [#7414](#7414) - add github backend for schemacrawler ([github:schemacrawler/SchemaCrawler](https://github.com/schemacrawler/SchemaCrawler)) by @jdx in [#7419](#7419) - add github backend for codeql by @jdx in [#7420](#7420) - use pipx backend for mitmproxy by @jdx in [#7425](#7425) - use pipx backend for sshuttle by @jdx in [#7426](#7426) - add github backend for quarkus by @jdx in [#7428](#7428) - add github backend for smithy by @jdx in [#7430](#7430) - add github backend for xchtmlreport ([github:XCTestHTMLReport/XCTestHTMLReport](https://github.com/XCTestHTMLReport/XCTestHTMLReport)) by @jdx in [#7431](#7431) - add github backend for grails by @jdx in [#7429](#7429) - use npm backend for esy by @jdx in [#7434](#7434) - add github backend for micronaut by @jdx in [#7433](#7433) - add github backend for dome by @jdx in [#7432](#7432) - use vfox backend for poetry by @jdx in [#7438](#7438) ### Chore - **(docker)** add Node LTS to mise Docker image by @jdx in [#7405](#7405) - rename mise-tools to mise-versions by @jdx in [ab3e1b8](ab3e1b8) - s/mise task/mise tasks/g in docs and tests by @muzimuzhi in [#7400](#7400) ### New Contributors - @muzimuzhi made their first contribution in [#7400](#7400)
Summary
.github/actions/fetch-tokencomposite action to fetch tokens from mise-versions token poolcoveragejobs in test.yml (8 parallel tranches)test-tooljobs in registry.yml (8 parallel tranches)GITHUB_TOKENif pool is unavailable (fork PRs, API downtime)::add-mask::How it works
GET /api/tokenfrom mise-versions independentlyGITHUB_TOKENTest plan
MISE_VERSIONS_API_SECRETsecret in repo settings🤖 Generated with Claude Code
Note
Adds a composite action to fetch pooled GitHub tokens and wires it into coverage and registry matrix jobs to distribute API rate limits with masked, conditional fallback.
./.github/actions/fetch-token:mise-versionsAPI, validates format, masks, and outputstoken/token_id.test.yml→coverage(8-tranche matrix): fetch token and setGITHUB_TOKENwhen available.registry.yml→test-tool(matrix): fetch token and setGITHUB_TOKENwhen available.Written by Cursor Bugbot for commit a608195. This will update automatically on new commits. Configure here.