ci(integrated-benchmark): post the comment from a workflow_run job#398
Conversation
The benchmark comment never landed on fork PRs (e.g. #391). The original workflow ran on `pull_request`, which means GitHub gives it a read-only `GITHUB_TOKEN` when the PR comes from a fork — `peter-evans/create-or-update-comment` would 403 trying to write back. The workflow guarded the write with `if: github.event.pull_request.head.repo.full_name == github.repository`, so it silently skipped instead of failing. Switch to the standard two-stage pattern: 1. `integrated-benchmark.yml` runs the benchmark and uploads `SUMMARY.md`, the PR number, and the runner OS as a single artifact. It no longer needs `issues` / `pull-requests` write permissions. 2. `integrated-benchmark-comment.yml` (new) is triggered on `workflow_run` of `Integrated-Benchmark` and runs in the base repository's privilege context, where it can post the comment back to a fork PR. It downloads the artifact via the GitHub API (`actions/download-artifact@v8` with `run-id` + `github-token`), sanitises both untrusted values (the artifact comes from a fork-controlled run), then runs the same find/create-or-update sequence the original workflow used. Same gating as before: only success-path runs of `pull_request` events post a comment. Comments from same-repo PRs continue to work; fork PRs now also get them.
|
ℹ️ Recent review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (2)
🚧 Files skipped from review as they are similar to previous changes (2)
📝 WalkthroughWalkthroughThis PR separates benchmark reporting concerns by delegating PR comment posting to a dedicated workflow. The integrated-benchmark workflow now runs with restricted permissions, uploads benchmark outputs as artifacts, and a new separate workflow consumes those artifacts to post or update PR comments. ChangesBenchmark Workflow Separation and Comment Automation
Sequence Diagram(s)sequenceDiagram
participant GH as GitHub
participant IB as Integrated-Benchmark Workflow
participant WFC as Integrated-Benchmark Comment Workflow
participant Art as Artifact Storage
participant PR as Pull Request
GH->>IB: run Integrated-Benchmark (on PR)
IB->>Art: upload integrated-benchmark-report-ubuntu-latest (SUMMARY.md)
GH->>WFC: workflow_run completed (success)
WFC->>Art: download artifact for run id
WFC->>GH: resolve PR number (payload or search by head_sha)
WFC->>GH: find or create bot comment on PR
WFC->>PR: post/update comment with SUMMARY.md
Estimated code review effort🎯 2 (Simple) | ⏱️ ~10 minutes Possibly related PRs
Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.github/workflows/integrated-benchmark-comment.yml:
- Around line 43-67: The step currently named "Read PR number and runner OS"
reads pr-number.txt from an untrusted artifact into pr_raw and uses it as the
authoritative PR/issue number; change it to derive the PR number from the
trusted workflow trigger metadata instead (use the workflow_run event JSON via
$GITHUB_EVENT_PATH or similar), e.g. extract the PR number with jq from
.workflow_run.pull_requests[0].number and assign that to pr (retain the existing
runner-os sanitisation for os_raw), and if you want keep the artifact value only
as a sanity-check: read pr-number.txt into a tmp var and compare it to the
trusted value and emit an error/warning on mismatch rather than using the
artifact as the source of truth; update any echoes that write to $GITHUB_OUTPUT
to use the trusted pr variable.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: 0fba34dc-cd2e-42c4-9bca-8bf7f0d81ca2
📒 Files selected for processing (2)
.github/workflows/integrated-benchmark-comment.yml.github/workflows/integrated-benchmark.yml
There was a problem hiding this comment.
Pull request overview
This PR fixes integrated-benchmark report comments not being posted on fork PRs by splitting the workflow into a two-stage pipeline: a pull_request job that produces an artifact, and a privileged workflow_run job that posts/updates the PR comment in the base repo context.
Changes:
- Updates
integrated-benchmark.ymlto stop commenting directly and instead uploadSUMMARY.md+ metadata as an artifact. - Adds
integrated-benchmark-comment.ymltriggered byworkflow_runto download the artifact and post/update the benchmark comment with basic input sanitization.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| .github/workflows/integrated-benchmark.yml | Removes direct PR commenting and uploads the benchmark summary + metadata as an artifact. |
| .github/workflows/integrated-benchmark-comment.yml | New workflow_run workflow that downloads the artifact and posts/updates the PR comment from the base repo context. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
… trigger Address CodeRabbit's PR-redirection concern on #398: a fork-controlled artifact must not be the source of truth for which PR receives the comment. Move the PR-number resolution into the comment workflow, sourced from `workflow_run` event metadata that GitHub itself fills in: - Same-repo PRs: read directly from `github.event.workflow_run.pull_requests[0].number`. - Fork PRs (where that array is empty): query `gh api repos/$REPO/commits/$HEAD_SHA/pulls` against the BASE repo, using `workflow_run.head_sha` from the same trusted payload. CodeRabbit's suggested fix used `pull_requests[0].number` unconditionally, which would have broken the fork-PR case this workflow exists to fix — GitHub's docs explicitly state that field is empty for fork-triggered runs. Drop `pr-number.txt` from the uploaded artifact entirely. The runner OS is also no longer carried; the matrix only runs on Linux today, and the comment-matching string is hard-coded to `Integrated-Benchmark Report (Linux)`. If the matrix expands later, both halves need updating together.
Summary
The integrated-benchmark report comment never landed on fork PRs (e.g. #391, where the benchmark ran successfully five times but no comment was posted). This PR switches to the standard two-stage
pull_request+workflow_runpattern so the comment can post regardless of whether the PR is from a fork.Why the old pattern silently dropped fork-PR comments
integrated-benchmark.ymlran onpull_request. When that event fires from a fork, GitHub gives the workflow a read-onlyGITHUB_TOKEN—peter-evans/create-or-update-commentwould 403 trying to write back. The workflow guarded the write with:so it skipped silently for fork PRs instead of failing. Same-repo PRs continued to get comments.
See https://securitylab.github.com/research/github-actions-preventing-pwn-requests/ for the security context.
What changes
Stage 1 —
integrated-benchmark.yml(modified)issues: write/pull-requests: writepermissions; the workflow no longer writes back.Find Comment+Create or update commentsteps with an artifact upload (actions/upload-artifact@v7) carryingSUMMARY.md, the PR number, andrunner.os.integrated-benchmark-report-${{ matrix.os }}so adding more OSes to the matrix later doesn't collide.Stage 2 —
integrated-benchmark-comment.yml(new)workflow_runofIntegrated-Benchmark, typecompleted. Runs in the base repository's privilege context, where it can post comments to fork PRs.contents: read,actions: read,issues: write,pull-requests: write.pull_requestand conclusion wassuccess— preserves the same gating the original workflow had implicitly via step ordering.actions/download-artifact@v8withrun-id+github-token(the documented cross-workflow recipe).pr-number.txtandrunner-os.txtbefore substituting them into action inputs, since the artifact comes from a fork-controlled run and is therefore untrusted input.peter-evans/find-comment@v4+peter-evans/create-or-update-comment@v5sequence the original workflow used.Test plan
actionlintclean on both workflow files (only the pre-existingactions/rustupmetadata warning remains, unrelated to this change).Saturate/pacquet(or any fork) will exercise the path that motivated this change.Security notes
The
workflow_runworkflow runs against the base ref's checkout-equivalent context, so the permission escalation is intentional and bounded. Two safeguards keep it tight:actions/checkoutstep. It can't be tricked into running fork-supplied scripts.The downloaded
SUMMARY.mdis treated as opaque markdown bypeter-evans/create-or-update-comment, which posts via the issues-comments API. Markdown is escaped in the API call; there's no shell evaluation path through which it could elevate.Written by an agent (Claude Code, claude-opus-4-7).
Summary by CodeRabbit