Skip to content

fix(ci): trust generated Agentic CI PRs#643

Merged
andreatgretel merged 8 commits into
mainfrom
andreatgretel/fix/ci-agentic-pr-bypass
May 20, 2026
Merged

fix(ci): trust generated Agentic CI PRs#643
andreatgretel merged 8 commits into
mainfrom
andreatgretel/fix/ci-agentic-pr-bypass

Conversation

@andreatgretel

@andreatgretel andreatgretel commented May 13, 2026

Copy link
Copy Markdown
Contributor

Summary

Generated Agentic CI PRs currently use GITHUB_TOKEN, so GitHub suppresses follow-on pull_request workflow runs for those bot-created updates. This PR keeps that token model and adds a maintainer-authorized /authorize-agentic-ci path to launch the required checks safely.

Findings

  • PRs created by github-actions[bot] with GITHUB_TOKEN do not automatically trigger normal PR checks.
  • The generated PRs also failed DCO and linked issue checks because the commit/PR identity is automation-owned rather than a normal contributor identity.
  • Commit authors are spoofable, so trust should not be based on agentic-ci[bot] commit metadata.
  • A PAT, GitHub App token, or machine user could make checks trigger automatically, but each option adds credential or org-management risk.
  • A slash-command authorization flow avoids adding a long-lived token while still giving maintainers a low-friction way to run checks.

Changes

Added

  • authorize-agentic-ci.yml adds /authorize-agentic-ci, limited to commenters with write, maintain, or admin permission.
  • agentic-ci-authorized-checks.yml emits the required authorization-side checks for DCO, semantic title, and linked issue policy on the authorized PR head SHA.
  • ci.yml accepts an optional expected_head_sha workflow dispatch input and fails dispatched CI if GitHub runs a different SHA.

Changed

  • dco-assistant.yml skips DCO Assistant only for trusted generated Agentic CI PRs.
  • pr-linked-issue.yml treats trusted generated Agentic CI PRs like collaborator-authored PRs for the linked issue check.

Fixed

  • Generated PRs from github-actions[bot] can now get required checks without switching Agentic CI to a PAT, GitHub App token, or machine-user token.
  • The trust predicate now uses GitHub-controlled PR metadata only: PR author github-actions[bot], same-repo head branch, agentic-ci/* branch name, and the generated Agentic CI body marker.
  • The agentic-ci user account is not trusted, avoiding reliance on an identity that could be confused with spoofable commit author metadata.
  • Authorization is refused if the PR changes .github/, if the PR head changes after the maintainer's /authorize-agentic-ci comment, or if the dispatched workflow runs a different SHA.

Attention Areas

Reviewers: Please pay special attention to the following:

  • authorize-agentic-ci.yml - Security boundary for who can authorize generated PR checks and what metadata makes a PR trusted.
  • agentic-ci-authorized-checks.yml - Required-check replacement contexts for generated PRs that cannot trigger normal checks automatically.
  • ci.yml - SHA pinning guard for manually dispatched CI.

Validation

  • YAML parsed with PyYAML
  • git diff --check
  • .venv/bin/ruff check --fix .
  • .venv/bin/ruff format .
  • Dry-run Agentic CI trust, semantic-title, and PR timeline parsing against existing PRs
  • Claude Code review completed; actionable workflow hardening feedback addressed
  • GitHub checks passing on this PR

Description updated with AI

Signed-off-by: Andre Manoel <amanoel@nvidia.com>
Signed-off-by: Andre Manoel <amanoel@nvidia.com>
Signed-off-by: Andre Manoel <amanoel@nvidia.com>
@andreatgretel andreatgretel marked this pull request as ready for review May 14, 2026 17:56
@andreatgretel andreatgretel requested a review from a team as a code owner May 14, 2026 17:56
@github-actions

Copy link
Copy Markdown
Contributor

Review: PR #643 — fix(ci): trust generated Agentic CI PRs

Summary

This PR adds a maintainer-gated /authorize-agentic-ci slash-command flow
so PRs opened by the Agentic CI runner (which uses GITHUB_TOKEN and
therefore cannot trigger child pull_request workflows) can still satisfy
required branch-protection checks without granting the runner a PAT,
GitHub App token, or machine-user identity.

The mechanism is:

  1. A maintainer with write|maintain|admin comments /authorize-agentic-ci.
  2. authorize-agentic-ci.yml validates the commenter's permission, the PR's
    trust metadata (author = github-actions[bot], same-repo head, branch
    agentic-ci/*, body contains the agentic-ci marker comment), that the
    authorization comment is in the PR timeline, that the head SHA has not
    moved since the comment, and that the PR does not touch .github/.
  3. The workflow then dispatches ci.yml and agentic-ci-authorized-checks.yml
    pinned to the authorized head SHA via a new expected_head_sha input.
  4. agentic-ci-authorized-checks.yml emits jobs whose names match the
    required-check contexts (DCOAssistant, semantic-pull-request, check)
    so branch protection sees them as satisfied for trusted PRs only.
  5. dco-assistant.yml and pr-linked-issue.yml short-circuit for trusted
    Agentic CI PRs so the regular pull_request_target / push paths don't fail
    them.

Findings

Security design — good

  • The trust predicate relies only on GitHub-controlled PR metadata
    (author login, head repo full name, head ref, body). Commit author is
    explicitly excluded — correctly noted in the inline comments as spoofable.
  • github-actions[bot] as PR author is only producible by GITHUB_TOKEN,
    which is only used by your own workflows; combined with the same-repo
    head-branch requirement, fork PRs cannot impersonate trust.
  • The PR-head-moved-after-comment check (timeline scan for committed,
    head_ref_force_pushed, head_ref_deleted, head_ref_restored)
    closes the obvious "authorize, then push malicious commit" race.
  • Even if that timeline check loses a race, validate-dispatch in
    ci.yml and the redundant GITHUB_SHA == HEAD_SHA == EXPECTED_HEAD_SHA
    checks in agentic-ci-authorized-checks.yml provide a second SHA-pin
    layer: if gh workflow run --ref HEAD_REF resolves to a different SHA
    than the one the maintainer authorized, every dispatched job hard-fails.
  • Blocking PRs that touch .github/ prevents an authorized run from
    modifying the trust mechanism itself. Good defense-in-depth.
  • Variables are passed through env: rather than interpolated into shell
    via ${{ }}, which avoids script injection from user-controlled fields
    (PR title, body). Title is base64-encoded across the job boundary —
    appropriate for a value that could contain shell metacharacters.

Required-check name spoofing is intentional but worth flagging

agentic-ci-authorized-checks.yml defines jobs named exactly
DCOAssistant, semantic-pull-request / semantic-pull-request, and
check — the same context names branch protection expects from
dco-assistant.yml, amannn/action-semantic-pull-request, and
pr-linked-issue.yml. This is the load-bearing trick that lets
workflow_dispatch runs satisfy pull_request-scoped required checks.

The mechanism is sound given the trust predicate is sound, but it
means that any future workflow that emits these names from any
trigger can satisfy required checks. Suggest adding a short comment at
the top of agentic-ci-authorized-checks.yml explaining that the job
names are deliberately load-bearing for branch protection so a future
maintainer doesn't innocently rename them.

Trust-predicate duplication

The 6-line trust predicate
(author == github-actions[bot] + head_repo == repo + head_ref == agentic-ci/*

  • body marker grep) is duplicated across four workflows:
  • agentic-ci-authorized-checks.yml
  • authorize-agentic-ci.yml
  • dco-assistant.yml
  • pr-linked-issue.yml

If the marker format ever changes, or the branch convention shifts,
all four must update in lockstep or the system silently desyncs (e.g.,
DCO blocks while authorize allows). Strongly recommend extracting this
into a composite action under .github/actions/agentic-ci-trust/ that
emits a single trusted output. The diff shrinks meaningfully and the
predicate becomes audit-reviewable in one place.

Semantic-title regex drift risk

agentic-ci-authorized-checks.yml re-implements the semantic-title
check inline:

TYPES='feat|fix|docs|style|refactor|perf|test|build|ci|chore|revert|cp'

The real amannn/action-semantic-pull-request job is configured
elsewhere; if its allowed types change (e.g., cp is added/removed,
new types added, or the 80-char cap is loosened), this duplicate will
drift and let through titles the regular check would reject — or vice
versa. At minimum, add a comment cross-referencing the source-of-truth
config. Better, derive the type list from a shared file.

Concurrency on authorize-agentic-ci.yml

agentic-ci-authorized-checks.yml has a concurrency group keyed by
pr_number, but authorize-agentic-ci.yml does not. If a maintainer
double-clicks the comment or two maintainers comment near-simultaneously,
two dispatch runs are kicked off and post duplicate "Authorized..."
comments. Consider:

concurrency:
  group: authorize-agentic-ci-${{ github.event.issue.number }}
  cancel-in-progress: false

Cosmetic, not a security issue, but reduces noise and double dispatches.

Minor

  • validate-dispatch runs on every pull_request and push to main,
    not just dispatch. With empty EXPECTED_HEAD_SHA it's a no-op, but it
    adds a runner-minute and an extra required-status edge in branch
    protection. Acceptable; just noting it isn't free.
  • authorize-agentic-ci.yml has actions: write at the workflow level.
    Only the final "Dispatch checks" step needs it; permissions: could
    be scoped per-job but the workflow has only one job, so the difference
    is academic.
  • gh api ... --paginate against /issues/.../timeline?per_page=100
    is fine for short-lived agentic PRs but unbounded in principle. Not
    worth fixing.
  • The 3-attempt loop with sleep 2 for timeline propagation is
    pragmatic. Worth a comment explaining why it retries (timeline
    eventual consistency after comment creation).
  • ${COMMENT_ID} defensive empty-check: in issue_comment events
    github.event.comment.id is always present, so the branch is dead
    but harmless.
  • /tmp/pr-body-raw.txt is written from PR body before grep — fine on
    isolated runners but the same path is reused across four workflows.
    No collision risk (different runners), but a unique-per-step name
    would be cleaner.

Test coverage

No unit-level tests are possible for workflow YAML in this repo.
The PR description notes manual dry-run validation against existing
PRs and successful checks on this PR itself. Given the security
sensitivity, recommend documenting the expected post-merge smoke test:
land an Agentic CI PR, comment /authorize-agentic-ci, verify all
three named checks turn green, then push a commit and verify the next
authorize attempt is rejected as stale.

Verdict

Approve with minor follow-ups. The security model is solid and
defense-in-depth is appropriate for a feature that exists specifically
to bypass branch protection. The two items I'd want addressed before
considering this stable:

  1. Extract the trust predicate into a composite action so the four
    copies cannot drift.
  2. Document the semantic-title regex source-of-truth or share the type
    list with the upstream check.

Concurrency, comments, and minor cleanups can land in a follow-up.

@greptile-apps

greptile-apps Bot commented May 14, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR introduces a maintainer-gated /authorize-agentic-ci slash command that lets trusted Agentic CI PRs (created by github-actions[bot] via GITHUB_TOKEN) receive required CI and policy checks without switching to a PAT or GitHub App token.

  • authorize-agentic-ci.yml validates commenter permission, a four-part trust predicate (bot author + same-repo + agentic-ci/* branch + HTML body marker), timeline ordering (no head push after the authorization comment), and a .github/ file guard — then dispatches both ci.yml and agentic-ci-authorized-checks.yml pinned to the authorized SHA.
  • agentic-ci-authorized-checks.yml re-validates the trust predicate and two independent SHA pinning checks (HEAD_SHA == EXPECTED_HEAD_SHA and GITHUB_SHA == HEAD_SHA) before emitting the DCOAssistant, semantic-pull-request / semantic-pull-request, and check job check runs required by branch protection.
  • ci.yml gains a validate-dispatch gate job that fails the run if dispatched on a different SHA than the maintainer authorized; dco-assistant.yml and pr-linked-issue.yml add matching early-exits for trusted Agentic CI PRs.

Confidence Score: 5/5

Safe to merge; the authorization flow is well-hardened and no regressions were identified in the normal PR path.

All five workflows were reviewed end-to-end. The trust predicate is consistently applied across every entry point, the SHA-pinning guard covers both the dispatch and the execution side, the timeline check closes the TOCTOU window, and the .github diff guard prevents privileged-file manipulation from being authorized. The normal pull_request path through ci.yml, dco-assistant.yml, and pr-linked-issue.yml is unchanged for all non-bot PRs.

No files require special attention; all changes are confined to GitHub Actions workflow files with no impact on application code.

Important Files Changed

Filename Overview
.github/workflows/authorize-agentic-ci.yml New workflow implementing the /authorize-agentic-ci slash command; validates commenter permission, trust predicate, timeline ordering, and .github file guard before dispatching CI and authorization checks.
.github/workflows/agentic-ci-authorized-checks.yml New workflow that re-validates trust and SHA pinning on dispatch, then emits the three required check runs (DCOAssistant, semantic-pull-request, linked-issue) for trusted Agentic CI PRs.
.github/workflows/ci.yml Adds validate-dispatch job that gates all downstream jobs when dispatched with an expected SHA, ensuring the workflow cannot run on a different commit than the maintainer authorized.
.github/workflows/dco-assistant.yml Adds a pre-check step that evaluates the Agentic CI trust predicate; the DCO Assistant action is skipped entirely for trusted bot-authored PRs.
.github/workflows/pr-linked-issue.yml Adds an early-exit path in the collaborator check step that treats trusted Agentic CI PRs as collaborator-authored, bypassing the linked-issue requirement.

Reviews (4): Last reviewed commit: "Merge branch 'main' into andreatgretel/f..." | Re-trigger Greptile

Comment thread .github/workflows/agentic-ci-authorized-checks.yml
Comment thread .github/workflows/authorize-agentic-ci.yml
@andreatgretel andreatgretel merged commit ff52770 into main May 20, 2026
50 checks passed
@andreatgretel andreatgretel deleted the andreatgretel/fix/ci-agentic-pr-bypass branch May 20, 2026 15:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants