Skip to content

fix(ci): avoid Agentic CI auth comment failure#686

Merged
johnnygreco merged 4 commits into
mainfrom
andreatgretel/fix/agentic-ci-auth-comment
May 26, 2026
Merged

fix(ci): avoid Agentic CI auth comment failure#686
johnnygreco merged 4 commits into
mainfrom
andreatgretel/fix/agentic-ci-auth-comment

Conversation

@andreatgretel

@andreatgretel andreatgretel commented May 20, 2026

Copy link
Copy Markdown
Contributor

📋 Summary

This PR prevents successful Agentic CI authorization runs from being marked failed after they already dispatched CI. The failure was caused by gh issue comment using GitHub's GraphQL addComment path, so the workflow now posts PR comments through the REST issue-comments endpoint and treats the final confirmation comment as non-fatal.

🔗 Related Issue

N/A - follow-up to the Agentic CI authorization workflow added in #643.

🔄 Changes

  • Replace every gh issue comment call in authorize-agentic-ci.yml with gh api --method POST repos/.../issues/.../comments.
  • Add small comment and comment_file helpers for denial comments, including multiline Markdown bodies.
  • Make comment-post failures emit workflow warnings instead of masking the intended authorization result.
  • Make the final success comment non-fatal after CI and authorization-check dispatches succeed.
  • Incorporate Claude review feedback in 2c053acd by hardening failure comments and using a temporary JSON payload for file-backed comment bodies.

🔍 Attention Areas

⚠️ Reviewers: Please pay special attention to the following:

  • authorize-agentic-ci.yml - this is the maintainer authorization path for generated Agentic CI PRs, and the comment behavior is intentionally split between fatal authorization failures and non-fatal informational comment failures.

🧪 Testing

  • make test passes - N/A, workflow-only change
  • Unit tests added/updated - N/A, workflow-only change
  • E2E tests added/updated - N/A, issue_comment workflows use the default-branch workflow file and need a post-merge smoke test
  • git diff --check
  • Parsed .github/workflows/authorize-agentic-ci.yml with PyYAML
  • bash -n over each workflow run: block
  • /home/ubuntu/Code/repos/DataDesigner/checkouts/main/.venv/bin/ruff check --fix .
  • /home/ubuntu/Code/repos/DataDesigner/checkouts/main/.venv/bin/ruff format .
  • Created and immediately deleted a temporary REST issue comment on PR fix(ci): avoid Agentic CI auth comment failure #686 using the same gh api endpoint shape
  • Claude review completed; non-blocking hardening feedback addressed
  • GitHub required checks passing on PR fix(ci): avoid Agentic CI auth comment failure #686

✅ Checklist

  • Follows commit message conventions
  • Commits are signed off (DCO)
  • Architecture docs updated - N/A, workflow-only CI fix

Signed-off-by: Andre Manoel <amanoel@nvidia.com>
@andreatgretel andreatgretel requested a review from a team as a code owner May 20, 2026 12:48
@greptile-apps

greptile-apps Bot commented May 20, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR fixes a workflow failure where successful Agentic CI authorization runs were being marked failed because gh issue comment routes through GitHub's GraphQL addComment path. All PR comment calls are now made through the REST issues-comments endpoint via gh api --method POST, and the final confirmation comment is treated as non-fatal.

  • All seven gh issue comment call sites are replaced with gh api --method POST repos/.../issues/.../comments, with new comment() and comment_file() shell helpers used in the "Validate Agentic CI PR" step.
  • Fatal authorization-failure paths (wrong permission, closed PR, stale head, blocked .github/ files) post a ::warning:: if the comment itself fails but still call exit 1 to mark the workflow failed.
  • The final success confirmation comment (after CI dispatch) is fully non-fatal, so a comment-post failure can no longer mask a successful authorization.

Confidence Score: 5/5

Safe to merge — the change is a targeted replacement of one comment-posting mechanism with another, with the intentional split between fatal authorization failures and non-fatal informational comment failures preserved correctly.

The authorization control flow is unchanged: every path that should call exit 1 still does. The new comment() and comment_file() helpers correctly inherit the step's env vars, the jq --rawfile idiom properly JSON-encodes multiline Markdown for the REST body, and the temp-file cleanup via trap RETURN fires before the subsequent exit 1 calls. No logic regressions were found.

No files require special attention.

Important Files Changed

Filename Overview
.github/workflows/authorize-agentic-ci.yml All gh issue comment calls replaced with REST gh api --method POST equivalents; comment() and comment_file() helpers added; failure comments are now non-fatal (emit ::warning::); success confirmation comment made non-fatal after CI dispatch.

Sequence Diagram

sequenceDiagram
    actor Maintainer
    participant GH as GitHub (issue_comment)
    participant WF as authorize-agentic-ci.yml
    participant API as GitHub REST API
    participant CI as ci.yml / authorized-checks.yml

    Maintainer->>GH: comment /authorize-agentic-ci
    GH->>WF: trigger workflow

    WF->>API: GET collaborators permission
    alt insufficient permission
        WF->>API: POST issues/comments (non-fatal warn on fail)
        WF-->>GH: exit 1 (step fails)
    end

    WF->>API: GET pulls/PR_NUMBER (PR metadata)

    alt PR not open / not trusted / comment ID missing / stale head / .github files changed
        WF->>API: POST issues/comments (non-fatal warn on fail)
        WF-->>GH: exit 1 (step fails)
    end

    WF->>CI: gh workflow run ci.yml
    WF->>CI: gh workflow run agentic-ci-authorized-checks.yml
    WF->>API: POST issues/comments Authorized... (warn on fail, non-fatal)
    WF-->>GH: success
Loading

Reviews (4): Last reviewed commit: "Merge branch 'main' into andreatgretel/f..." | Re-trigger Greptile

@github-actions

Copy link
Copy Markdown
Contributor

Review: PR #686 — fix(ci): avoid Agentic CI auth comment failure

Summary

This PR modifies .github/workflows/authorize-agentic-ci.yml to:

  1. Replace gh issue comment with gh api --method POST repos/.../issues/.../comments for every comment posted by the workflow.
  2. Introduce two local shell helpers (comment and comment_file) inside the "Validate Agentic CI PR" step to DRY the repeated gh api calls.
  3. Make the final "authorization succeeded" comment non-fatal — if posting the confirmation comment fails after the dispatch has already happened, the workflow no longer reports authorization as failed; it emits a ::warning:: instead.

Diff is small (+22/-14, single workflow file). PR-only docs change in the workflow itself; no production code, no tests required.

Findings

Correctness

  • Equivalence of gh issue comment vs gh api — both endpoints ultimately POST to /repos/:owner/:repo/issues/:number/comments. The -f body=... form-encodes the value, which is the right transport for plain string bodies. The semantic behavior is the same; the swap is reasonable as a workaround for whatever was making gh issue comment fail (the PR description doesn't explain the underlying cause — a sentence on the failure mode would help future readers, but isn't blocking).
  • Multi-line body handling via comment_file is correct: jq -n --rawfile body "$1" '{body: $body}' safely produces a JSON object from arbitrary file content (newlines, backticks, quotes), and --input sends it as the request body. Better than relying on shell expansion for multi-line bodies.
  • Non-fatal success comment — the final || echo "::warning::..." is the substantive behavioral change. The dispatches (gh workflow run ...) on lines 179–183 happen before the comment, and an unposted confirmation does not invalidate the launched runs, so degrading to a warning is the correct semantics.
  • Error-path comments are still fatal — they remain followed by exit 1, which preserves the original semantics: when authorization is denied, the workflow fails. Good asymmetry between success path (best-effort comment) and failure path (must report).
  • Stderr visibility>/dev/null redirects only stdout, so any gh api error output still surfaces in the runner logs for debugging. Correct.

Style / consistency

  • Helper duplication across steps. The comment() / comment_file() helpers are defined in the "Validate Agentic CI PR" step, but identical inline gh api ... comments invocations appear in two other steps ("Check commenter permission", line 44–45; "Dispatch checks", line 185–186). Bash functions don't cross step boundaries in GitHub Actions, so duplication here is structurally necessary, but it's worth noting that any future change to the comment idiom (auth header, pagination, error handling) needs to be applied in three places. A future refactor could extract a composite action or a single script file under .github/actions/ if this pattern keeps growing — not blocking for this PR.
  • comment_file overwrites the same temp path (/tmp/agentic-ci-auth-comment.json) across both call sites. Calls are sequential within a single step, so no race, but a mktemp would be marginally safer if this helper ever moves into a context with concurrency.
  • JSON-encoded vs form-encoded bodies are inconsistentcomment() uses -f body=... (form), comment_file() uses --input <json> (JSON). Both work for this endpoint. Worth a one-line comment in the workflow noting why two paths exist (single-line vs multi-line bodies), since a reader might otherwise wonder.

Security

  • Untrusted-content surface unchanged. All bodies posted by these helpers are workflow-controlled strings or files written by the workflow itself (/tmp/agentic-ci-auth-stale.md, /tmp/agentic-ci-auth-failed.md). The BLOCKED shell variable derived from gh pr diff --name-only is interpolated into a markdown bullet list in the failed.md case, but it's escaped only as backtick-wrapped paths — a maliciously named file like `; rm -rf /` would not execute (it's printed inside backticks in markdown, not eval'd), so this remains safe.
  • Authorization gating is unchanged. Permission check, trusted-PR check, comment-found check, head-not-changed check, and .github/ blocklist are all preserved. This PR changes only the comment transport and the success-path failure handling, not the security envelope.

Performance / scope

  • No measurable runtime change — same number of HTTP calls, same payloads.
  • Diff is contained to one workflow file. Within .github/workflows/, which is correctly outside the agentic-CI-modifiable surface.

Suggestions (non-blocking)

  1. Consider adding a short comment in the workflow explaining why gh api is used in place of gh issue comment (e.g., "gh issue comment intermittently fails with X — using REST directly avoids that path"). Otherwise a future reader will be tempted to "simplify" it back.
  2. If the comment idiom keeps spreading, factor it into a composite action (.github/actions/post-pr-comment) so the three call sites collapse to one definition.
  3. mktemp for the JSON staging file in comment_file is a cheap defensive upgrade.

Verdict

Approve with minor optional improvements. The change is small, targeted, and improves robustness of the authorization workflow without weakening its security checks. The non-fatal success comment is the right call — once the dispatches have fired, a comment failure should not retroactively mark the run as a failed authorization. The gh api swap is fine as a transport-level workaround, though the PR would benefit from one sentence explaining the original failure mode it's working around.

Signed-off-by: Andre Manoel <amanoel@nvidia.com>
@johnnygreco

Copy link
Copy Markdown
Contributor

Andre, this looks good to me. I reviewed the workflow change for correctness and didn't find any issues; the REST comment path and non-fatal confirmation comment handling both look appropriate.

@johnnygreco johnnygreco left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me.

@johnnygreco johnnygreco merged commit 15d0b35 into main May 26, 2026
50 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants