Post GitHub PR comment on CI test failure#6818
Conversation
Posts a sticky GitHub PR comment with failing test names when a CI job fails, by parsing JUnit XML from fastlane/test_output/. Wired into the lint job as an initial test. Requires GITHUB_TOKEN_RC_OPS in the slack-secrets CircleCI context. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
CircleCI v2.1 requires << to be escaped as \<< anywhere in the config. Replaced the <<< here-string with an echo pipe and escaped the Ruby array append operator. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Intentional failure to verify github-notify-pr-on-fail posts a comment. Remove before merging. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…d logging Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Each job now uses <!-- cci-test-failure:$CIRCLE_JOB --> so a passing job only cleans up its own failure comment, not another job's. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…ring Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
ajpallares
left a comment
There was a problem hiding this comment.
Very nice QoL improvement indeed! I like it! I have added some comments and questions. Also, I wonder if we should have this logic in fastfile, and only have the orchestration in CircleCI. Then we would have the github-notify-pr-on-fail command be something like
- run:
when: on_fail
command: bundle exec fastlane notify_pr_ci_failure
- run:
when: on_success
command: bundle exec fastlane clear_pr_ci_failure
My thinking is that, if we're ever going to move to fully dynamic job execution (we're closer now that we have the dynamic config), keeping the config file as thin as possible will make that transition easier.
WDYT?
tonidero
left a comment
There was a problem hiding this comment.
Haven't reviewed this yet, but a thought that came to mind... We should make sure any output on these comments (from the tests results) does not contain any environment variable. CircleCI does obfuscate all environment variables appearances in logs/results, but not sure if the same could happen here... So we should be very careful with that
- Extract notify_pr_ci_failure and clear_pr_ci_failure into Fastlane lanes - Replace inline shell script in github-notify-pr-on-fail CCI command with thin bundle exec wrappers - Broaden XPath to //testcase[failure or error] to catch crashes and timeouts - Add early exit when DANGER_GITHUB_API_TOKEN is unset - Use Net::HTTP instead of curl for cleaner error handling - Fix missing slack-secrets context for lint job in release-or-main workflow and generate-requested-jobs-config.js Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
|
Moved all logic to Fastlane: two lanes ( |
|
The comment body only includes |
- test_output/ not fastlane/test_output/ (Fastlane CWD is ./fastlane/) - Wrap clear_pr_ci_failure in rescue so network errors don't fail passing CI jobs Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
ajpallares
left a comment
There was a problem hiding this comment.
Thanks for iterating on this! I think it looks great! Just one potential cleanup about using fastlane's github_api, but other than that, I think it's good to go!
…tion Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 83e6c6f. Configure here.
This reverts commit 83e6c6f.
Summary
github-notify-pr-on-failCircleCI command that posts (or updates) a sticky comment on the PR when a job failsfastlane/test_output/using Ruby stdlib REXML — no API calls to CircleCI neededlintjob as an initial test target🤖 Generated with Claude Code
Note
Medium Risk
Changes CI behavior by posting/updating/deleting GitHub PR comments via the GitHub API using a token from CircleCI contexts; misconfiguration could spam PRs or fail jobs if secrets/permissions are wrong.
Overview
Adds a new CircleCI command,
github-notify-pr-on-fail, that posts/updates a per-job “sticky” GitHub PR comment on failure (including failed test names parsed from JUnit XML) and deletes the comment on success.Implements the behavior in Fastlane via new lanes
notify_pr_ci_failureandclear_pr_ci_failure, plus a helper to find existing marker-tagged comments, and wires this into thelintandrun-test-ios-26jobs. Updates on-demand job config and workflows solintruns with theslack-secretscontext (to ensure required tokens are available).Reviewed by Cursor Bugbot for commit 37ead5e. Bugbot is set up for automated code reviews on this repo. Configure here.