• Replace hardcoded ruleset IDs with dynamic JSON-based configuration
• Create and delete rulesets programmatically during release workflow
• Add CI success verification job to validate all required checks
• Improve release process robustness and maintainability

Release passing checks ruleset configuration

• New ruleset configuration file for release phase
• Enforces required status checks on default branch
• Prevents deletion and non-fast-forward pushes
• Specifies four required CI checks: RBE tests, CI Success, workflow validation, and format check

.github/rulesets/release-require-passing.json

Release trunk restriction ruleset configuration

• New ruleset configuration file for release phase
• Restricts trunk branch updates during release
• Prevents deletion and non-fast-forward pushes
• Allows release team to bypass restrictions

.github/rulesets/release-restrict-trunk.json

Add CI success verification job

• Add new ci-success job that runs after all other CI jobs
• Verifies that no required jobs failed or were cancelled
• Exits with error code if any job failed, blocking release

.github/workflows/ci.yml

Description

The nightly workflow only sets release-in-progress=true when the rulesets query succeeds and
returns true; if gh api fails (e.g., auth/API error), the condition evaluates false and the output
is left unset, so nightly releases proceed despite an unknown release state. This can cause the
nightly pipeline to run during an active release or when the ruleset check is unavailable.

Code

.github/workflows/nightly.yml[R55-59]

+          GH_TOKEN: ${{ secrets.SELENIUM_CI_TOKEN }}
+          GH_REPO: ${{ github.repository }}
        run: |
          set -euo pipefail
-          if ! ENFORCEMENT=$(gh api /repos/${{ github.repository }}/rulesets/11911909 --jq '.enforcement' 2>/dev/null); then
-            echo "::warning::Failed to check release ruleset, assuming release in progress"
+          if gh api "repos/$GH_REPO/rulesets" --jq 'any(.[]; .name | startswith("Release:"))' | grep -q true; then

Evidence

The release-in-progress workflow output is consumed to decide whether to run nightly releases, but
the check step only sets it in the then branch and has no else/fallback, so errors or non-matches
leave it unset and the nightly-release job runs.

.github/workflows/nightly.yml[35-40]
.github/workflows/nightly.yml[52-61]
.github/workflows/nightly.yml[63-67]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

### Issue description
The `prepare` job’s “Check if release ruleset is active” step only writes `release-in-progress=true` in the success path. If the `gh api` call fails, the step produces no output, and downstream logic treats it as not in progress.

### Issue Context
`needs.prepare.outputs.release-in-progress` gates the nightly release job. An unset output is not equal to `'true'`, so the nightly release will run even when the ruleset check failed.

### Fix Focus Areas
- .github/workflows/nightly.yml[52-61]

### Suggested fix
Update the step to be fail-safe and to always write an explicit output value.

For example:
- Call `gh api` and check its exit status.
- If the API call fails, emit a warning and set `release-in-progress=true` (safe default).
- Otherwise set `release-in-progress=true|false` based on the jq result.

(Any equivalent approach that distinguishes “false” from “error” and always sets the output is fine.)

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

Description

The nightly release gate checks for "Release:" rulesets using gh api "repos/$GH_REPO/rulesets"
without pagination, so it may miss matching rulesets beyond the first page and incorrectly proceed
with nightly releases during an active release. This makes the workflow’s behavior non-deterministic
based on ruleset count/order, violating the script hardening requirement.

Code

.github/workflows/nightly.yml[59]

+          if gh api "repos/$GH_REPO/rulesets" --jq 'any(.[]; .name | startswith("Release:"))' | grep -q true; then

Evidence

PR Compliance ID 13 requires hardened scripts with deterministic behavior. The new check uses `gh
api to list rulesets but does not use --paginate`, so the jq/grep test can miss matches when
results span multiple pages.

.github/workflows/nightly.yml[55-60]
Best Practice: Learned patterns

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The workflow queries `repos/$GH_REPO/rulesets` without pagination, so the "release in progress" detection can be incomplete and non-deterministic.

## Issue Context
GitHub API list endpoints can paginate; if the target rulesets are not returned on the first page, the check may incorrectly return `false` and allow nightly jobs to run.

## Fix Focus Areas
- .github/workflows/nightly.yml[55-60]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

Description

The ruleset existence check uses jq -e any(.[]; .name == $n) against $existing without ensuring
$existing is a single JSON array, so API errors or multi-document output (e.g., from `gh api
--paginate) can cause parse/iteration failures or make jq` decide based only on the last page,
potentially missing an existing ruleset and creating a duplicate that breaks or complicates the
trunk restrict/unrestrict workflow.

Code

.github/workflows/restrict-trunk.yml[66]

+            if jq -e --arg n "$name" 'any(.[]; .name == $n)' <<<"$existing" >/dev/null; then

Evidence

PR Compliance ID 13 requires robust handling of external inputs, but the workflow captures the `gh
api ... --paginate response into existing and then runs jq -e --arg n "$name" 'any(.[]; .name ==
$n)' without first slurping/merging paginated pages into a single array. Because --paginate` can
emit multiple JSON documents (one per page) or non-array/error output, iterating .[] assumes a
single array and can fail to parse/iterate, and even when it parses as multiple documents, jq -e’s
exit status is determined by the last document so a ruleset present on an earlier page can be missed
and a duplicate created.

.github/workflows/restrict-trunk.yml[66-66]
.github/workflows/restrict-trunk.yml[61-71]
Best Practice: Learned patterns

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The workflow’s ruleset existence check uses `jq -e --arg n "$name" 'any(.[]; .name == $n)' <<<"$existing"`, which assumes `$existing` is a single valid JSON array. When `gh api ... --paginate` emits multiple JSON documents (one per page), or when the API returns unexpected/error output, this can cause parse/iteration failures or make the check effectively depend on only the last page, potentially missing an existing ruleset and creating a duplicate, disrupting trunk restrict/unrestrict behavior.

## Issue Context
This job locks/unlocks trunk during release; failures can block releases or leave trunk in the wrong state. Missing an existing ruleset due to pagination can create duplicate rulesets with the same `.name`, making subsequent deletion/unlock behavior ambiguous or incomplete.

## Fix Focus Areas
- .github/workflows/restrict-trunk.yml[61-67]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

Description

The new nightly release gate assumes the GitHub API response is a JSON array and that every element
has a string .name, which can cause jq to error and fail the workflow rather than
deterministically handling an unexpected/invalid response. This violates the requirement to validate
external/protocol inputs early with actionable handling.

Code

.github/workflows/nightly.yml[R59-60]

+          rulesets=$(gh api "repos/$GH_REPO/rulesets" 2>/dev/null || echo '[{"name":"Release: api-error"}]')
+          if jq -e 'any(.[]; .name | startswith("Release:"))' <<<"$rulesets" >/dev/null; then

Evidence

Rule 12 requires validating external/protocol-derived inputs before use. The added code directly
iterates .[] and applies startswith() to .name without guarding for non-array responses or
missing/non-string .name, which can produce non-deterministic failures when the API response shape
is unexpected.

.github/workflows/nightly.yml[59-60]
Best Practice: Learned patterns

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The workflow parses `gh api` output with `jq -e 'any(.[]; .name | startswith("Release:"))'`, which assumes the response is a JSON array and `.name` is always a string. If the API ever returns an unexpected shape (or a page/object), `jq` can error and fail the job instead of handling the condition deterministically.

## Issue Context
This step controls whether `nightly-release` runs. Since the input comes from an external API, it should be validated/handled defensively.

## Fix Focus Areas
- .github/workflows/nightly.yml[59-60]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

Description

The nightly release gate suppresses gh api stderr and falls back to a synthetic ruleset name that
forces release-in-progress=true, so nightlies can be skipped with no visible indication that the
GitHub API call actually failed. This makes operational triage difficult (release vs auth/rate-limit
outage) and can lead to unexplained skipped nightlies.

Code

.github/workflows/nightly.yml[R59-60]

+          rulesets=$(gh api "repos/$GH_REPO/rulesets" 2>/dev/null || echo '[{"name":"Release: api-error"}]')
+          if jq -e 'any(.[]; .name | startswith("Release:"))' <<<"$rulesets" >/dev/null; then

Evidence

The gate hides API failures by discarding stderr and substituting a JSON payload that will match the
Release: prefix check, which then sets release-in-progress=true and causes nightly-release to
be skipped with no warning.

.github/workflows/nightly.yml[52-62]
.github/workflows/nightly.yml[64-68]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

### Issue description
The nightly workflow suppresses `gh api` errors (`2>/dev/null`) and uses a fallback JSON that intentionally matches `startswith("Release:")`. That correctly fail-safes by skipping nightlies, but it provides no warning explaining the skip was due to an API/auth failure.

### Issue Context
This happens in the `prepare` job’s “Check if release ruleset is active” step. The `nightly-release` job is gated by `needs.prepare.outputs.release-in-progress != 'true'`, so the silent fallback can skip the main work without any signal as to why.

### Fix Focus Areas
- `.github/workflows/nightly.yml[59-60]`

### Suggested change
Refactor the command substitution to detect failure and emit a `::warning::` (or at minimum `echo` a message) before applying the fallback. For example:

```bash
if ! rulesets=$(gh api "repos/$GH_REPO/rulesets" 2>/dev/null); then
 echo "::warning::Failed to fetch rulesets via gh api; treating as release-in-progress"
 rulesets='[{"name":"Release: api-error"}]'
fi

if jq -e 'any(.[]; .name | startswith("Release:"))' <<<"$rulesets" >/dev/null; then
 echo "release-in-progress=true" >> "$GITHUB_OUTPUT"
fi
```

(Keep the current fail-safe behavior, but make the reason visible in logs.)

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

Description

The existence check is implemented as a jq command inside an if, so any non-zero jq exit code
(including parse/runtime errors) is treated the same as “not found” and falls through to create a
ruleset. This can cause the workflow to attempt POSTs when it cannot correctly interpret the current
ruleset state.

Code

.github/workflows/restrict-trunk.yml[R63-70]

+          existing=$(gh api "repos/$GH_REPO/rulesets")
+          for f in .github/rulesets/release-*.json; do
+            name=$(jq -r .name "$f")
+            if jq -e --arg n "$name" 'any(.[]; .name == $n)' <<<"$existing" >/dev/null; then
+              echo "Ruleset '$name' already exists; skipping"
+            else
+              echo "Creating ruleset: $name"
+              gh api -X POST "repos/$GH_REPO/rulesets" --input "$f"

Evidence

The current code runs jq ... directly in the if condition and uses else as the default path;
this means any jq non-zero exit status will take the creation path, regardless of why jq failed.

.github/workflows/restrict-trunk.yml[63-70]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

### Issue description
`jq` is used as the `if` predicate for “already exists”; if `jq` fails for reasons other than “no match”, the script executes the `else` branch and attempts to create a ruleset.

### Issue Context
This makes API/JSON issues behave like a missing ruleset, which can lead to unintended POSTs and partial/incorrect state.

### Fix
Explicitly distinguish:
- exit 0: found
- exit 1: not found
- exit >1: error (fail the step)

Example:
```bash
jq -e --arg n "$name" 'any(.[]; .name == $n)' <<<"$existing" >/dev/null
rc=$?
if [ $rc -eq 0 ]; then
 echo "Ruleset '$name' already exists; skipping"
elif [ $rc -eq 1 ]; then
 echo "Creating ruleset: $name"
 gh api -X POST "repos/$GH_REPO/rulesets" --input "$f"
else
 echo "Failed to evaluate existing rulesets (jq rc=$rc)" >&2
 exit $rc
fi
```

### Fix Focus Areas
- .github/workflows/restrict-trunk.yml[63-70]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools