feat(ci): advisory ripr lane (#182, thin adoption)#230
Conversation
Adopts the external EffortlessMetrics/ripr CLI as a never-blocking PR-time
exposure lane. Shipper consumes ripr; Shipper does not embed RIPR analysis.
ripr is static mutation-exposure analysis (crates.io/crates/ripr 0.5.0).
It reads a PR diff, builds mutation-shaped probes from changed behaviour,
and asks whether existing tests appear to expose that behaviour through a
meaningful oracle. It does NOT run mutants — mutation testing remains the
runtime backstop, scoped to nightly/release lanes via mutation.yml (no
changes here).
New files:
ripr.toml
Canonical schema regenerated via `ripr init --root . --dry-run` against
ripr 0.5.0. One Shipper override: `[suppressions].path` is rerouted to
`policy/ripr-suppressions.toml` so the ledger lives alongside other
Shipper policy artifacts rather than at the ripr default
`.ripr/suppressions.toml`.
policy/ripr-suppressions.toml
Empty receipt ledger. ripr's required `[[suppression]]` fields are
`finding_id`, `path`, `owner`, `reason`; Shipper adds the `created` /
`review_after` conventions so suppressions age out in line with the
rest of policy/.
.github/workflows/ripr.yml
Advisory PR-only workflow. Triggers on changes to crates/, xtask/,
Cargo.toml, Cargo.lock, ripr.toml, the suppressions ledger, and the
workflow itself. Installs `ripr` pinned via `cargo install ripr
--locked --version 0.5.0`, runs `cargo xtask ripr-pr`, uploads
target/ripr/ as a 30-day artifact. `continue-on-error: true` keeps a
failing ripr exit code from breaking the PR's overall status — ripr is
a signal, not a gate. Concurrency cancels in-progress runs for the
same PR so a new commit supersedes the prior report.
xtask/src/ripr.rs + `cargo xtask ripr-pr`
Thin wrapper around the external CLI. If `ripr` is missing on PATH
locally, prints install instructions and exits advisory-success (does
not break the developer's session). In CI, the workflow pre-installs a
pinned version. Two unit tests guard the version pin stays in sync
between the install hint and the workflow.
docs/ci/ripr.md
Updated to current ripr terminology — "static mutation-exposure
analysis" rather than the stale "reachable incremental PR coverage" /
"reachable mutants not covered by any test" wording. Adds the
three-tier framing (coverage → ripr → mutation testing), the canonical
`ripr.toml` schema, and the ripr-required suppression fields.
Receipts:
policy/workflow-allowlist.toml
New `[[workflow]]` entry for .github/workflows/ripr.yml, kind
"ci_advisory", process/network policies "ci". Existing process and
network profiles already cover `cargo install` and crates.io, so no
changes there.
policy/non-rust-allowlist.toml
New `[[file]]` entry for `ripr.toml` at the repo root (the policy/**
glob already covers policy/ripr-suppressions.toml).
Scope intentionally excludes:
- `cargo xtask mutants-pr --changed` — deferred; ripr lane is the
priority.
- Mutation workflow scoping changes (PR-trigger gating on `mutation`
label) — deferred to keep this PR thin.
- Required-status promotion for the ripr job — ripr is and stays
advisory.
CHANGELOG.md Added entry; #182 referenced (not yet closed)
|
Warning You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again! |
|
Droid finished @EffortlessSteven's task —— View job Validation Complete ✓Review Status: Summary: LGTM — all PR changes reviewed: the ripr advisory CI lane (continue-on-error, pinned version, thin xtask wrapper with version-sync tests), the ripr.toml config, policy ledger entries, and updated docs are consistent, well-structured, and free of functional or security issues. Candidates Validated: 0 No inline review comments were generated by the primary reviewer; the review focuses on the overall PR structure and integration correctness. |
|
Warning Rate limit exceeded
You’ve run out of usage credits. Purchase more in the billing tab. ⌛ How to resolve this issue?After the wait time has elapsed, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout. Please see our FAQ for further information. ℹ️ Review info⚙️ Run configurationConfiguration used: Organization UI Review profile: ASSERTIVE Plan: Pro Run ID: 📒 Files selected for processing (9)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 76a9c9171b
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| schema_version = "1.0" | ||
| policy = "ripr-suppressions" | ||
| owner = "EffortlessMetrics" | ||
| status = "active" |
There was a problem hiding this comment.
Use ripr's suppressions manifest schema
When ripr pilot loads this configured suppressions path to render the repo-exposure/badge artifacts, this file is not parseable by ripr 0.5.0: its suppressions parser accepts only an integer schema_version = 1 header at top level and reports any other top-level key as an unsupported field, while these lines use a string version plus policy, owner, and status. That makes the new advisory lane fail before producing the artifacts it is meant to upload; make this file match ripr's manifest shape, e.g. only schema_version = 1 until real [[suppressions]] entries are added.
Useful? React with 👍 / 👎.
…on} projection (#182 follow-up) (#231) Brings `cargo xtask ripr-pr` and `.github/workflows/ripr.yml` up to the PR 1 spec that #230 partially shipped. Two surfaces change: 1. `cargo xtask ripr-pr [--base <ref>]`. Defaults to `origin/main`. The flag is currently forward-looking: `ripr pilot` operates on the working tree and has no `--base` flag, so the wrapper acknowledges and ignores non-default values today. The shape is stable for the eventual switch to `ripr check --base <ref>` once that format contract is something Shipper wants to consume. 2. Projection step after `ripr pilot` writes its native outputs: target/ripr/pilot/pilot-summary.md -> target/policy/ripr-report.md target/ripr/pilot/pilot-summary.json -> target/policy/ripr-report.json So ripr's report sits alongside the other policy reports in `target/policy/` and a future `policy-report` integration can read it without crawling into ripr's per-mode directory layout. Each side is best-effort: a missing source is a quiet skip, not a wrapper failure. The JSON source is `pilot-summary.json` (~13 KB), not `repo-exposure. json` (~53 MB) or `agent-seam-packets.json` (~34 MB) which would be too heavy to republish as a CI artifact. The comment in `ripr.rs` records the choice. 3. `.github/workflows/ripr.yml`: - `cargo xtask ripr-pr --base origin/main` (explicit form for log legibility) - artifact upload glob expands from `target/ripr/` to `target/ripr/` + `target/policy/ripr-report.*` Tests: ripr::tests::install_hint_mentions_pinned_version (pre-existing) ripr::tests::install_hint_pinned_version_matches_workflow (pre-existing) ripr::tests::project_one_skips_missing_source (new) ripr::tests::args_default_base_is_origin_main (new) 19/19 tests passing (15 no_panic + 4 ripr). Clippy and fmt clean.
Adds public README badges for `ripr` and `ripr+` as Shields endpoint JSON committed under `badges/`. Per upstream ripr policy, README badges must be repo-scoped — a diff-scoped artifact would read `0` on `main` simply because no diff exists, not because the repo is clean. The PR-time pilot artifact added in #230 stays diff-scoped and is never republished as a README badge. cargo xtask repo-ripr-badge-artifacts runs `ripr check --root . --mode ready --format repo-exposure-json`, extracts `metrics.headline_eligible` (the count of repo seams the configured `[severity.seams]` policy treats as non-off), maps the number to a Shields color, and writes both endpoint files. The command REQUIRES `ripr` on PATH — unlike `cargo xtask ripr-pr` which is advisory locally, regenerating badges with stale numbers is worse than failing loud. Threshold mapping for Shields color: 0 brightgreen (inbox-zero goal) 1..=99 yellowgreen 100..=999 orange 1000+ red Initial Shipper count: 2,711 (red). Both badges currently project the same count; differentiating `ripr+` to include unsuppressed test-efficiency findings is upstream territory and deferred. Receipts and metadata: policy/generated-allowlist.toml `badges/*.json` glob, kind `shields_badge_endpoint`, classification `generated` .gitattributes `badges/*.json` linguist-generated README.md two new badges (ripr, ripr+) linking docs/ci/ripr.md docs/ci/ripr.md new "Repo Badges" section: scope (repo not diff), Shields URLs, schema, regen command, threshold mapping, refresh cadence Refresh cadence is intentionally manual — the badge is a repo health endpoint, not a PR tax. An optional workflow_dispatch refresh job is mentioned as a future extension but not added here. Tests: ripr::tests::shields_color_thresholds (new) ripr::tests::write_shields_endpoint_shape (new) 21/21 tests passing.
* feat(ci): mutation testing PR-time lane + cargo xtask mutants-pr (#182 PR 3) Adds a label-gated PR-time mutation lane and the xtask wrapper it invokes, keeping mutation off every PR's hot path while making it cheap to opt in when warranted. cargo xtask mutants-pr --changed [--base origin/main] [--dry-run] is a thin wrapper around `cargo-mutants` that: 1. Refuses to run without `--changed`. Whole-workspace mutation is intentionally not part of the PR-time lane — that lives in the weekly schedule (see docs/ci/test-evidence-lanes.md). 2. Computes `git diff <base>...HEAD --name-only -- '*.rs'`, filters out `tests/` and `benches/` paths (cargo-mutants only mutates production source). 3. Runs `cargo mutants --no-shuffle --file <each-path>` (or `--list` when `--dry-run` is passed). 4. If `cargo-mutants` is missing locally, prints install instructions and exits advisory-success rather than erroring. CI installs the tool before invoking. 5. Surfaces `cargo mutants` non-zero exit codes — surviving mutants are load-bearing failures, unlike ripr's purely advisory findings. `.github/workflows/mutation.yml` adds the `mutants-pr` job alongside the existing `mutants-weekly` job. The PR job's gate: if: github.event_name == 'pull_request' && ( contains(github.event.pull_request.labels.*.name, 'mutation') || contains(github.event.pull_request.labels.*.name, 'full-ci') ) The weekly job's crate list stays unchanged (shipper-duration, shipper-types, shipper-config). Expanding it to the full trust-critical surface (shipper-core, shipper-encrypt, etc.) is too expensive for a 60-minute job and is its own future rollout step; the policy stays about routing, not coverage scope. xtask/src/mutants.rs new — 3 unit tests xtask/src/main.rs `mutants-pr` subcommand wiring .github/workflows/mutation.yml two-job split (mutants-weekly + mutants-pr) docs/ci/test-evidence-lanes.md advisory-routed table + targeted mutation section updated CHANGELOG.md Added entry Tests: mutants::tests::install_hint_mentions_cargo_install (new) mutants::tests::args_defaults_are_explicit (new) mutants::tests::changed_requires_the_flag (new) 24/24 tests passing. Refs #182 (this is the third PR in the #182 sequence: advisory lane in #230, --base + projection in #231, badges in #232, mutation scoping here). * fix(xtask): wrap mutants-pr doc-comment Behaviour block in fenced code `<REF>` inside a doc comment is parsed by rustdoc as an invalid HTML tag under `-D rustdoc::invalid-html-tags` (implied by RUSTDOCFLAGS=-D warnings on the CI Documentation job). Wrap the whole Behaviour block in a ```text fence so the angle brackets are literal.
Summary
Thin adoption of the external EffortlessMetrics/ripr CLI (
crates.io/crates/ripr0.5.0) as a never-blocking PR-time exposure lane.Shipper consumes ripr; Shipper does not embed RIPR analysis. Authoring RIPR inside
xtaskwould explode scope to thousands of LOC and duplicate a maintained external crate. This PR wires the integration only.What changed
ripr.toml(new) — canonical schema generated viaripr init --root . --dry-runagainst ripr 0.5.0, with one Shipper override:[suppressions].path = \"policy/ripr-suppressions.toml\"so the ledger lives alongside other policy artifacts instead of the ripr default.ripr/suppressions.toml.policy/ripr-suppressions.toml(new) — empty schema-valid ledger. ripr requires[[suppression]]withfinding_id,path,owner,reason; Shipper addscreated/review_after..github/workflows/ripr.yml(new) — advisory PR-only workflow. Triggers oncrates/**,xtask/**,Cargo.{toml,lock},ripr.toml, the suppressions ledger, and itself. Installs ripr pinned viacargo install ripr --locked --version 0.5.0, runscargo xtask ripr-pr, uploadstarget/ripr/artifact.continue-on-error: true; never required. Concurrency cancels in-progress runs.xtask/src/ripr.rs+cargo xtask ripr-pr(new) — thin wrapper that shells out toripr pilot --root .. Ifripris missing on PATH locally, prints install instructions and exits advisory-success so a missing binary never breaks a developer's session. Two unit tests guard that the install hint's pinned version stays in sync with the workflow.docs/ci/ripr.md— updated to current ripr terminology. The doc previously described "reachable incremental PR coverage" / "reachable mutants not covered by any test." That is stale relative to ripr 0.5.0, which is static mutation-exposure analysis (does NOT find or run mutants). The doc now uses the three-tier framing (coverage → ripr → mutation testing), the canonicalripr.tomlschema, and the ripr-required suppression fields.policy/workflow-allowlist.toml— receipt for.github/workflows/ripr.ymlaskind = \"ci_advisory\".policy/non-rust-allowlist.toml— receipt forripr.toml(thepolicy/**glob already covers the suppressions ledger).CHANGELOG.md— Added entry under [Unreleased].Scope explicitly excludes (deferred)
cargo xtask mutants-pr --changed— ripr lane is the priority; revisit when needed.mutation.ymlPR-trigger gating onmutationlabel) — keep this PR thin.Test plan
cargo build -p xtask— cleancargo test -p xtask— 17/17 passing (15 no_panic + 2 ripr)cargo clippy --workspace --all-targets -- -D warnings— cleancargo fmt --all -- --check— cleancargo xtask ripr-prlocally —ripr pilot --root .ran against Shipper and surfaced its top recommendation (engine/parallel/mod.rs:281)cargo xtask check-file-policy --mode blocking-allowlist— unreceipted=0cargo xtask policy-report— all 8 areas cleanWhy the issue's wording was stale
docs/ci/ripr.mdand #182's investigation section were written before ripr 0.5.0 settled on the current terminology and schema. First step was reconciling the issue against ripr's current README andripr init --dry-runoutput, not coding to the stale plan. Same "claim → check current truth → act" pattern that produced thecfg(not(test))fix and themanual_pop_ifrejection.Refs #182.