Skip to content

xtask: add scanner-safe-reference --check command#577

Merged
EffortlessSteven merged 1 commit into
mainfrom
xtask/scanner-safe-reference-check
May 11, 2026
Merged

xtask: add scanner-safe-reference --check command#577
EffortlessSteven merged 1 commit into
mainfrom
xtask/scanner-safe-reference-check

Conversation

@EffortlessSteven

Copy link
Copy Markdown
Member

Why

The scanner-safe bundle reference under examples/scanner-safe-bundle/expected/ (added in #564 as part of v0.7.0) is part of the user-visible release story documented in examples/scanner-safe-bundle/README.md and docs/how-to/generate-scanner-safe-k8s-secret.md. Downstream consumers diff their regenerated bundle metadata against those committed reference files to verify deterministic generator stability.

Until now nothing in CI caught silent drift between the committed reference and the actual generator output. This is v0.7.1 PR 3: turn the reference outputs into an executable gate so the user-facing workflow stays trustworthy.

What

Adds cargo xtask scanner-safe-reference --check, which:

  • Regenerates the bundle under target/scanner-safe-reference/ via the existing uselesskey-cli commands (bundle, verify-bundle, inspect-bundle, export k8s, export vault-kv-json).
  • Byte-equal compares the regenerated manifest.json, receipts/audit-surface.json, and receipts/materialization.json against examples/scanner-safe-bundle/expected/. On drift it reports the first differing line; it does NOT overwrite the committed reference.
  • Asserts the encoded payloads secret.yaml and kv-v2.json are NOT committed under examples/scanner-safe-bundle/expected/ — they must stay under target/ (the scanner-safe lesson: commit metadata + low-entropy shapes, never encoded Kubernetes / Vault payloads).
  • Re-runs cargo xtask no-blob and cargo xtask check-file-policy against the reference directory.

No CLI changes; this only wires existing CLI commands into a new xtask gate.

Test plan

  • cargo build -p xtask
  • cargo fmt --check -p xtask
  • cargo clippy -p xtask --all-targets -- -D warnings
  • cargo xtask scanner-safe-reference --check passes locally against the currently committed reference: scanner-safe-reference: ok (3 files matched)
  • CI green

Add a `cargo xtask scanner-safe-reference --check` command that turns the
committed scanner-safe-bundle reference outputs under
`examples/scanner-safe-bundle/expected/` into an executable gate.

The command:

- Regenerates the scanner-safe bundle under `target/scanner-safe-reference/`
  via the existing `uselesskey-cli` commands (bundle, verify-bundle,
  inspect-bundle, export k8s, export vault-kv-json).
- Byte-equal compares the regenerated `manifest.json`,
  `receipts/audit-surface.json`, and `receipts/materialization.json` against
  the committed reference; on drift it reports the first differing line
  rather than silently overwriting.
- Asserts that the encoded payloads `secret.yaml` and `kv-v2.json` are NOT
  committed under `examples/scanner-safe-bundle/expected/`; those must stay
  under `target/`.
- Re-runs `cargo xtask no-blob` and `cargo xtask check-file-policy` against
  the reference directory.

User-path protection: the scanner-safe bundle reference is part of the
v0.7.0 user story documented in `examples/scanner-safe-bundle/README.md`
and `docs/how-to/generate-scanner-safe-k8s-secret.md`. Without this gate,
silent reference drift would degrade that workflow for downstream
consumers diffing against the committed metadata. This wires the
reference into CI as v0.7.1 prep.
@gemini-code-assist

Copy link
Copy Markdown

Warning

You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again!

@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, add credits to your account and enable them for code reviews in your settings.

@coderabbitai

coderabbitai Bot commented May 11, 2026

Copy link
Copy Markdown

Warning

Rate limit exceeded

@EffortlessSteven has exceeded the limit for the number of commits that can be reviewed per hour. Please wait 23 minutes and 4 seconds before requesting another review.

You’ve run out of usage credits. Purchase more in the billing tab.

⌛ How to resolve this issue?

After the wait time has elapsed, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout.

Please see our FAQ for further information.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: 691e1a34-9025-473f-8d73-849c7597698f

📥 Commits

Reviewing files that changed from the base of the PR and between 1a5b944 and d152eb3.

📒 Files selected for processing (2)
  • docs/how-to/generate-scanner-safe-k8s-secret.md
  • xtask/src/main.rs
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch xtask/scanner-safe-reference-check

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@EffortlessSteven EffortlessSteven merged commit d6db725 into main May 11, 2026
6 checks passed
EffortlessSteven added a commit that referenced this pull request May 11, 2026
Why patch mode
==============

v0.7.0 ran the full minor-release evidence lane, including expensive
gates like `cargo xtask mutants-nightly --scope public`, the broad
perf suite, and new product profile proofs. That investment is right
for a minor release that changes the user-facing platform.

v0.7.1 is different: it hardens release tooling and the user install
path without changing public behavior. It only needs confidence that
release-system gates and the user path still work. Running the full
minor lane on every patch wastes CI time and dulls the signal of the
minor lane itself.

What the patch lane runs
========================

`cargo xtask release-evidence --version <V> --patch [--summary]`
selects `release_evidence_steps_patch()` which runs 10 focused steps:

1.  `public-surface`            — re-confirm exported API
2.  `check-file-policy`         — non-Rust file allowlist
3.  `publish-preflight`         — metadata + cargo package --no-verify
4.  `publish-check`             — ordered publish dry-runs (includes
                                  topology guard from #572 and
                                  publish=false dep guard from #578)
5.  `scanner-safe-reference --check` (#577) — scanner reference paths
6.  `cratesio-smoke --path . --skip-install-cli` (#580) — user path
7.  `docs-sync --check`         — docs are in sync with code
8.  `no-blob`                   — no secret-shaped blobs in test paths
9.  `examples-smoke`            — examples still compile / run
10. `impacted-evidence --base origin/main` — change-impact routing

What it deliberately omits
==========================

- `mutants-nightly --scope public` — full nightly mutation runs
  against `main` on schedule; targeted mutation already runs through
  `cargo xtask pr` when `impacted-evidence` flags the change.
- broad perf suite (`perf --compare`) — patch releases should not
  shift baselines; if they do, escalate to the minor lane.
- new product profile proofs (scanner-safe bundle, OIDC contract
  pack) — patch releases are not the venue to re-prove new profiles.

Receipt shape
=============

`ReleaseEvidenceReceipt` gains a `lane_mode` field (`"minor"` or
`"patch"`). The existing `lane: "release-evidence"` string is
preserved for backward compatibility with anything consuming the
shape; only the new field distinguishes the two lanes. The
release-evidence markdown gains a `Mode:` line; the summary opens
with a "Patch-mode evidence lane" callout when patch is set and
swaps the v0.7.0-specific gate table for the patch step list.

Tests
=====

- `release_evidence_patch_step_list_excludes_mutants_nightly`
- `release_evidence_minor_step_list_includes_mutants_nightly`
- `release_evidence_patch_step_list_includes_scanner_safe_reference`
- `release_evidence_patch_step_list_includes_cratesio_smoke`
- `release_evidence_patch_receipt_records_patch_mode`
- `release_evidence_patch_summary_announces_patch_lane`

Minor-lane callers (`release-evidence --version X [--summary]` with
no `--patch`) are unchanged in behavior and continue to run the
full 15-step lane.
@EffortlessSteven EffortlessSteven mentioned this pull request May 11, 2026
4 tasks
EffortlessSteven added a commit that referenced this pull request May 11, 2026
Bump workspace version 0.7.0 -> 0.7.1 and seed the CHANGELOG entry
describing the release-hardening additions (#572, #577, #578, #580,
#581) and docs (#571, #579). No new product surface; no shim removal;
no profile work. Patch evidence lane passes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant