Skip to content

Rehearsal registry: strengthen preflight from dry-run to actual publish + install proof #97

Description

@EffortlessSteven

Problem

Workspace dry-run (via cargo publish --dry-run) proves Cargo can package each crate, but does not prove artifacts will resolve and install correctly from a registry. The v0.3.0-rc.1 first-publish exposed this gap: preflight correctly could not prove finishability for all 12 new crates (since they lacked owner records), but it also cannot catch the scenario where locally-resolvable workspace-path dependencies fail to resolve through a registry index after publish (the publish train fixes resolution as it goes, but preflight has no visibility).

Current preflight

File: crates/shipper/src/engine/mod.rs:74 (run_preflight())
Lines: 93-293

What gets checked:

  • Git cleanliness (engine/mod.rs:95)
  • Registry reachability (engine/mod.rs:99)
  • Workspace-level or per-package dry-run via cargo publish --dry-run (engine/mod.rs:126–191)
  • Per-crate version-not-taken check (engine/mod.rs:204)
  • Per-crate new-crate detection (engine/mod.rs:205)
  • Per-crate ownership verification (best-effort or strict; engine/mod.rs:230–256)

Finishability determination (engine/mod.rs:283–293):

  • Failed if any package dry-run failed
  • NotProven if any ownership unverified (new crates have no owners endpoint)
  • Proven if all checks passed (but only for existing crates; new crates are forced to NotProven)

What doesn't get checked: Whether published artifacts will actually resolve and install from a registry with real Cargo.lock resolution via workspace-path dependencies transitively resolved through registry index entries.

Proposed two-phase preflight

Phase 1 — validate (existing, unchanged)

  • Git cleanliness
  • Registry reachability
  • Cargo dry-run (workspace or per-package)
  • Version-not-taken check
  • New-crate detection
  • Ownership verification (best-effort or strict)

Phase 2 — rehearse (new)

Execute the following sequentially:

  1. Publish the same packaged tarballs (already generated in Phase 1) to a configured alternate registry
  2. Verify presence on the rehearsal registry (API or sparse index, same as live verification)
  3. Install/build from the rehearsal registry: run cargo install --registry <rehearsal> on the crates, or cargo build on a consumer workspace, to verify end-to-end resolution
  4. Record proof in a new event type RehearsalComplete { passed: bool, summary: String } and persist to .shipper/events.jsonl

Configuration surface

TOML config ([rehearsal] section in .shipper.toml):

[rehearsal]
enabled = false                    # opt-in initially, opt-out once stable
registry = "alternate-registry"    # cargo registry name from ~/.cargo/config.toml or Shipper [registries] section

CLI flags:

  • --rehearsal-registry <name> — override configured rehearsal target (or enable rehearsal with explicit registry)
  • --skip-rehearsal — disable rehearsal for this run (with loud warning if policy would require it)

Defaults: Rehearsal disabled for safety; opt-in.

Hard gate

Live crates.io dispatch refuses to publish (from run_publish()) unless:

  • Rehearsal succeeded for the same plan_id (stored alongside preflight report), OR
  • Operator explicitly provides --skip-rehearsal (logged as a loud warning, captured in events)

This binding ensures rehearsal artifacts are tied to their corresponding plan and prevents mismatches.

Why this matters

Closes the gap from v0.3.0-rc.1 retrospective:
Preflight transitions from "we believe this should work based on local dry-run" to "we proved it works against a registry-shaped target with real resolution semantics."

Workspace-path dependency resolution through a registry index is the exact scenario that local cargo package --list cannot validate ahead of live publish (and the scenario that killed the first-publish attempt before the backfill train caught up).

Implementation considerations

Rehearsal target deployment (link for operator reference, do not prescribe):

  • In CI: spin up a transient kellnr instance (lightweight, supports Cargo protocol)
  • Self-hosted: Cloudsmith / Cargo-protocol mirror in rehearsal mode
  • Git-based registries in scratch mode (slower but zero infra)
  • Cargo registries documentation

Why NOT source replacement:
Source replacement ([source] in .cargo/config.toml) is for equivalent mirrors of live registries, not for rehearsal targets. Rehearsal requires a real registry (with real index entries and visibility endpoints) to prove resolution will work on live.

Plan ID binding:
Rehearsal artifacts are tied to the plan_id generated in plan.rs. Live dispatch checks: stored rehearsal_proof matches plan_id before allowing crates.io publish.

State schema:
Extend ExecutionState (or create a sibling RehearsalState) to track: plan_id, rehearsal_registry, rehearsal_start, rehearsal_complete, rehearsal_summary.

Acceptance criteria

  • [rehearsal] config section parses and validates correctly
  • --rehearsal-registry and --skip-rehearsal CLI flags work end-to-end
  • Rehearsal phase publishes tarballs to alternate registry without mutating plan or crates.io
  • Rehearsal phase verifies presence on alternate registry (API or index)
  • Rehearsal phase runs cargo operations (install or build) consuming published crates
  • Rehearsal results are logged to events and bound to plan_id
  • Live dispatch gate refuses to publish without rehearsal success (or --skip-rehearsal)
  • Loud warning is logged when --skip-rehearsal is used
  • Tests cover: rehearsal success path, alternate registry reachability, workspace-path dependency resolution
  • Documentation added to CLAUDE.md with example .shipper.toml config

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requesthelp wantedExtra attention is needed

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions