Skip to content

Ambiguous publish reconciliation: state machine against registry truth, no blind retry #99

Description

@EffortlessSteven

Problem

Cargo's documented behavior is: cargo publish uploads the crate to the registry FIRST, then polls the index for visibility, and the index poll can time out without affecting the upload. This means "the cargo publish command failed" and "the crate actually published" can BOTH be true at the same time.

Why this matters more for Shipper

Shipper exists to be safer than naive cargo publish loops. Ambiguity is the highest-stakes case — Shipper's entire value proposition hinges on being able to reconcile these corner cases deterministically.

Current behavior

ErrorClass::Ambiguous definition: /H/Code/Rust/shipper/crates/shipper-types/src/lib.rs:937

  • Enum variant exists: Ambiguous
  • Classification happens in /H/Code/Rust/shipper/crates/shipper/src/runtime/execution/mod.rs:61-70 via classify_cargo_failure()
  • Current handling in /H/Code/Rust/shipper/crates/shipper/src/engine/parallel/publish.rs:328 treats Ambiguous identically to Retryable — blind retry with backoff.

Readiness verification exists (/H/Code/Rust/shipper/crates/shipper/src/engine/parallel/readiness.rs) with:

  • API check: reg.version_exists()
  • Sparse-index check: reg.fetch_sparse_index_file()
  • Both methods with configurable priority
  • Exponential backoff with jitter

But readiness verification only runs after a successful cargo publish. On Ambiguous cargo exit, the engine blindly retries without querying the registry first.

Proposed state machine

UploadedUnknown      ← initial state when cargo publish returns ambiguously
Reconciling          ← actively probing registry for ground truth
Published            ← reconciled to true (via sparse index OR API)
NotPublished         ← reconciled to false (registry doesn't know about it)
StillUnknown         ← couldn't reconcile within timeout (operator decision required)

Transitions:

  1. Cargo exits with Ambiguous error → UploadedUnknown
  2. Query sparse index + API in parallel → Reconciling
  3. If found → Published (record evidence, skip future attempts, mark state as Published)
  4. If not found → NotPublished (safe to retry — upload truly failed)
  5. If timeout/connection-drop during reconciliation → StillUnknown (operator-actionable, not auto-recoverable)

Hard rule

After timeout, cancellation, connection drop, or unknown publish output: NEVER blind-retry. Always reconcile first against:

  1. Sparse index: GET https://index.crates.io/<prefix>/<name>
  2. Registry API: GET https://crates.io/api/v1/crates/<name>/<version>
  3. Resolve to one of: Published, NotPublished, StillUnknown

Only retry if NotPublished. Skip to Published if already found. Escalate StillUnknown to the operator.

Demotes Cargo stdout parsing to "hint"

Parsing Cargo's human-readable output stays as a fast-path classification hint, but never as the authoritative answer for safety-critical decisions. Ambiguity always falls through to registry-truth reconciliation rather than text-pattern guessing.

Implementation sketch

  • New struct PublishReconciliation in crate::runtime::execution or crate::engine::parallel to encapsulate the state machine
  • Integrate into /H/Code/Rust/shipper/crates/shipper/src/engine/parallel/publish.rs around line 328 (current Ambiguous branch)
  • Reuse existing is_version_visible_with_backoff() from readiness.rs but trigger it earlier (before retry sleep)
  • On PublishReconciliation::Published → skip remaining attempts, mark state as Published
  • On PublishReconciliation::NotPublished → allow retry (continue loop)
  • On PublishReconciliation::StillUnknown → mark state as Ambiguous, escalate to operator (no more auto-retry)

New events

  • publish_reconciling { package, method } — starting reconciliation (API or sparse index)
  • publish_reconciled { package, outcome, evidence } — result of reconciliation (Published/NotPublished/StillUnknown with proof)

Acceptance criteria

  • Ambiguous cargo exits trigger immediate registry reconciliation (not blind retry)
  • Sparse index check runs in parallel with API check (or prefer_index honored)
  • Reconciliation result persists to state.json (resumable)
  • StillUnknown is operator-visible in structured events, not a silent retry loop
  • Tests: reconciliation correctly handles all three outcomes (Published, NotPublished, StillUnknown)
  • Tests: reconciliation respects existing readiness_config (method, prefer_index, timeouts)
  • Tests: crate marked Published after reconciliation does not attempt re-upload on resume
  • Cargo stdout still used for fast-path classification, but not for safety-critical final decisions

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions