test(reconcile): BDD scenario for Ambiguous → StillUnknown (#99 follow-on) by EffortlessSteven · Pull Request #144 · EffortlessMetrics/shipper

EffortlessSteven · 2026-04-18T22:15:07Z

Summary

Adds the third reconciliation BDD scenario called out in Ambiguous publish reconciliation: state machine against registry truth, no blind retry #99's acceptance criteria: Ambiguous → StillUnknown. The existing suite covered Published and NotPublished; this closes the gap.
No product-code change. Reconciliation behavior was already implemented in PR feat(reconcile): close ambiguous publish outcomes against registry truth (#99) #111 and PR feat(reconcile): resume-path reconciliation for Ambiguous packages (#99 follow-on) #115; this test pins the operator-facing contract.

What the test exercises

Registry returns 500 on every query → version_exists bails → reconcile_ambiguous_upload returns StillUnknown.
Verifies the publish path:
- Returns Err with "reconciliation inconclusive".
- Marks the package PackageState::Ambiguous so a subsequent resume enters the reconcile block, not a silent retry.
- Invokes cargo exactly once — hardcodes the "no blind-retry after StillUnknown" invariant.
- Persists PublishReconciling + PublishReconciled { outcome: StillUnknown } events for operator visibility.

Test plan

cargo test -p shipper-core --lib reconcile_bdd_ambiguous_resolves_to_still_unknown passes
Full reconcile_bdd_* suite (4 tests) passes
cargo clippy -p shipper-core --all-targets --all-features -- -D warnings clean
cargo fmt -p shipper-core -- --check clean

… follow-on) Completes the third acceptance-criteria BDD scenario for #99. The existing suite already covered Published (registry confirms upload landed) and NotPublished (registry confirms upload didn't land); this adds the operator-actionable StillUnknown case where reconciliation itself can't reach a verdict because the registry is unhealthy. Setup drives every registry query on /api/v1/crates/demo/0.1.0 to 500, so version_exists bails with Err for non-200/404 statuses. reconcile_ambiguous_upload translates that Err into ReconciliationOutcome::StillUnknown. Asserts: - publish returns Err with "reconciliation inconclusive" message - package state is PackageState::Ambiguous (so resume's reconcile block will fire rather than a silent retry) - exactly one cargo invocation — StillUnknown must not blind-retry - PublishReconciling + PublishReconciled{ StillUnknown } events are persisted to events.jsonl for operator visibility Closes the last open AC on #99 acceptance list; reconciliation behavior itself was already implemented in PR #111 and PR #115.

coderabbitai · 2026-04-18T22:15:21Z

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: bd3255aa-1a0e-4eb7-8b07-12c86a1a41a7

📥 Commits

Reviewing files that changed from the base of the PR and between 5322a89 and bc0f3c5.

📒 Files selected for processing (1)

crates/shipper-core/src/engine/parallel/tests.rs

Summary by CodeRabbit

Tests
- Added a new test to verify package publication correctly handles ambiguous reconciliation scenarios when registry endpoints fail, ensuring proper state transitions, appropriate retry limits, and accurate event recording.

Walkthrough

A new serialized reconciliation test reconcile_bdd_ambiguous_resolves_to_still_unknown is added to verify behavior when cargo fails ambiguously (non-zero exit with empty output) and the registry returns 500 responses. The test validates error handling, state persistence, and that cargo is invoked exactly once.

Changes

Cohort / File(s)	Summary
Reconciliation Test Coverage `crates/shipper-core/src/engine/parallel/tests.rs`	Added new BDD test validating the `StillUnknown` reconciliation outcome path, including mocked 500 registry responses, cargo failure configuration, package state transitions to `Ambiguous`, single invocation verification, and event persistence assertions.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Poem

🐰✨ Another test case hops into the warren!
When ambiguity reigns and servers say "nope,"
We check the right paths, the state persists true,
One cargo call only—no blind retries brew! 🎯

🚥 Pre-merge checks | ✅ 3

✅ Passed checks (3 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly and specifically describes the main change: adding a BDD test scenario for the Ambiguous → StillUnknown reconciliation path as a follow-up to issue `#99`.
Description check	✅ Passed	The description is directly related to the changeset, providing context about the test being added, what it exercises, and the test plan showing verification steps.
Docstring Coverage	✅ Passed	Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

📝 Generate docstrings

Create stacked PR
Commit on current branch

🧪 Generate unit tests (beta)

Create PR with unit tests
Commit unit tests in branch feat/99-bdd-still-unknown

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

gemini-code-assist

Code Review

This pull request adds a new integration test, reconcile_bdd_ambiguous_resolves_to_still_unknown, to the shipper-core engine. The test verifies the system's behavior when a cargo publish fails ambiguously while the registry is unhealthy (returning 500 errors). It ensures that the package is correctly transitioned to an Ambiguous state, that the process halts with an error without blind retries, and that the appropriate reconciliation events are recorded in the event log. I have no feedback to provide.

codecov · 2026-04-18T22:45:29Z

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

gemini-code-assist Bot reviewed Apr 18, 2026

View reviewed changes

EffortlessSteven merged commit 4898df2 into main Apr 18, 2026
21 checks passed

EffortlessSteven deleted the feat/99-bdd-still-unknown branch April 18, 2026 23:43

This was referenced Apr 18, 2026

Ambiguous publish reconciliation: state machine against registry truth, no blind retry #99

Closed

Competency: Reconcile — gap audit and roadmap #102

Closed

Roadmap: post-v0.3.0-rc.1 product scorecard across nine competencies #109

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

test(reconcile): BDD scenario for Ambiguous → StillUnknown (#99 follow-on)#144

test(reconcile): BDD scenario for Ambiguous → StillUnknown (#99 follow-on)#144
EffortlessSteven merged 1 commit into
mainfrom
feat/99-bdd-still-unknown

EffortlessSteven commented Apr 18, 2026

Uh oh!

coderabbitai Bot commented Apr 18, 2026 •

edited

Loading

Uh oh!

gemini-code-assist Bot left a comment

Uh oh!

codecov Bot commented Apr 18, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Conversation

EffortlessSteven commented Apr 18, 2026

Summary

What the test exercises

Test plan

Uh oh!

coderabbitai Bot commented Apr 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary by CodeRabbit

Walkthrough

Changes

Estimated code review effort

Poem

Uh oh!

gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

codecov Bot commented Apr 18, 2026

Codecov Report

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

coderabbitai Bot commented Apr 18, 2026 •

edited

Loading