feat(rehearse): smoke-install closes #97 Prove tier 2 (#97 PR 4)#137
Conversation
Adds the final piece of #97's acceptance: \`cargo install --registry <rehearsal> <crate>\` after rehearsal publishes succeed, as end-to-end proof that the crate actually resolves and installs via the registry index — the scenario that workspace-path dependencies defeat and that killed the rc.1 first-publish. ## What's new - **EventType variants**: \`RehearsalSmokeCheckStarted\`, \`RehearsalSmokeCheckSucceeded\`, \`RehearsalSmokeCheckFailed\`. - **RuntimeOptions**: \`rehearsal_smoke_install: Option<String>\`. - **CliOverrides + CLI flag**: \`--smoke-install <CRATE>\` (global). Crate must be in the rehearsal plan and have a \`[[bin]]\` target; library-only crates can't be smoke-installed directly (consumer- workspace build variant is a follow-on). - **\`ops::cargo::cargo_install_smoke\`**: wrapper around \`cargo install --registry <name> <crate> --version <v> --root <tmp> --force --locked\`. - **engine::run_rehearsal** post-publish step: if all packages passed AND \`--smoke-install\` names an in-plan crate, install it from the rehearsal registry. Install failure fails the whole rehearsal (sets \`first_failure\` + \`RehearsalComplete { passed: false }\`). Named-but-not-in-plan: warn only; don't fail rehearsal over a typo. ## Tests - \`run_rehearsal_smoke_install_happy_path_emits_succeeded_event\` — flag named an in-plan crate, fake cargo returns 0, assert \`rehearsal_smoke_check_started\` + \`_succeeded\` events. - \`run_rehearsal_smoke_install_missing_target_warns_without_failing\` — flag named a crate not in the plan, assert reporter warn + rehearsal still passes. 19 rehearsal+gate tests pass (up from 17). ## Closes #97 With this PR, #97's issue checklist is 100% fulfilled: - [x] Publish packaged tarballs to alt registry (#127) - [x] Verify presence on rehearsal registry (#127) - [x] Install/build from rehearsal registry (**this PR**) - [x] Record proof in RehearsalComplete event (#127) - [x] Hard gate binding on plan_id (#133) The \`#97\` issue can be closed once this merges.
|
Important Review skippedReview was skipped due to path filters ⛔ Files ignored due to path filters (21)
CodeRabbit blocks several paths by default. You can override this behavior by explicitly including those paths in the path filters. For example, including ⚙️ Run configurationConfiguration used: Organization UI Review profile: ASSERTIVE Plan: Pro Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
WalkthroughAdded a new optional Changes
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~28 minutes Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Code Review
This pull request implements a "smoke-install" feature for the rehearsal process, allowing users to verify crate resolution and installation from a rehearsal registry via a new CLI flag. The implementation spans CLI arguments, configuration, event logging, and engine logic. Feedback points out the use of unstable Rust syntax, recommends better error handling for directory removal, and identifies a bug in registry name comparison.
| if first_failure.is_none() | ||
| && let Some(ref smoke_name) = opts.rehearsal_smoke_install | ||
| { |
There was a problem hiding this comment.
The let_chains syntax (&& let) is an unstable Rust feature. Unless the project is strictly nightly-only and has enabled the #![feature(let_chains)] attribute, this will cause a compilation error on stable and beta channels. You can achieve the same result using a tuple match which is stable and idiomatic.
if let (None, Some(smoke_name)) = (&first_failure, &opts.rehearsal_smoke_install) {| event_log.clear(); | ||
|
|
||
| let install_root = state_dir.join("smoke-install"); | ||
| let _ = std::fs::remove_dir_all(&install_root); |
There was a problem hiding this comment.
Ignoring errors from remove_dir_all can hide underlying filesystem issues (such as permission denied) that will cause the subsequent cargo install to fail in a confusing way. It's better to check for existence and handle errors properly. Since run_rehearsal returns a Result, you can use the ? operator to propagate unexpected errors.
| let _ = std::fs::remove_dir_all(&install_root); | |
| if install_root.exists() { | |
| std::fs::remove_dir_all(&install_root)?; | |
| } |
| if !registry_name.trim().is_empty() && registry_name != "crates-io" { | ||
| args.push("--registry"); | ||
| args.push(registry_name); | ||
| } |
There was a problem hiding this comment.
There are two issues here:
- The comparison
registry_name != "crates-io"is performed on the untrimmed string, while the emptiness check usestrim(). Ifregistry_namecontains leading or trailing whitespace (e.g.," crates-io "), it will incorrectly pass the check and add the--registryflag. "crates-io"is a magic string. It should ideally be a constant to ensure consistency across the codebase.
I suggest trimming the name once and using the trimmed version for both the check and the argument.
| if !registry_name.trim().is_empty() && registry_name != "crates-io" { | |
| args.push("--registry"); | |
| args.push(registry_name); | |
| } | |
| let registry_name = registry_name.trim(); | |
| if !registry_name.is_empty() && registry_name != "crates-io" { | |
| args.push("--registry"); | |
| args.push(registry_name); | |
| } |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 6230d8cf4e
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| first_failure = Some(format!( | ||
| "smoke-install of {smoke_name}@{} failed: cargo exit {}", | ||
| smoke_pkg.version, out.exit_code | ||
| )); |
There was a problem hiding this comment.
Track smoke-check failures without package increment
Setting first_failure in the smoke-install branch reuses the package-failure accounting path, so when install fails after all publishes, the completion summary/receipt report packages_published + 1 attempted (for example 2/1). This produces invalid package counts in RehearsalComplete/rehearsal.json, which can mislead operators and any downstream tooling that reads these fields; smoke-check failure should be tracked separately from per-package publish failure accounting.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Actionable comments posted: 6
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
crates/shipper-config/tests/config_runtime_proptest.rs (1)
271-379: 🧹 Nitpick | 🔵 TrivialExercise
Somesmoke-install values in the proptest generator.Hard-coding
Nonemeans these property tests cannot catch regressions where the new CLI override is dropped before reaching runtime options.🧪 Proposed test hardening
( any::<bool>(), any::<bool>(), any::<bool>(), proptest::option::of(1usize..32), proptest::option::of(arb_nonzero_duration_ms(1_000, 600_000)), proptest::option::of(arb_url()), proptest::option::of(arb_name()), any::<bool>(), proptest::option::of(arb_name()), + proptest::option::of(arb_name()), ), @@ webhook_secret, encrypt, encrypt_passphrase, + rehearsal_smoke_install, ), )| { @@ resume_from: None, rehearsal_registry: None, skip_rehearsal: false, - rehearsal_smoke_install: None, + rehearsal_smoke_install, } },Add a focused property as well:
proptest! { #[test] fn cli_smoke_install_always_reaches_runtime( cfg in arb_shipper_config(), smoke in arb_name(), ) { let rt = into_runtime_options(cfg.build_runtime_options(CliOverrides { rehearsal_smoke_install: Some(smoke.clone()), ..Default::default() })); prop_assert_eq!(rt.rehearsal_smoke_install.as_deref(), Some(smoke.as_str())); } }🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@crates/shipper-config/tests/config_runtime_proptest.rs` around lines 271 - 379, The proptest generator arb_cli_overrides builds CliOverrides with rehearsal_smoke_install always None so tests never exercise Some(...) values; update arb_cli_overrides to include a proptest::option::of(arb_name()) (or similar arb for smoke-install) in the tuple used to construct CliOverrides.rehearsal_smoke_install and adjust the prop_map to assign that generated value instead of hard-coding None, and also add the focused property test (proptest! test function) that builds runtime options from a CliOverrides with rehearsal_smoke_install set and asserts that into_runtime_options(cfg.build_runtime_options(...)).rehearsal_smoke_install matches the original value to ensure the override is preserved.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@crates/shipper-config/src/runtime/mod.rs`:
- Around line 121-123: Add a focused unit test that asserts the positive mapping
of the smoke-install flag: construct a fixture/input that enables
"smoke-install" (so the mapper will set a value) call the mapping function that
produces the runtime struct, and assert that the resulting struct’s
rehearsal_smoke_install field is Some(true) (instead of always None). Locate the
code path that produces the struct with the rehearsal_smoke_install field and
add the test to cover the positive case so the suite would fail if the mapper
drops the Some value.
In `@crates/shipper-types/src/lib.rs`:
- Around line 1565-1584: The test suite currently iterates over a fixed range
(0u8..18) and doesn't exercise the new EventType variants
RehearsalSmokeCheckStarted, RehearsalSmokeCheckSucceeded, and
RehearsalSmokeCheckFailed; add new serde roundtrip/snapshot unit tests that
construct each of these three variants, serialize to JSON (ensuring snake_case
tags/field names), deserialize back and assert equality, and update the
snapshots to lock their JSON representations; reference the enum variants
RehearsalSmokeCheckStarted/RehearsalSmokeCheckSucceeded/RehearsalSmokeCheckFailed
and adjust the “all variants” test or add separate tests to cover these new
cases.
- Around line 596-602: Update the documentation for the field
rehearsal_smoke_install to accurately state the runtime contract: if the named
crate is absent from the publish plan it is skipped and only a warning is
emitted (does not fail the run), but if the crate is present in the plan yet
lacks an installable [[bin]] target or otherwise cannot be installed the smoke
install should fail the check. Mention that None disables the smoke install as
already stated and keep the note that the named crate must be in the plan only
as a desirable condition (with the new explicit behavior for the missing-case).
In `@crates/shipper/src/cli/mod.rs`:
- Around line 197-208: Add a validator to reject empty smoke-install crate names
by implementing a small parser function (e.g., fn parse_non_empty_crate_name(s:
&str) -> Result<String, String>) near the other parse helpers and attach it to
the rehearsal_smoke_install arg so that --smoke-install "" fails at parse time;
the parser should trim the input and return Err("crate name cannot be
empty".to_string()) for empty strings and Ok(trimmed.to_string()) otherwise, and
the rehearsal_smoke_install field (the #[arg(...)] on rehearsal_smoke_install)
should reference this parser via value_parser/value_parser =
parse_non_empty_crate_name so empty names are rejected during argument parsing.
In `@crates/shipper/src/engine/mod.rs`:
- Around line 7102-7135: The test currently only asserts emitted events; also
assert the fake cargo invocation to catch regressions in cargo_install_smoke
wiring: after run_rehearsal(&ws, &opts, &mut reporter) read the recorded fake
cargo args produced by fake_program_env_vars (or the mechanism used by your test
harness to capture program invocations) and assert it contains the expected
subcommand and parameters (e.g. "install", the rehearsal registry name
"rehearsal" or "--registry rehearsal" as your implementation emits, the crate
"demo" and the expected version if applicable). Locate the capture point created
by fake_program_env_vars and add assertions near the existing event checks
(after outcome) to validate the exact cargo invocation used by
cargo_install_smoke so wrong subcommand, registry, crate or version will fail
the test.
---
Outside diff comments:
In `@crates/shipper-config/tests/config_runtime_proptest.rs`:
- Around line 271-379: The proptest generator arb_cli_overrides builds
CliOverrides with rehearsal_smoke_install always None so tests never exercise
Some(...) values; update arb_cli_overrides to include a
proptest::option::of(arb_name()) (or similar arb for smoke-install) in the tuple
used to construct CliOverrides.rehearsal_smoke_install and adjust the prop_map
to assign that generated value instead of hard-coding None, and also add the
focused property test (proptest! test function) that builds runtime options from
a CliOverrides with rehearsal_smoke_install set and asserts that
into_runtime_options(cfg.build_runtime_options(...)).rehearsal_smoke_install
matches the original value to ensure the override is preserved.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro
Run ID: 644b7b49-0eef-4755-b431-45d14e457873
⛔ Files ignored due to path filters (33)
crates/shipper-cli/tests/snapshots/cli_snapshots__ci_help.snapis excluded by!**/*.snapcrates/shipper-cli/tests/snapshots/cli_snapshots__clean_help.snapis excluded by!**/*.snapcrates/shipper-cli/tests/snapshots/cli_snapshots__config_help.snapis excluded by!**/*.snapcrates/shipper-cli/tests/snapshots/cli_snapshots__doctor_help.snapis excluded by!**/*.snapcrates/shipper-cli/tests/snapshots/cli_snapshots__help_text.snapis excluded by!**/*.snapcrates/shipper-cli/tests/snapshots/cli_snapshots__no_subcommand_error.snapis excluded by!**/*.snapcrates/shipper-cli/tests/snapshots/cli_snapshots__plan_help.snapis excluded by!**/*.snapcrates/shipper-cli/tests/snapshots/cli_snapshots__preflight_help.snapis excluded by!**/*.snapcrates/shipper-cli/tests/snapshots/cli_snapshots__publish_help.snapis excluded by!**/*.snapcrates/shipper-cli/tests/snapshots/cli_snapshots__resume_help.snapis excluded by!**/*.snapcrates/shipper-cli/tests/snapshots/cli_snapshots__status_help.snapis excluded by!**/*.snapcrates/shipper-cli/tests/snapshots/e2e_expanded__error_missing_ci_subcommand.snapis excluded by!**/*.snapcrates/shipper-cli/tests/snapshots/e2e_expanded__error_missing_config_subcommand.snapis excluded by!**/*.snapcrates/shipper-cli/tests/snapshots/e2e_expanded__help_ci.snapis excluded by!**/*.snapcrates/shipper-cli/tests/snapshots/e2e_expanded__help_ci_github_actions.snapis excluded by!**/*.snapcrates/shipper-cli/tests/snapshots/e2e_expanded__help_ci_gitlab.snapis excluded by!**/*.snapcrates/shipper-cli/tests/snapshots/e2e_expanded__help_clean.snapis excluded by!**/*.snapcrates/shipper-cli/tests/snapshots/e2e_expanded__help_completion.snapis excluded by!**/*.snapcrates/shipper-cli/tests/snapshots/e2e_expanded__help_config.snapis excluded by!**/*.snapcrates/shipper-cli/tests/snapshots/e2e_expanded__help_config_init.snapis excluded by!**/*.snapcrates/shipper-cli/tests/snapshots/e2e_expanded__help_config_validate.snapis excluded by!**/*.snapcrates/shipper-cli/tests/snapshots/e2e_expanded__help_doctor.snapis excluded by!**/*.snapcrates/shipper-cli/tests/snapshots/e2e_expanded__help_fix_forward.snapis excluded by!**/*.snapcrates/shipper-cli/tests/snapshots/e2e_expanded__help_inspect_events.snapis excluded by!**/*.snapcrates/shipper-cli/tests/snapshots/e2e_expanded__help_inspect_receipt.snapis excluded by!**/*.snapcrates/shipper-cli/tests/snapshots/e2e_expanded__help_plan.snapis excluded by!**/*.snapcrates/shipper-cli/tests/snapshots/e2e_expanded__help_plan_yank.snapis excluded by!**/*.snapcrates/shipper-cli/tests/snapshots/e2e_expanded__help_preflight.snapis excluded by!**/*.snapcrates/shipper-cli/tests/snapshots/e2e_expanded__help_publish.snapis excluded by!**/*.snapcrates/shipper-cli/tests/snapshots/e2e_expanded__help_resume.snapis excluded by!**/*.snapcrates/shipper-cli/tests/snapshots/e2e_expanded__help_root.snapis excluded by!**/*.snapcrates/shipper-cli/tests/snapshots/e2e_expanded__help_status.snapis excluded by!**/*.snapcrates/shipper-cli/tests/snapshots/e2e_expanded__help_yank.snapis excluded by!**/*.snap
📒 Files selected for processing (10)
crates/shipper-config/src/lib.rscrates/shipper-config/src/runtime/mod.rscrates/shipper-config/tests/config_runtime_bdd.rscrates/shipper-config/tests/config_runtime_proptest.rscrates/shipper-types/src/lib.rscrates/shipper/src/cli/mod.rscrates/shipper/src/engine/mod.rscrates/shipper/src/engine/parallel/tests.rscrates/shipper/src/ops/cargo/mod.rscrates/shipper/src/runtime/policy/mod.rs
| rehearsal_registry: None, | ||
| rehearsal_skip: false, | ||
| rehearsal_smoke_install: None, |
There was a problem hiding this comment.
🧹 Nitpick | 🔵 Trivial
Add a positive mapping test for smoke-install.
All updated fixtures use None, so the suite would still pass if line 44 accidentally dropped a Some value.
🧪 Proposed focused assertion
#[test]
+ fn maps_rehearsal_smoke_install() {
+ let mut opts = sample_runtime_options();
+ opts.rehearsal_smoke_install = Some("shipper".to_string());
+
+ let converted = into_runtime_options(opts);
+
+ assert_eq!(
+ converted.rehearsal_smoke_install.as_deref(),
+ Some("shipper")
+ );
+ }
+
+ #[test]
fn maps_simple_discriminants() {🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@crates/shipper-config/src/runtime/mod.rs` around lines 121 - 123, Add a
focused unit test that asserts the positive mapping of the smoke-install flag:
construct a fixture/input that enables "smoke-install" (so the mapper will set a
value) call the mapping function that produces the runtime struct, and assert
that the resulting struct’s rehearsal_smoke_install field is Some(true) (instead
of always None). Locate the code path that produces the struct with the
rehearsal_smoke_install field and add the test to cover the positive case so the
suite would fail if the mapper drops the Some value.
| /// Crate name to install via `cargo install --registry <rehearsal>` | ||
| /// after all rehearsal publishes succeed (#97 PR 4). This is the | ||
| /// install/smoke check that proves end-to-end registry-index | ||
| /// resolution — the scenario that killed the rc.1 first-publish. | ||
| /// `None` means no smoke install (opt-in). The named crate must | ||
| /// exist in the plan AND have a `[[bin]]` target. | ||
| pub rehearsal_smoke_install: Option<String>, |
There was a problem hiding this comment.
Clarify missing-plan behavior in the option docs.
Line 600 says the named crate “must exist in the plan”, but the PR behavior says a missing target only warns and does not fail. Document the actual contract: missing from plan skips/warns; present-but-not-installable fails the smoke check.
📝 Proposed wording
- /// `None` means no smoke install (opt-in). The named crate must
- /// exist in the plan AND have a `[[bin]]` target.
+ /// `None` means no smoke install (opt-in). If the named crate is not
+ /// in the plan, rehearsal warns and skips the smoke check; if it is in
+ /// the plan, it must have an installable binary target or the smoke
+ /// check fails.📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| /// Crate name to install via `cargo install --registry <rehearsal>` | |
| /// after all rehearsal publishes succeed (#97 PR 4). This is the | |
| /// install/smoke check that proves end-to-end registry-index | |
| /// resolution — the scenario that killed the rc.1 first-publish. | |
| /// `None` means no smoke install (opt-in). The named crate must | |
| /// exist in the plan AND have a `[[bin]]` target. | |
| pub rehearsal_smoke_install: Option<String>, | |
| /// Crate name to install via `cargo install --registry <rehearsal>` | |
| /// after all rehearsal publishes succeed (`#97` PR 4). This is the | |
| /// install/smoke check that proves end-to-end registry-index | |
| /// resolution — the scenario that killed the rc.1 first-publish. | |
| /// `None` means no smoke install (opt-in). If the named crate is not | |
| /// in the plan, rehearsal warns and skips the smoke check; if it is in | |
| /// the plan, it must have an installable binary target or the smoke | |
| /// check fails. | |
| pub rehearsal_smoke_install: Option<String>, |
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@crates/shipper-types/src/lib.rs` around lines 596 - 602, Update the
documentation for the field rehearsal_smoke_install to accurately state the
runtime contract: if the named crate is absent from the publish plan it is
skipped and only a warning is emitted (does not fail the run), but if the crate
is present in the plan yet lacks an installable [[bin]] target or otherwise
cannot be installed the smoke install should fail the check. Mention that None
disables the smoke install as already stated and keep the note that the named
crate must be in the plan only as a desirable condition (with the new explicit
behavior for the missing-case).
| // #97 PR 4 — install/smoke check. Opt-in post-publish step that runs | ||
| // `cargo install --registry <rehearsal> <crate>` to prove end-to-end | ||
| // registry-index resolution — the scenario that killed the rc.1 | ||
| // first-publish. Events bracket the check so an auditor replaying | ||
| // events.jsonl can see the proof (or failure) inline with publishes. | ||
| RehearsalSmokeCheckStarted { | ||
| name: String, | ||
| version: String, | ||
| registry: String, | ||
| }, | ||
| RehearsalSmokeCheckSucceeded { | ||
| name: String, | ||
| version: String, | ||
| duration_ms: u128, | ||
| }, | ||
| RehearsalSmokeCheckFailed { | ||
| name: String, | ||
| version: String, | ||
| message: String, | ||
| }, |
There was a problem hiding this comment.
Add shipper-types serialization coverage for the new event variants.
These EventType variants extend the JSON event stream, but the existing “all variants” tests still range over 0u8..18 and never construct the smoke-check events. Add explicit serde roundtrip/snapshot coverage for the three new variants so their snake_case tags and fields are locked.
🧪 Example coverage
+ #[test]
+ fn event_type_rehearsal_smoke_check_variants_roundtrip() {
+ let events = vec![
+ EventType::RehearsalSmokeCheckStarted {
+ name: "my-cli".to_string(),
+ version: "1.0.0".to_string(),
+ registry: "rehearsal".to_string(),
+ },
+ EventType::RehearsalSmokeCheckSucceeded {
+ name: "my-cli".to_string(),
+ version: "1.0.0".to_string(),
+ duration_ms: 1234,
+ },
+ EventType::RehearsalSmokeCheckFailed {
+ name: "my-cli".to_string(),
+ version: "1.0.0".to_string(),
+ message: "install failed".to_string(),
+ },
+ ];
+
+ for event in events {
+ let json = serde_json::to_string(&event).unwrap();
+ let parsed: EventType = serde_json::from_str(&json).unwrap();
+ assert_eq!(serde_json::to_string(&parsed).unwrap(), json);
+ }
+ }As per coding guidelines, crates/shipper-types/**/*.rs: “Update tests and related snapshots in the shipper-types crate when touching serialization or state formats”.
Also applies to: 4527-4547, 5322-5345
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@crates/shipper-types/src/lib.rs` around lines 1565 - 1584, The test suite
currently iterates over a fixed range (0u8..18) and doesn't exercise the new
EventType variants RehearsalSmokeCheckStarted, RehearsalSmokeCheckSucceeded, and
RehearsalSmokeCheckFailed; add new serde roundtrip/snapshot unit tests that
construct each of these three variants, serialize to JSON (ensuring snake_case
tags/field names), deserialize back and assert equality, and update the
snapshots to lock their JSON representations; reference the enum variants
RehearsalSmokeCheckStarted/RehearsalSmokeCheckSucceeded/RehearsalSmokeCheckFailed
and adjust the “all variants” test or add separate tests to cover these new
cases.
| /// Crate name to smoke-install after a successful rehearsal (#97 PR 4). | ||
| /// | ||
| /// Runs `cargo install --registry <rehearsal> <CRATE>` against the | ||
| /// rehearsal registry to prove the crate actually resolves and | ||
| /// installs end-to-end — the scenario that workspace-path | ||
| /// dependencies defeat and that killed the rc.1 first-publish. | ||
| /// | ||
| /// The named crate must be in the plan AND have a `[[bin]]` target. | ||
| /// Library-only crates cannot be smoke-installed directly; use a | ||
| /// consumer-workspace build instead (follow-on). | ||
| #[arg(long = "smoke-install", global = true, value_name = "CRATE")] | ||
| rehearsal_smoke_install: Option<String>, |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
# Verify the smoke-install flag is wired to a non-empty parser and covered in CLI parsing tests.
rg -n --type rust -C3 'smoke-install|rehearsal_smoke_install|parse_non_empty_crate_name' crates/shipper/src/cli/mod.rsRepository: EffortlessMetrics/shipper
Length of output: 1816
🏁 Script executed:
#!/bin/bash
# Find how rehearsal_smoke_install is used throughout the codebase
rg -n 'rehearsal_smoke_install' crates/shipper/src/ --type rust -A 5 -B 2Repository: EffortlessMetrics/shipper
Length of output: 8322
🏁 Script executed:
#!/bin/bash
# Search for any validation or processing of smoke_install values
rg -n 'smoke.install|smoke_install' crates/shipper/src/ --type rust -A 3 -B 1Repository: EffortlessMetrics/shipper
Length of output: 9804
🏁 Script executed:
#!/bin/bash
# Search for other value_parser examples in cli/mod.rs that validate non-empty or similar constraints
rg -n 'value_parser' crates/shipper/src/cli/mod.rs -B 2 -A 2Repository: EffortlessMetrics/shipper
Length of output: 245
🏁 Script executed:
#!/bin/bash
# Check if there are any tests that specifically test empty string handling for CLI arguments
rg -n '""' crates/shipper/src/cli/mod.rs | grep -i 'test\|mock' -A 5 -B 5Repository: EffortlessMetrics/shipper
Length of output: 51
Reject empty smoke-install crate names at argument parsing.
--smoke-install "" currently reaches runtime as Some(""); it will not match any crate in the plan and triggers a warning-only fallback, making the explicitly requested smoke test a silent no-op.
🛡️ Proposed parser
- #[arg(long = "smoke-install", global = true, value_name = "CRATE")]
+ #[arg(
+ long = "smoke-install",
+ global = true,
+ value_name = "CRATE",
+ value_parser = parse_non_empty_crate_name
+ )]
rehearsal_smoke_install: Option<String>,Add the parser near the other parse helpers:
fn parse_non_empty_crate_name(s: &str) -> std::result::Result<String, String> {
let trimmed = s.trim();
if trimmed.is_empty() {
Err("crate name cannot be empty".to_string())
} else {
Ok(trimmed.to_string())
}
}🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@crates/shipper/src/cli/mod.rs` around lines 197 - 208, Add a validator to
reject empty smoke-install crate names by implementing a small parser function
(e.g., fn parse_non_empty_crate_name(s: &str) -> Result<String, String>) near
the other parse helpers and attach it to the rehearsal_smoke_install arg so that
--smoke-install "" fails at parse time; the parser should trim the input and
return Err("crate name cannot be empty".to_string()) for empty strings and
Ok(trimmed.to_string()) otherwise, and the rehearsal_smoke_install field (the
#[arg(...)] on rehearsal_smoke_install) should reference this parser via
value_parser/value_parser = parse_non_empty_crate_name so empty names are
rejected during argument parsing.
| first_failure = Some(format!( | ||
| "smoke-install of {smoke_name}@{} failed: cargo exit {}", | ||
| smoke_pkg.version, out.exit_code | ||
| )); | ||
| } | ||
| event_log.write_to_file(&events_path)?; | ||
| event_log.clear(); | ||
| } | ||
| None => { | ||
| // Operator named a crate that isn't in the plan. Warn, | ||
| // don't fail — their intent is clear but the workspace | ||
| // shape disagrees, and failing the whole rehearsal over | ||
| // a typo would be overkill. | ||
| reporter.warn(&format!( | ||
| "smoke-install target '{smoke_name}' is not in the rehearsal plan; skipping. \ | ||
| Available crates: {}", | ||
| ws.plan | ||
| .packages | ||
| .iter() | ||
| .map(|p| p.name.as_str()) | ||
| .collect::<Vec<_>>() | ||
| .join(", ") | ||
| )); | ||
| } |
There was a problem hiding this comment.
Keep rehearsal receipts accurate for smoke-install outcomes.
A smoke-install failure happens after all package publishes, but the later failure summary and receipt still treat it as another package attempt. That can produce impossible audit data such as rehearsal stopped at 2/1 and packages_attempted = packages_published + 1. Also, the warn-only “target not in plan” path is not persisted in the durable summary, so the receipt can claim full success even though the requested smoke check was skipped.
🛠️ Proposed fix
let mut packages_published: usize = 0;
let mut first_failure: Option<String> = None;
+ let mut smoke_note: Option<String> = None;
@@
- first_failure = Some(format!(
+ let failure_summary = format!(
"smoke-install of {smoke_name}@{} failed: cargo exit {}",
smoke_pkg.version, out.exit_code
- ));
+ );
+ smoke_note = Some(failure_summary.clone());
+ first_failure = Some(failure_summary);
}
@@
- reporter.warn(&format!(
+ let msg = format!(
"smoke-install target '{smoke_name}' is not in the rehearsal plan; skipping. \
Available crates: {}",
ws.plan
.packages
.iter()
.map(|p| p.name.as_str())
.collect::<Vec<_>>()
.join(", ")
- ));
+ );
+ reporter.warn(&msg);
+ smoke_note = Some(msg);
}
}
}
let passed = first_failure.is_none();
+ let publish_failed = !passed && packages_published < ws.plan.packages.len();
let summary = if passed {
- format!("rehearsed {packages_published} packages against '{rehearsal_name}' successfully")
- } else {
+ let base =
+ format!("rehearsed {packages_published} packages against '{rehearsal_name}' successfully");
+ match smoke_note.as_deref() {
+ Some(note) => format!("{base}; {note}"),
+ None => base,
+ }
+ } else if publish_failed {
format!(
"rehearsal stopped at {}/{}: {}",
packages_published + 1,
ws.plan.packages.len(),
first_failure.as_deref().unwrap_or("")
)
+ } else {
+ format!(
+ "rehearsed {packages_published} packages against '{rehearsal_name}', but {}",
+ first_failure.as_deref().unwrap_or("smoke-install failed")
+ )
};
@@
- let packages_attempted = packages_published + if passed { 0 } else { 1 };
+ let packages_attempted = if publish_failed {
+ packages_published + 1
+ } else {
+ packages_published
+ };Also applies to: 1642-1652, 1672-1681
| let env_vars = fake_program_env_vars(&bin); | ||
| temp_env::with_vars(env_vars, || { | ||
| let rehearsal_server = spawn_registry_server( | ||
| std::collections::BTreeMap::from([( | ||
| "/api/v1/crates/demo/0.1.0".to_string(), | ||
| vec![(200, "{}".to_string())], | ||
| )]), | ||
| 1, | ||
| ); | ||
|
|
||
| let ws = planned_workspace(td.path(), "http://127.0.0.1:1".into()); | ||
| let mut opts = default_opts(PathBuf::from(".shipper")); | ||
| opts.rehearsal_registry = Some("rehearsal".to_string()); | ||
| opts.registries = vec![Registry { | ||
| name: "rehearsal".to_string(), | ||
| api_base: rehearsal_server.base_url.clone(), | ||
| index_base: None, | ||
| }]; | ||
| opts.rehearsal_smoke_install = Some("demo".to_string()); | ||
|
|
||
| let mut reporter = CollectingReporter::default(); | ||
| let outcome = run_rehearsal(&ws, &opts, &mut reporter).expect("rehearse"); | ||
| assert!(outcome.passed, "outcome: {outcome:?}"); | ||
|
|
||
| let events = read_events_raw(&td.path().join(".shipper")); | ||
| let types: Vec<String> = events.iter().filter_map(event_discriminator).collect(); | ||
| assert!( | ||
| types.contains(&"rehearsal_smoke_check_started".to_string()), | ||
| "types: {types:?}" | ||
| ); | ||
| assert!( | ||
| types.contains(&"rehearsal_smoke_check_succeeded".to_string()), | ||
| "types: {types:?}" | ||
| ); |
There was a problem hiding this comment.
🧹 Nitpick | 🔵 Trivial
Assert the actual cargo install invocation.
This happy-path test only checks emitted events. Capture the fake cargo args so regressions in cargo_install_smoke wiring—wrong subcommand, registry, crate, or version—fail the test.
🧪 Proposed test strengthening
let bin = td.path().join("bin");
write_fake_tools(&bin);
- let env_vars = fake_program_env_vars(&bin);
+ let args_log = td.path().join("cargo_args.txt");
+ let mut env_vars = fake_program_env_vars(&bin);
+ env_vars.push((
+ "SHIPPER_CARGO_ARGS_LOG",
+ Some(args_log.to_str().expect("utf8").to_string()),
+ ));
temp_env::with_vars(env_vars, || {
@@
assert!(
types.contains(&"rehearsal_smoke_check_succeeded".to_string()),
"types: {types:?}"
);
+
+ let cargo_log = fs::read_to_string(&args_log).expect("read cargo args log");
+ let install_line = cargo_log
+ .lines()
+ .find(|line| line.split_whitespace().next() == Some("install"))
+ .expect("expected cargo install invocation");
+ assert!(install_line.contains("--registry rehearsal"), "{install_line}");
+ assert!(install_line.contains("demo"), "{install_line}");
+ assert!(install_line.contains("--version 0.1.0"), "{install_line}");
+ assert!(install_line.contains("--locked"), "{install_line}");
rehearsal_server.join();
});🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@crates/shipper/src/engine/mod.rs` around lines 7102 - 7135, The test
currently only asserts emitted events; also assert the fake cargo invocation to
catch regressions in cargo_install_smoke wiring: after run_rehearsal(&ws, &opts,
&mut reporter) read the recorded fake cargo args produced by
fake_program_env_vars (or the mechanism used by your test harness to capture
program invocations) and assert it contains the expected subcommand and
parameters (e.g. "install", the rehearsal registry name "rehearsal" or
"--registry rehearsal" as your implementation emits, the crate "demo" and the
expected version if applicable). Locate the capture point created by
fake_program_env_vars and add assertions near the existing event checks (after
outcome) to validate the exact cargo invocation used by cargo_install_smoke so
wrong subcommand, registry, crate or version will fail the test.
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
Summary
Final piece of #97's acceptance. Adds an opt-in `cargo install --registry ` step after rehearsal publishes succeed, as end-to-end proof that the crate resolves + installs via the registry index. This is the scenario workspace-path dependencies defeat and that killed the rc.1 first-publish.
What's new
Why opt-in
Closes #97
Once merged, #97 can be closed.
Tests
19/19 rehearsal+gate tests pass (up from 17).
Test plan
Follow-ons (not this PR)