docs: align roadmap, add mission/steering docs, Diátaxis reorg, tighten README#110
Conversation
…ncy thesis Rewrites ROADMAP.md around the post-rc.1 product thesis (nine competencies: Prove, Survive, Reconcile, Narrate, Remediate, Harden, Profile, Integrate, Ergonomics) with status, sequencing, and links to tracking issues #100-#109. Old per-feature roadmap tables were largely shipped or now tracked as issues. Adds docs/INVARIANTS.md articulating the events-as-truth / state-as-projection contract per #93. README gains a Status section and honest install note re #95. CLAUDE.md and GEMINI.md gain short pointers to the invariant and thesis so AI collaborators have the same north star.
WalkthroughThis pull request updates documentation across multiple files to establish a product-centric framework. It introduces events-as-truth invariants for persisted artifacts, defines system consistency guarantees, restructures the roadmap around nine competencies, and updates the README with release status and installation guidance. All changes are documentation-only. Changes
Estimated code review effort🎯 2 (Simple) | ⏱️ ~12 minutes Poem
🚥 Pre-merge checks | ✅ 3✅ Passed checks (3 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Code Review
This pull request updates the project documentation to reflect the v0.3.0-rc.1 release and introduces a new product thesis centered on nine competencies. It establishes the 'Events-as-truth' invariant, clarifying that events.jsonl is the authoritative source of truth while state.json and receipt.json are projections and summaries, respectively. Feedback includes minor suggestions for link consistency in the roadmap and improved phrasing of the invariant definition.
| |---|---| | ||
| | Version bumping | [cargo-release](https://github.com/crate-ci/cargo-release) | | ||
| | Changelog generation | [release-plz](https://github.com/MarcoIeni/release-plz) | | ||
| | Git tag creation | cargo-release | |
There was a problem hiding this comment.
For consistency with the link on line 73, cargo-release should also be a link in this line.
| | Git tag creation | cargo-release | | |
| | Git tag creation | [cargo-release](https://github.com/crate-ci/cargo-release) | |
|
|
||
| ## The invariant | ||
|
|
||
| > The set of `package_published` events in `events.jsonl` MUST equal the set of packages with `state.state == "published"` in `state.json`. |
There was a problem hiding this comment.
The current wording is slightly imprecise as it compares a "set of events" to a "set of packages". To improve clarity and correctness, consider rephrasing to compare two sets of packages.
| > The set of `package_published` events in `events.jsonl` MUST equal the set of packages with `state.state == "published"` in `state.json`. | |
| > The set of packages with a `package_published` event in `events.jsonl` MUST equal the set of packages with `state.state == "published"` in `state.json`. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: da88f8a9ae
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
|
||
| `shipper resume` reads `state.json` to decide which packages to skip. If state.json drifts from events, resume could either re-publish duplicates (state under-reports success) or refuse to continue valid work (state over-reports success). | ||
|
|
||
| The contract guarantees: even if `state.json` is corrupted or deleted, the run can be reconstructed from `events.jsonl` alone (per [#101](https://github.com/EffortlessMetrics/shipper/issues/101)'s state-rebuild capability). |
There was a problem hiding this comment.
Remove unsupported state-rebuild guarantee
This sentence guarantees that a run can be reconstructed from events.jsonl when state.json is missing, but run_resume currently hard-fails when no state file exists (crates/shipper/src/engine/mod.rs:1026-1030), so operators following this doc during recovery will hit a dead end instead of a supported path. Please either downgrade this to future work or document the current manual recovery procedure.
Useful? React with 👍 / 👎.
|
|
||
| > The set of `package_published` events in `events.jsonl` MUST equal the set of packages with `state.state == "published"` in `state.json`. | ||
|
|
||
| Drift between events and state is a bug. Per [#93](https://github.com/EffortlessMetrics/shipper/issues/93), an end-of-run consistency check enforces this and emits `state_event_drift_detected` on mismatch. |
There was a problem hiding this comment.
Drop claim about drift event emission
The doc says an end-of-run consistency check emits state_event_drift_detected, but there is no such event variant in the serialized event schema (EventType in crates/shipper-types/src/lib.rs) and no corresponding emission path in the engine, so tooling authors may wait for an event that can never appear. This should be reworded as planned work unless the check/event is implemented.
Useful? React with 👍 / 👎.
…README Adds the north-star and orientation docs that were missing: - MISSION.md — mission, vision, audience, and the nine convictions that produce every design decision (events-as-truth, irreversibility shapes defaults, ambiguity reconciles against registry truth, engine-is-library, forbid unsafe, etc.). Referenced by README/ROADMAP/CLAUDE/GEMINI. - docs/product.md, docs/structure.md, docs/tech.md — Kiro-style steering docs giving a quick orientation snapshot for contributors and AI tools: what the product is, how the repo is laid out, what's in the tech stack. Each links back to MISSION/ROADMAP for the deeper why. README is rewritten to be tighter: adds a "How it works" section showing the plan->preflight->publish->resume flow and the events/state/receipt trio, an audience routing table, and a Mission link. Removes a 60-line Options table and ~100 lines of redundant Examples that just duplicated 'shipper --help' output. Net README -244 / +110. CLAUDE.md and GEMINI.md gain an "Orientation" section pointing at the new docs so AI collaborators land in the right place first.
There was a problem hiding this comment.
Actionable comments posted: 3
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@docs/INVARIANTS.md`:
- Line 26: Remove or rephrase the false guarantee sentence that currently reads
"The contract guarantees: even if `state.json` is corrupted or deleted, the run
can be reconstructed from `events.jsonl` alone" in INVARIANTS.md: either delete
that sentence entirely or change it to clearly state this is a planned/desired
capability (e.g., "planned: rebuilding state from `events.jsonl` is a future
enhancement tracked in `#101`") so the document no longer claims an implemented
feature; ensure the updated text references issue `#101` as future work rather
than a guaranteed capability.
In `@ROADMAP.md`:
- Around line 33-46: The markdown linter flags MD022 because the subheadings
"### Reliability over speed", "### Determinism", "### Events are truth, state is
a projection", "### Engine is library; CLI is thin", and "### Forbid unsafe;
respect MSRV" are not surrounded by blank lines; fix this by inserting a single
blank line immediately before each of those subheading lines (and ensure there
is at least one blank line after the preceding paragraph), then re-run the
linter to confirm MD022 is resolved.
- Around line 60-65: In the "Next" section, the ordered list items currently
show 6, 7, 8—change them to a sequential 1, 2, 3 so the list restarts within
that section (the items referencing issues `#106/`#94, `#104/`#98, and `#100/`#97);
likewise in the "Later — once the engine is closure-complete" section change the
lone item currently numbered 9 to 1 (the item referencing `#107`) so each section
uses an explicit 1-based sequence and avoids carrying numbering across sections.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro
Run ID: fe033cc2-1094-47d1-a98d-1b749467efcd
📒 Files selected for processing (5)
CLAUDE.mdGEMINI.mdREADME.mdROADMAP.mddocs/INVARIANTS.md
|
|
||
| `shipper resume` reads `state.json` to decide which packages to skip. If state.json drifts from events, resume could either re-publish duplicates (state under-reports success) or refuse to continue valid work (state over-reports success). | ||
|
|
||
| The contract guarantees: even if `state.json` is corrupted or deleted, the run can be reconstructed from `events.jsonl` alone (per [#101](https://github.com/EffortlessMetrics/shipper/issues/101)'s state-rebuild capability). |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
# Verify if state reconstruction from events is implemented
echo "=== Searching for state reconstruction logic ==="
rg -n -C3 'reconstruct|rebuild.*state|fold.*events' --type=rust
echo ""
echo "=== Checking issue `#101` for context ==="
gh issue view 101 --repo EffortlessMetrics/shipper --json title,body,state | jq -r '.title, .body, .state'Repository: EffortlessMetrics/shipper
Length of output: 4195
Remove false guarantee about state reconstruction from events.jsonl alone.
Line 26 states "The contract guarantees: even if state.json is corrupted or deleted, the run can be reconstructed from events.jsonl alone" and references issue #101. However, issue #101 explicitly lists "State rebuild from events.jsonl alone (corruption recovery)" under "What's missing" and recommends it as a future task. This capability does not exist in the current codebase.
The documentation must not claim a guarantee for unimplemented functionality. Either remove this statement or rephrase it to clarify this is a planned capability, not a current contract.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@docs/INVARIANTS.md` at line 26, Remove or rephrase the false guarantee
sentence that currently reads "The contract guarantees: even if `state.json` is
corrupted or deleted, the run can be reconstructed from `events.jsonl` alone" in
INVARIANTS.md: either delete that sentence entirely or change it to clearly
state this is a planned/desired capability (e.g., "planned: rebuilding state
from `events.jsonl` is a future enhancement tracked in `#101`") so the document no
longer claims an implemented feature; ensure the updated text references issue
`#101` as future work rather than a guaranteed capability.
| ### Reliability over speed | ||
| Default behaviors verify, log, and provide evidence. Faster paths are explicit opt-ins. The default publish policy (`safe`) includes all verification. | ||
|
|
||
| ### 1. Reliability Over Speed | ||
| ### Determinism | ||
| Publish order is reproducible. Plan IDs are SHA256 of the workspace plan and stable across environments. The same workspace state always produces the same `plan_id`. | ||
|
|
||
| Shipper prioritizes correct behavior and data integrity over fast execution. When in doubt, Shipper verifies, logs, and provides evidence rather than assuming success. | ||
| ### Events are truth, state is a projection | ||
| Per [docs/INVARIANTS.md](docs/INVARIANTS.md): `events.jsonl` is authoritative and append-only. `state.json` is a projection over events for resume convenience. `receipt.json` is a summary derived from events at end-of-run. The relationship is contractual; see [#93](https://github.com/EffortlessMetrics/shipper/issues/93) for enforcement. | ||
|
|
||
| - **Default to safe behavior**: The default publish policy (`safe`) includes all verification steps | ||
| - **Evidence over assumptions**: Every operation captures evidence for debugging and auditing | ||
| - **Explicit over implicit**: Users must opt-in to faster but riskier behaviors | ||
| ### Engine is library; CLI is thin | ||
| All domain logic lives in `crates/shipper`. `crates/shipper-cli` parses args and calls into the library. Other frontends (IDP plugins, dashboards, automation) consume the library directly. | ||
|
|
||
| ### 2. Determinism | ||
| ### Forbid unsafe; respect MSRV | ||
| `unsafe_code = "forbid"` workspace-wide. Edition 2024, MSRV 1.92. |
There was a problem hiding this comment.
🧹 Nitpick | 🔵 Trivial
Consider adding blank lines around subheadings for better markdown compliance.
The markdown linter flags several MD022 violations where subheadings under "Design principles" lack surrounding blank lines (lines 33, 36, 39, 42, 45). While not critical, adding blank lines before each subheading (e.g., before "### Reliability over speed") would improve document formatting consistency.
🧰 Tools
🪛 LanguageTool
[grammar] ~46-~46: Ensure spelling is correct
Context: ..."forbid"` workspace-wide. Edition 2024, MSRV 1.92. ## Now / Next / Later Sequencin...
(QB_NEW_EN_ORTHOGRAPHY_ERROR_IDS_1)
🪛 markdownlint-cli2 (0.22.0)
[warning] 33-33: Headings should be surrounded by blank lines
Expected: 1; Actual: 0; Below
(MD022, blanks-around-headings)
[warning] 36-36: Headings should be surrounded by blank lines
Expected: 1; Actual: 0; Below
(MD022, blanks-around-headings)
[warning] 39-39: Headings should be surrounded by blank lines
Expected: 1; Actual: 0; Below
(MD022, blanks-around-headings)
[warning] 42-42: Headings should be surrounded by blank lines
Expected: 1; Actual: 0; Below
(MD022, blanks-around-headings)
[warning] 45-45: Headings should be surrounded by blank lines
Expected: 1; Actual: 0; Below
(MD022, blanks-around-headings)
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@ROADMAP.md` around lines 33 - 46, The markdown linter flags MD022 because the
subheadings "### Reliability over speed", "### Determinism", "### Events are
truth, state is a projection", "### Engine is library; CLI is thin", and "###
Forbid unsafe; respect MSRV" are not surrounded by blank lines; fix this by
inserting a single blank line immediately before each of those subheading lines
(and ensure there is at least one blank line after the preceding paragraph),
then re-run the linter to confirm MD022 is resolved.
| 6. **[#106](https://github.com/EffortlessMetrics/shipper/issues/106) Profile** — registry-aware backoff layered on the regime tag preflight already detects ([#94](https://github.com/EffortlessMetrics/shipper/issues/94)). Preflight discovers `is_new_crate`; publish discards it. Threading the tag through is the entire mechanism. | ||
| 7. **[#104](https://github.com/EffortlessMetrics/shipper/issues/104) Remediate** — receipt-driven yank/fix-forward for compromised releases ([#98](https://github.com/EffortlessMetrics/shipper/issues/98)). | ||
| 8. **[#100](https://github.com/EffortlessMetrics/shipper/issues/100) Prove tier 2** — rehearsal registry as the next preflight strength ([#97](https://github.com/EffortlessMetrics/shipper/issues/97)). Promotes preflight from "we believe" to "we proved against a registry-shaped target". | ||
|
|
||
| - **Clear progress indication**: Each step's purpose and status is visible | ||
| - **Detailed logging**: Event log captures every operation with context | ||
| - **Evidence preservation**: Failed operations retain stdout/stderr for debugging | ||
| ### Later — once the engine is closure-complete | ||
| 9. **[#107](https://github.com/EffortlessMetrics/shipper/issues/107) Integrate** — IDP plugin examples (Backstage / Port / Cortex), HTTP query API, webhook reliability semantics, library consumer guide. Best done after the engine offers a stable closure story to integrate against. |
There was a problem hiding this comment.
🧹 Nitpick | 🔵 Trivial
Fix ordered list numbering to continue sequentially.
The "Next" section list items are numbered 6, 7, 8 instead of restarting at 1. While some markdown renderers handle this, explicitly numbering as 1, 2, 3 within each section improves readability and avoids confusion.
📝 Proposed fix for list numbering
### Next — past stable
-6. **[`#106`](https://github.com/EffortlessMetrics/shipper/issues/106) Profile** — registry-aware backoff layered on the regime tag preflight already detects ([`#94`](https://github.com/EffortlessMetrics/shipper/issues/94)). Preflight discovers `is_new_crate`; publish discards it. Threading the tag through is the entire mechanism.
-7. **[`#104`](https://github.com/EffortlessMetrics/shipper/issues/104) Remediate** — receipt-driven yank/fix-forward for compromised releases ([`#98`](https://github.com/EffortlessMetrics/shipper/issues/98)).
-8. **[`#100`](https://github.com/EffortlessMetrics/shipper/issues/100) Prove tier 2** — rehearsal registry as the next preflight strength ([`#97`](https://github.com/EffortlessMetrics/shipper/issues/97)). Promotes preflight from "we believe" to "we proved against a registry-shaped target".
+1. **[`#106`](https://github.com/EffortlessMetrics/shipper/issues/106) Profile** — registry-aware backoff layered on the regime tag preflight already detects ([`#94`](https://github.com/EffortlessMetrics/shipper/issues/94)). Preflight discovers `is_new_crate`; publish discards it. Threading the tag through is the entire mechanism.
+2. **[`#104`](https://github.com/EffortlessMetrics/shipper/issues/104) Remediate** — receipt-driven yank/fix-forward for compromised releases ([`#98`](https://github.com/EffortlessMetrics/shipper/issues/98)).
+3. **[`#100`](https://github.com/EffortlessMetrics/shipper/issues/100) Prove tier 2** — rehearsal registry as the next preflight strength ([`#97`](https://github.com/EffortlessMetrics/shipper/issues/97)). Promotes preflight from "we believe" to "we proved against a registry-shaped target".Similarly for line 65 in the "Later" section.
🧰 Tools
🪛 markdownlint-cli2 (0.22.0)
[warning] 60-60: Ordered list item prefix
Expected: 1; Actual: 6; Style: 1/2/3
(MD029, ol-prefix)
[warning] 61-61: Ordered list item prefix
Expected: 2; Actual: 7; Style: 1/2/3
(MD029, ol-prefix)
[warning] 62-62: Ordered list item prefix
Expected: 3; Actual: 8; Style: 1/2/3
(MD029, ol-prefix)
[warning] 64-64: Headings should be surrounded by blank lines
Expected: 1; Actual: 0; Below
(MD022, blanks-around-headings)
[warning] 65-65: Ordered list item prefix
Expected: 1; Actual: 9; Style: 1/1/1
(MD029, ol-prefix)
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@ROADMAP.md` around lines 60 - 65, In the "Next" section, the ordered list
items currently show 6, 7, 8—change them to a sequential 1, 2, 3 so the list
restarts within that section (the items referencing issues `#106/`#94, `#104/`#98,
and `#100/`#97); likewise in the "Later — once the engine is closure-complete"
section change the lone item currently numbered 9 to 1 (the item referencing
`#107`) so each section uses an explicit 1-based sequence and avoids carrying
numbering across sections.
…e prop Layers in the framing crystallized by the external review of the v0.3.0-rc.1 retrospective: - ROADMAP.md gains a "Five existential pillars" section (Prove, Dispatch, Reconcile, Recover, Remediate) above the nine-competency table. The pillars are the safety claim — the five guarantees Shipper must own to justify existing distinct from Cargo. The nine competencies remain the granular scorecard; the pillars cluster the existential ones. - MISSION.md and README.md adopt the sharper single-sentence test: "you can start a release train, stop staring at the terminal, and still trust the outcome." - docs/architecture.md rewritten. The previous version claimed 31 crates and listed many "Absorbed" microcrates that no longer exist as standalone packages. New version: 12 crates (the actual workspace), the architectural rule "Crates are semver promises, folders are ownership boundaries", what's a published crate vs an internal module, current dependency graph, and consequences of the rule (small public surface, pub(crate) by default, no deep lateral imports). - docs/structure.md: fixed an inaccuracy claiming 15 crates with 3 workspace-internal. Actual count is 12 published crates; the three "internal" entries (shipper-events/state/storage) were stray test- output directories on disk, not crates. Removed the directories.
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
Introduces a reader-purpose-organized docs tree on top of the existing flat docs/ directory. Existing docs stay in place (no broken links); new content fills the gaps the external review flagged. New docs: - docs/README.md — the Diátaxis index (tutorials / how-to / reference / explanation) - docs/tutorials/first-publish.md — end-to-end: install, plan, preflight, publish, inspect - docs/tutorials/recover-from-interruption.md — deliberate interrupt + resume + plan-ID guard - docs/how-to/run-in-github-actions.md — minimal workflow, key considerations (hidden-files flag, timeout, OIDC) - docs/how-to/inspect-state-and-receipts.md — which file answers which question - docs/explanation/why-shipper.md — the "why", finishability tri-state, events-as-truth, forbid unsafe - docs/reference/cli.md — topical map pointing to --help as canonical README's Documentation section collapses into a tight "Quick links" block grouped by reader intent. CLAUDE.md and GEMINI.md orientation sections now point at docs/README.md as the entry point + docs/explanation/why-shipper.md as the distilled "why".
Summary
Post-v0.3.0-rc.1 documentation alignment pass. Four categories of change across four commits.
1. Roadmap alignment
2. North star + orientation
3. Architecture doc rewrite
4. Diátaxis reorg
5. README + AI-doc polish
Why
We're about to start closing the gaps from #109. Without this pass, every contributor gets stale priorities from the old ROADMAP; every AI session lacks the events-as-truth contract; new users get the wrong install command; and there's no navigable docs structure for anyone who arrives at docs/.
Files
17 files changed, +1398 / -1097.
Plus removal of three stray test-artifact directories.
Test plan
Related