Skip to content

feat(workspace): preserve comments in workspace yaml edits via yamlpatch#511

Merged
jdx merged 6 commits intomainfrom
refactor/yamlpatch-comment-preserving
May 5, 2026
Merged

feat(workspace): preserve comments in workspace yaml edits via yamlpatch#511
jdx merged 6 commits intomainfrom
refactor/yamlpatch-comment-preserving

Conversation

@jdx
Copy link
Copy Markdown
Contributor

@jdx jdx commented May 4, 2026

Summary

  • Route every workspace yaml writer (allowBuilds, patchedDependencies, catalog upsert/cleanup, config TUI key writes) through yamlpatch instead of yaml_serde::to_string. Keys, comments, and whitespace the closure didn't touch land back on disk byte-identical, so the daily install (cleanupUnusedCatalogs) and one-shot edits (approve-builds, patch-commit, patch-remove, config set --location workspace) no longer wipe user annotations.
  • edit_workspace_yaml keeps its closure-based public API. Internally it diffs the parsed before/after yaml_serde::Mapping, reduces the diff to a minimal sequence of Add / Replace / Remove patches, and applies them via yamlpatch::apply_yaml_patches. Empty/missing files still go through yaml_serde since there are no comments to preserve.
  • Bonus fix: aube remove now leaves the surviving dep entries in their original on-disk order. Previously sync_manifest_dep_sections alphabetized the manifest as a side effect of dropping one name. Switched to in-place shift_remove against the parsed JSON object, mirroring pnpm/npm. (aube add is unaffected — sorted inserts there are intentional.)

Test plan

  • cargo clippy --all-targets -- -D warnings clean
  • cargo fmt --check clean
  • cargo test --workspace passes (1442 tests across all crates)
  • New tests:
    • edit_workspace_yaml_preserves_comments_around_unchanged_keys
    • upsert_workspace_patched_dependency_preserves_comments_on_real_change
    • remove_workspace_patched_dependency_preserves_comments_on_real_remove
    • prune_preserves_comments_when_dropping_one_entry
    • remove_preserves_dep_order_in_section
  • Manual smoke: run aube approve-builds esbuild against a workspace yaml with # ... annotations on adjacent keys; confirm annotations survive.

Notes

  • New deps: yamlpatch 1.24, yamlpath 1.24 (transitive: tree-sitter, tree-sitter-yaml), serde_yaml 0.9 (yamlpatch's value type — yaml_serde stays for typed reads).
  • Orphaned annotation comments on pruned catalog entries are kept in place by yamlpatch's Remove op, which only deletes the key: value line. Documented in the cleanupUnusedCatalogs setting docs as "clean up by hand if you don't want orphaned annotations" — the alternative (heuristic comment ownership) would risk eating real ones.

🤖 Generated with Claude Code


Note

Medium Risk
Touches core workspace-YAML write paths and introduces a new diff/patch mechanism plus dependencies, which could cause subtle YAML output/edge-case regressions despite added tests.

Overview
Workspace YAML mutations are now comment/format preserving by routing writes through yamlpatch/yamlpath instead of round-tripping via yaml_serde, with edit_workspace_yaml diffing the before/after yaml_serde::Mapping and applying minimal Remove/Replace ops plus custom injection for nested mapping adds.

Workspace-level writers (e.g. allowBuilds, patchedDependencies, catalog cleanup, and config TUI workspace writes) are updated to use this shared helper, and docs/settings are adjusted to reflect preserved comments (with note about possible orphaned annotations on removed entries).

Separately, aube remove now deletes dependencies and sidecar metadata in-place using shift_remove to avoid reordering remaining package.json keys, with new regression tests covering comment preservation and key-order stability.

Reviewed by Cursor Bugbot for commit 2445eb2. Bugbot is set up for automated code reviews on this repo. Configure here.

jdx and others added 2 commits May 4, 2026 16:28
Route every workspace yaml writer (allowBuilds, patchedDependencies,
catalog upsert/cleanup, config TUI key writes) through `yamlpatch`
instead of round-tripping through `yaml_serde::to_string`. yamlpatch
edits the original source surgically — keys, comments, and whitespace
the closure didn't touch land back on disk byte-identical, so the
daily install (cleanupUnusedCatalogs) and one-shot edits
(`approve-builds`, `patch-commit`, `patch-remove`,
`config set --location workspace`) no longer wipe user annotations.

`edit_workspace_yaml` keeps its closure-based public API. Internally
it diffs the parsed before/after `yaml_serde::Mapping`, reduces the
diff to a minimal sequence of `Add` / `Replace` / `Remove` patches,
and applies them via `yamlpatch::apply_yaml_patches`. The empty-file
path still falls through to `yaml_serde` since there are no
comments to preserve.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
`aube remove` previously rebuilt every dep section from `BTreeMap`
via `sync_manifest_dep_sections`, which alphabetized the surviving
keys whenever the user dropped one — a one-line uninstall would
reshuffle the entire manifest as a side effect. Pnpm and npm leave
the on-disk order alone and only delete the named entries, and that
is what `aube remove` should do too.

Mutate the parsed `serde_json::Map` in place inside
`update_manifest_json_object` instead. Use `shift_remove` instead of
the default `remove`, since serde_json's `Map` is an `IndexMap` under
the `preserve_order` feature and the default is `swap_remove`, which
would still scramble surviving keys. Same fix applied to
`prune_sidecar_entries_json` so namespaced and top-level cleanups
also keep their order.

`aube add` continues to go through `sync_manifest_dep_sections` —
inserts are expected to land in a stable sorted spot, so the
BTreeMap path is the right call there.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@greptile-apps
Copy link
Copy Markdown

greptile-apps Bot commented May 4, 2026

Greptile Summary

  • Routes all workspace YAML writers through a new yaml_patch module that diffs before/after yaml_serde::Mapping state and applies a minimal set of yamlpatch operations to the original source, preserving comments and formatting on untouched keys. Fresh or empty files fall back to the existing yaml_serde::to_string path.
  • Previous review concerns (to_serde_value panicking via .expect(), non-string keys silently dropped) have both been resolved: to_serde_value now returns Result, and key_str returns a hard error for non-string keys.
  • aube remove now uses shift_remove directly on the raw JSON IndexMap instead of rebuilding sections from BTreeMap, preserving on-disk key order; the typed manifest struct is still updated via prune_sidecar_entries and used for downstream resolution and lockfile writes.

Confidence Score: 4/5

Safe to merge; no P0/P1 issues found, and previous review concerns have been addressed.

No critical or likely-bug findings. The one P2 (silent unwrap_or_default() fallback in render_scalar/scalar_key_str) is vanishingly unlikely to trigger in practice given the types involved, but is worth a follow-up. Test coverage for the new patching path is thorough.

The yaml_patch module in crates/aube-manifest/src/workspace.rs (~410 new lines) is the core of this change; render_scalar and scalar_key_str in particular use silent error fallbacks.

Important Files Changed

Filename Overview
crates/aube-manifest/src/workspace.rs Core change: adds yaml_patch module (~410 lines) that diffs before/after Mapping state and applies minimal yamlpatch ops to preserve comments. Previous .expect() and silent-drop concerns have been addressed — to_serde_value now propagates errors and key_str returns a hard error for non-string keys.
crates/aube/src/commands/remove.rs Replaces sync_manifest_dep_sections with direct shift_remove on the raw JSON object to preserve on-disk key order; typed manifest struct is still updated via prune_sidecar_entries and used for downstream resolution/lockfile steps, so no state divergence.
crates/aube/src/commands/config/tui.rs Routes write_workspace_value and clear_workspace_value through edit_workspace_yaml; pre-flight read in clear_workspace_value preserves the original graceful Ok(false) for empty/non-mapping files.
crates/aube/src/commands/catalogs.rs Test-only addition: prune_preserves_comments_when_dropping_one_entry validates comment survival through the new patching path.
Cargo.toml Adds yamlpatch 1.24, yamlpath 1.24, and serde_yaml 0.9 (deprecated crate, required as yamlpatch's value type). The serde_yaml 0.9+deprecated crate is a known-maintained snapshot; pulling it alongside yaml_serde (its maintained fork) is the intended usage pattern per the PR notes.

Fix All in Claude Code

Reviews (5): Last reviewed commit: "fix(workspace): harden yaml diff against..." | Re-trigger Greptile

Comment thread crates/aube-manifest/src/workspace.rs Outdated
Comment thread crates/aube-manifest/src/workspace.rs
Comment thread crates/aube/src/commands/config/tui.rs
CI surfaced two yamlpatch issues the first version didn't catch:

1. `Op::Add { value: Mapping(...) }` for non-empty nested mappings
   strips the inner indentation hierarchy and emits invalid YAML
   (every nested line lands at the parent's column). Trips on
   first-time `catalogs:` upserts and any new `allowBuilds:` block.
2. `Op::Add { key }` interpolates the key verbatim, so a YAML-special
   key like `@pnpm.e2e/...` lands unquoted and breaks the document
   on the next parse — repros from the install-time auto-deny seed
   when the existing entry uses a quoted key.

Route every Add (mapping or scalar) through a small block-style
emitter that resolves the parent's byte span via `yamlpath`, infers
the child indent from `extract_with_leading_whitespace`, and
inserts a properly-quoted, properly-indented entry into the source
text. yamlpatch is still used for `Remove` and `Replace` — those
work correctly and are where the comment-preservation win lives.

Other fallout this commit cleans up:

- Drop `yamlpatch` / `yamlpath` / `serde_yaml` from `aube/Cargo.toml`.
  They were transitively pulled via `aube-manifest`; cargo-machete
  failed CI because nothing in the `aube` crate referenced them
  directly.
- Address Greptile P1: `to_serde_value` no longer `.expect()`s on
  fallible serialization; errors propagate as `crate::Error`.
- Address Greptile P2: non-string mapping keys now error out instead
  of being silently dropped from the diff.

New regression tests cover all three failure modes:
`edit_workspace_yaml_adds_nested_mapping_and_round_trips`,
`edit_workspace_yaml_adds_nested_mapping_under_existing_parent`,
`add_to_allow_builds_merges_with_quoted_existing_key`.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Comment thread crates/aube-manifest/src/workspace.rs
…eful

Two follow-ups from the review bots on the previous commit:

1. `scalar_key_str` had a hand-rolled "safe character" set that
   omitted `@`, so non-leading-`@` keys like `b@2.0.0` and
   `is-positive@3.1.0` got unnecessarily double-quoted on the wire —
   contradicted the function's own doc string and produced style
   inconsistencies with the existing entries on disk. Defer to
   serde_yaml's emitter so plain-scalar quoting matches the rest of
   the file: `b@2.0.0` stays unquoted, `@scope/pkg@1.0.0` (leading
   reserved indicator) gets the canonical quoted form.

2. `clear_workspace_value` in the config TUI used to no-op silently
   on a workspace yaml whose top level wasn't a mapping (`Ok(false)`).
   Routing through `edit_workspace_yaml` turned that into a hard
   error. Restore the graceful path with a pre-flight check —
   clearing one key is a poor place to surface a parse error.

New tests pin both behaviors:
`upsert_workspace_patched_dependency_does_not_quote_unreserved_at_keys`,
`upsert_workspace_patched_dependency_quotes_leading_at_keys`.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Comment thread crates/aube/src/commands/config/tui.rs Outdated
Comment thread crates/aube/src/commands/config/tui.rs
Cursor flagged the previous revision: silently mapping a yaml parse
error to `Ok(false)` masks genuine file corruption from the user. A
malformed workspace yaml is a real failure — surface it. Empty,
missing, and non-mapping top-level shapes stay graceful (matches the
pre-migration behavior of this command).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Copy link
Copy Markdown

@cursor cursor Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 2 potential issues.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 425b039. Configure here.

Comment thread crates/aube-manifest/src/workspace.rs Outdated
Comment thread crates/aube-manifest/src/workspace.rs
Three review-flagged latent issues in `yaml_patch::apply_diff`:

- `render_entry`'s catch-all arm fed sequence values through
  `render_scalar`, producing structurally invalid YAML when a sequence-
  valued key was added (`key: - a\n- b\n`). Add an explicit Sequence arm
  that emits block-style items at child indent, with a delegated path
  for the rare nested-mapping/sequence-in-list case.
- `Op::Replace` of a scalar with a non-empty mapping went through
  yamlpatch's same broken serializer as `Op::Add` and would land children
  at the parent's column. Detect the type-change and split into a
  Remove + manual injection so `inject_entry` re-emits children at the
  canonical indent.
- `detect_child_indent` fell back to a hard-coded column 2 for empty
  parents at depth >= 2, making new children siblings rather than
  descendants. Pass the parent's depth and use parent_depth *
  INDENT_STEP instead.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@jdx
Copy link
Copy Markdown
Contributor Author

jdx commented May 5, 2026

Addressed review feedback in 2445eb2:

Greptile P2 / Cursor — render_entry sequence handling (workspace.rs:1317): added an explicit Sequence arm so block-sequence values emit - item lines at child indent instead of inlining the multi-line scalar dump on the key: line. Nested mapping/sequence list items fall back to serde_yaml::to_string with the dash continuation applied. Locked behind edit_workspace_yaml_adds_sequence_value_as_block_style.

Greptile P2 — Op::Replace for mapping values (workspace.rs:1214–1224): the type-change "scalar → non-empty mapping" path now emits a Remove + Edit::Add pair so the new sub-mapping flows through inject_entry (canonical indent) instead of yamlpatch's broken Op::Replace serializer (parent-column children). Locked behind edit_workspace_yaml_replaces_scalar_with_nested_mapping.

Cursor — detect_child_indent fallback for nested mappings (workspace.rs:1358): the fallback now uses parent_depth * INDENT_STEP instead of a hard-coded column 2, so an empty depth-N parent's children land at column N×2 rather than aliased to column 2. Locked behind unit tests inside mod yaml_patch::tests (detect_child_indent_falls_back_to_parent_depth, etc.).

Cursor — scalar_key_str @ handling (workspace.rs:1342): already addressed in 629276d by deferring quoting to serde_yaml's emitter; upsert_workspace_patched_dependency_does_not_quote_unreserved_at_keys covers the b@2.0.0 regression.

Cursor — clear_workspace_value (TUI) (tui.rs:897-922): non-mapping returns Ok(false) and parse errors propagate (already addressed in 425b039). The remaining double-read concern is intentional and justified by the inline comment — TUI-only path, not the install hot path.

cargo clippy --all-targets -- -D warnings, cargo fmt --check, cargo test --workspace all clean.

This comment was generated by Claude.

@jdx jdx merged commit 9592446 into main May 5, 2026
18 checks passed
@jdx jdx deleted the refactor/yamlpatch-comment-preserving branch May 5, 2026 15:19
@greptile-apps greptile-apps Bot mentioned this pull request May 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant