Replace ruamel.yaml with yamltrip for YAML round-tripping#1937
Replace ruamel.yaml with yamltrip for YAML round-tripping#1937nathanjmcdougall wants to merge 29 commits into
Conversation
ruamel.yaml has been showing signs of potential abandonment from PyPI (see #1190). This commit replaces it with yamltrip, a Rust-backed (tree-sitter-yaml) round-tripping YAML library that provides the same comment- and formatting-preservation guarantees with better performance and a smaller dependency footprint. Key changes: Dependency swap: - Remove ruamel-yaml from project dependencies - Add yamltrip>=0.2.0 Core YAML I/O rewrite (src/usethis/_file/yaml/io_.py): - Rewrite YAMLFileManager to use yamltrip.Document internally - Introduce YAMLDocument wrapper (replacing the raw ruamel CommentedMap/CommentedSeq) - Add _upsert_safe() for robust scalar upserts handling empty documents, non-mapping routes, and exists_ok semantics - Add _upsert_complex() for list/dict values using a document-rebuild approach (yamltrip only supports scalar upsert/replace natively) - Add _serialize_yaml_value/_serialize_yaml_scalar helpers for producing valid block-style YAML from Python objects - Handle flow-sequence fallback in extend_list when yamltrip cannot append to flow sequences (e.g. 'repos: []') Pre-commit integration (src/usethis/_integrations/pre_commit/): - Rewrite PreCommitConfigYAMLManager to use YAMLDocument - Refactor remove_hook() to use surgical remove_from_list for whole- repo removal (preserves inline comments on sibling repos) and only falls back to full rebuild when hooks within a repo are modified - Remove _pre_commit_fancy_dump (no longer needed; serialization is handled by the YAML I/O layer) Deleted modules: - src/usethis/_file/yaml/typing_.py (ruamel type aliases) - src/usethis/_file/yaml/update.py (ruamel update helpers) - tests/usethis/_file/yaml/test_update.py Test migration: - Rewrite all YAML I/O tests to use yamltrip assertions - Update pre-commit tests (test_hooks.py, test_yaml.py) to use yamltrip.loads() and YAMLDocument semantics - Relax exact-format assertions where the new serializer produces semantically equivalent but stylistically different output Documentation: - Update FAQ and pipeweld docs to reference yamltrip - Regenerate functions.txt, module-tree.txt, AGENTS.md via hooks Resolves #1190
Add explicit type arguments to dict and list parameters to satisfy pyright reportMissingTypeArgument checks.
- Suppress unreachable code warning in fancy_model_dump else branch - Add pyright: ignore for _FillValue type narrowing in zip_longest loop - Fix dict type annotations in test references - Remove unnecessary pyright ignore comments
Merging this PR will improve performance by 5.69%
|
| Mode | Benchmark | BASE |
HEAD |
Efficiency | |
|---|---|---|---|---|---|
| ⚡ | Simulation | test_several_tools_add_and_remove |
1.2 s | 1.1 s | +5.71% |
| ⚡ | Simulation | test_several_tools_add_and_remove_no_backend |
467.2 ms | 442.1 ms | +5.68% |
Tip
Curious why this is faster? Comment @codspeedbot explain why this is faster on this PR, or directly use the CodSpeed MCP with your agent.
Comparing refactor/replace-ruamel-yaml-with-yamltrip (5424968) with main (9c9ae10)
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
Demonstrates that removing one hook from a multi-hook repo currently loses inline comments due to the document-rebuild workaround. This test will pass once yamltrip supports complex values in upsert/replace. Tracks: usethis-python/yamltrip#18
The check-comment-keywords hook bans 'pragma:' in comments.
…ative complex values yamltrip 0.3.0 adds native complex value support in upsert/replace and a .root property. This eliminates ~130 lines of custom serialization code: - Remove _upsert_complex, _deep_set, and all _serialize_yaml_* functions - Replace doc[()] with doc.root throughout - Add placeholder-then-replace pattern in _upsert_safe to force block-style output for new complex value keys - Update test assertions for yamltrip's compact list indentation style - Update xfail reason for partial hook removal test
- Add 13 new tests covering edge cases: scalar/empty root docs, regex keys, flow sequence fallback, complex values on empty docs - Merge duplicate QueryError/PatchError handlers in __delitem__ (prune_remove never raises QueryError with yamltrip) - io_.py coverage: 87% -> 99% (1 defensive line remaining)
e1a5f61 to
fc5c913
Compare
ecdbedc to
e0c4ba0
Compare
There was a problem hiding this comment.
Pull request overview
This pull request migrates usethis’s YAML round-tripping backend from ruamel.yaml to yamltrip, updating the core YAML file manager, the pre-commit YAML integration, and the surrounding tests/docs to match the new YAML editing model.
Changes:
- Replace
ruamel-yamldependency withyamltrip(lockfile + project metadata updated). - Rewrite YAML read/modify/write logic to use
yamltrip.Documentand new upsert/sync behavior. - Update pre-commit/YAML-related tests and documentation to reflect new round-tripping semantics and formatting.
Reviewed changes
Copilot reviewed 20 out of 31 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| uv.lock | Removes ruamel-yaml(+clib) and adds yamltrip locked to 0.5.0. |
| pyproject.toml | Swaps runtime dependency from ruamel-yaml to yamltrip>=0.5.0. |
| src/usethis/file/yaml/io.py | Core YAML I/O manager rewritten to use yamltrip documents and patch operations. |
| src/usethis/_file/yaml/update.py | Deleted ruamel-specific update/LCS logic. |
| src/usethis/file/yaml/typing.py | Deleted ruamel scalar/type alias module. |
| src/_test.py | Updates test YAML helper to load via yamltrip and map parse errors to YAMLDecodeError. |
| src/usethis/_integrations/pre_commit/yaml.py | Updates pre-commit YAML manager to validate from yamltrip and commit via sync. |
| src/usethis/_integrations/pydantic/dump.py | Adjusts dump logic (incl. list diffing) and tightens type exhaustiveness handling. |
| src/usethis/integrations/pydantic/typing.py | Updates ModelRepresentation typing after removing ruamel-specific YAML literals. |
| src/usethis/_fallback.py | Bumps fallback tool versions (uv/ruff). |
| tests/usethis/file/yaml/test_yaml_io.py | Migrates YAML file manager tests from ruamel-specific behaviors to yamltrip semantics. |
| tests/usethis/_file/yaml/test_update.py | Deletes tests for the removed ruamel update helpers. |
| tests/usethis/_integrations/pre_commit/test_yaml.py | Updates tests to validate via yamltrip and new manager methods. |
| tests/usethis/_integrations/pre_commit/test_version.py | Updates YAML fixtures/expectations after YAML formatting changes. |
| tests/usethis/_integrations/pre_commit/test_language.py | Updates YAML fixtures/expectations after YAML formatting changes. |
| tests/usethis/_integrations/pre_commit/test_hooks.py | Updates YAML parsing to use yamltrip and adjusts expected YAML output strings. |
| tests/usethis/_integrations/pydantic/test_dump.py | Adjusts typing/pyright expectations related to model dumping changes. |
| tests/usethis/_tool/test_base.py | Updates embedded pre-commit YAML fixtures for the new formatting/round-tripping. |
| tests/usethis/_tool/impl/base/test_ty.py | Updates embedded pre-commit YAML fixtures for the new formatting/round-tripping. |
| tests/usethis/_tool/impl/base/test_tach.py | Updates embedded pre-commit YAML fixtures for the new formatting/round-tripping. |
| tests/usethis/_tool/impl/base/test_ruff.py | Updates embedded pre-commit YAML fixtures for the new formatting/round-tripping. |
| tests/usethis/_tool/impl/base/test_requirements_txt.py | Updates embedded pre-commit YAML fixtures for the new formatting/round-tripping. |
| tests/usethis/_tool/impl/base/test_import_linter.py | Updates embedded pre-commit YAML fixtures for the new formatting/round-tripping. |
| tests/usethis/_tool/impl/base/test_codespell.py | Updates embedded pre-commit YAML fixtures for the new formatting/round-tripping. |
| tests/usethis/_core/test_core_tool.py | Updates embedded pre-commit YAML fixtures for the new formatting/round-tripping. |
| tests/test__test.py | Adds coverage for the updated _test.read_yaml() behavior and error mapping. |
| docs/about/faq.md | Updates FAQ to reference yamltrip and describe formatting behavior. |
| docs/pipeweld.md | Updates documentation references from ruamel-based updates to yamltrip-based mutations. |
| docs/module-tree.txt | Removes deleted YAML modules from the documented module tree. |
| docs/functions.txt | Removes deleted YAML helper functions from the documented function list. |
| docs/superpowers/specs/2026-05-15-yamltrip-migration-design.md | Adds a design spec describing the migration approach and constraints. |
| docs/superpowers/plans/2026-05-15-yamltrip-migration.md | Adds an implementation plan for the migration work. |
| AGENTS.md | Updates agent/module/function inventories to reflect YAML module deletions. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
…github.com/usethis-python/usethis-python into refactor/replace-ruamel-yaml-with-yamltrip
ruamel.yaml has been showing signs of potential abandonment from PyPI (see #1190). This commit replaces it with yamltrip, a Rust-backed (tree-sitter-yaml) round-tripping YAML library that provides the same comment- and formatting-preservation guarantees with better performance and a smaller dependency footprint.
Key changes:
Dependency swap:
Core YAML I/O rewrite (src/usethis/file/yaml/io.py):
Pre-commit integration (src/usethis/_integrations/pre_commit/):
Deleted modules:
Test migration:
Documentation:
Resolves #1190