fix(lockfile): preserve non-registry and bun platform entries#338
Conversation
f9948a0 to
f1787b1
Compare
Greptile SummaryThis PR fixes two lockfile compatibility bugs: npm's Confidence Score: 4/5Safe to merge; no P0/P1 issues found — changes are targeted, well-tested, and the round-trip for git resolved URLs is verified by both unit and integration tests. No critical or blocking defects identified. The canonical-map double-insertion for git packages (under both crates/aube-lockfile/src/npm.rs — specifically the canonical map insertion loop and Important Files Changed
Reviews (4): Last reviewed commit: "fix(lockfile): preserve non-registry and..." | Re-trigger Greptile |
f1787b1 to
78c5e34
Compare
78c5e34 to
a48393a
Compare
a48393a to
eecae7a
Compare
eecae7a to
45f5a25
Compare
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 45f5a25. Configure here.
## Summary - Preserve existing top-level `package.json` key order when commands rewrite dependency sections. - Parse Bun lockfile local tarball entries whose package ident omits the `file:` prefix. - Accept abbreviated git commit SHAs from existing Bun GitHub-shorthand lockfile entries during checkout verification. ## Details `aube add` already preserved manifest order by editing the parsed raw `package.json` object and syncing only dependency sections. This PR moves that targeted raw-JSON dependency-section writer into the shared command helpers so `add`, `remove`, and `update --latest` use the same order-preserving path. `remove` still prunes pnpm/aube sidecar metadata, but now applies those removals directly to the parsed raw JSON object instead of replacing unrelated fields from the typed `PackageJson.extra` map. That keeps existing key positions and avoids churn in unrelated or nested manifest data. The Bun regression matrix also exposed two plain-install failures not covered by the currently open lockfile PRs: - Bun can write local tarball package entries as `local-helper@tarballs/local-helper-1.0.0.tgz` while the workspace dependency remains `file:tarballs/local-helper-1.0.0.tgz`. Aube now recognizes that prefixless `.tgz` ident as `LocalSource::Tarball` instead of treating it as a registry version. - Bun records GitHub shorthand lock entries with abbreviated commit IDs. Aube's clone verifier now accepts a checked-out full SHA when it starts with the abbreviated lockfile SHA, while still rejecting non-hex refs and mismatched SHAs. I checked the other open lockfile PRs before adding these: #337 covers pnpm scalar platform fields, and #338 covers package-lock git resolved URLs plus pnpm/Bun scalar platform metadata. Those fixes are intentionally not duplicated here. ## Validation - `cargo fmt --check` - `cargo build` - `cargo test -p aube write_manifest_dep_sections` - `cargo test -p aube-lockfile test_parse_prefixless_local_tarball` - `cargo test -p aube-lockfile test_parse_github_dep` - `cargo test -p aube-store git_commit_matches_abbreviated_sha` - `mise run test:bats test/remove.bats` - `mise run test:bats test/update.bats` - Bun regression matrix from `johnpyp/2026-04-23-aube-bun-lock-regression-matrix` against `target/debug/aube`; `github-shorthand` and `local-tarball` now pass plain install as `plain-aube-unchanged` - pre-commit hook: cargo fmt and cargo clippy for staged Rust files <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Medium Risk** > Medium risk because it changes how `package.json` is rewritten across multiple commands and relaxes git checkout verification to accept abbreviated SHAs, which could affect correctness of dependency updates and git-sourced installs. > > **Overview** > Improves compatibility and reduces churn when working with Bun and when editing `package.json`. > > Commands that mutate dependencies (`add`, `remove`, `update --latest`) now update only the dependency sections in the existing parsed `package.json` object (via shared helpers) so **top-level key order is preserved** and empty dep sections are removed without reserializing unrelated fields; `remove` also prunes pnpm/aube sidecar metadata directly in raw JSON to avoid reordering. > > Bun lockfile parsing now treats prefixless local tarball idents (e.g. `pkg@tarballs/foo.tgz` without `file:`) as `LocalSource::Tarball`, and git dependency checkouts now accept **abbreviated hex SHAs** by verifying the full HEAD starts with the requested short SHA. Added targeted unit tests and a bats test to lock in these behaviors. > > <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit edcf920. Bugbot is set up for automated code reviews on this repo. Configure [here](https://www.cursor.com/dashboard/bugbot).</sup> <!-- /CURSOR_SUMMARY -->
45f5a25 to
a9c29ac
Compare
## Summary - For github / gitlab / bitbucket deps, re-derive an HTTPS URL from `(host, owner, repo, sha)` at fetch time instead of dialing whatever scheme the lockfile recorded. Matches npm `pacote` and pnpm `gitHostedTarballFetcher`. - SHA-pinned hosted reads go through `https://codeload.github.com/<owner>/<repo>/tar.gz/<sha>` (no `git` binary, no SSH key); on any HTTP error, fall back to a shallow `git clone` over the rewritten HTTPS URL so a system git credential helper (gh CLI etc.) keeps private repos working. - Branch / tag committishes are pinned via `git ls-remote` on the same rewritten HTTPS URL before reaching the codeload path. Non-hosted hosts (self-hosted GitLab / Gitea / arbitrary) still use the URL as written, preserving SSH-only setups. ## Why Reported in [discussion #335](#335) as a follow-up to #338. After #338 made `package-lock.json` git deps parse correctly, aube was dialing the lockfile-canonical `git+ssh://git@github.com/…#<sha>` directly, which fails for users with HTTPS-only git auth (e.g. `gh` CLI's git credential helper, no `~/.ssh/`). npm/pnpm never dial that SSH URL — it's an identity form, not a fetch URL. `LocalSource::Git.url` in the lockfile is preserved verbatim so cross-tool round-trip with pnpm / npm / yarn is unaffected. ## Test plan - [x] `cargo test -p aube-lockfile -p aube-store -p aube-resolver --lib` (459 unit tests, all green) - [x] `cargo clippy --all-targets -- -D warnings` - [x] `cargo fmt --check` - [x] `mise run test:bats test/git_deps.bats test/install.bats test/package_lock_write.bats` (87 / 87 pass — clone fallback path unchanged) - [x] 5 new unit tests covering: `parse_hosted_git` across all clone-URL forms (https / ssh / git+ / scp / `github:`), per-provider codeload URL synthesis, `extract_codeload_tarball` wrapper-strip + caching, defense against `..`/absolute symlink targets in tarball entries, short-commit rejection. 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Medium Risk** > Changes git dependency fetching/materialization and introduces new tarball extraction logic, which can affect install reliability and security if edge cases slip through. Mitigated by strict SHA gating, cache reuse, and explicit unsafe-path/symlink defenses with clone fallback. > > **Overview** > Git dependencies on GitHub/GitLab/Bitbucket are now resolved/installed using npm/pnpm-like *hosted routing*: derive a canonical `(host, owner, repo)` identity from the lockfile URL, pin refs via `git ls-remote` over derived HTTPS, then prefer fetching a provider-specific codeload-style tarball for full-SHA commits (with fallback to shallow `git clone`). > > This adds a new codeload extraction cache and hardened tarball extraction path in `aube-store` (wrapper-dir stripping, atomic cache population, entry/path/symlink validation, and Windows symlink handling), and wires the new flow through both the resolver (`resolve_git_source` now takes an optional `RegistryClient`) and installer while keeping the original lockfile `url` bytes unchanged for cross-tool round-tripping. > > <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit 8bd9198. Bugbot is set up for automated code reviews on this repo. Configure [here](https://www.cursor.com/dashboard/bugbot).</sup> <!-- /CURSOR_SUMMARY --> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Summary
resolvedURLs frompackage-lock.jsonentries asLocalSource::Gitinstead of registry packages.<name>@<version>from npm.resolvedfields with the npm-compatiblegit+prefix, while keeping unsupported non-git local sources out of package-lock output.os/cpu/libcmetadata in Bun lockfile package entries.Root Cause
The npm lockfile reader only preserved
resolvedvalues that looked like HTTP tarball URLs. Git resolved URLs such asgit+ssh://...#<sha>were dropped, leaving only<pkg>@<version>, which sent install down the npm registry tarball path. The npm writer also needed to preserve the external npm-compatiblegit+form when writing those pinned git sources back.The Bun reader assumed platform metadata was always serialized as arrays. Real Bun lockfiles can carry single platform values as scalars, for example
{ "os": "darwin", "cpu": "arm64" }, which caused parse failures or metadata loss.Note: the related pnpm scalar
os/cpu/libcfix is now onmainvia #337, so this PR keeps only the remaining package-lock and Bun fixes.Validation
cargo fmt --checkcargo test -p aube-lockfilecargo clippy --all-targets -- -D warningscargo buildmise run test:bats test/git_deps.bats