…252)

## Summary

Follow-up to #251. Pacquet's `StoreIndex::set` was writing `index.db` rows via `rmp_serde::to_vec_named` — plain MessagePack maps. pnpm reads the same file with `Packr({useRecords: true, moreTypes: true}).unpack(…)`, and in that mode **every msgpack map at any nesting level decodes as a JS `Map`**. Records are the escape hatch that tells msgpackr "this one's a plain object".

So pacquet-written rows came back as a top-level `Map`, `pkgIndex.files` (a property access on that `Map`) was `undefined`, and pnpm's `for (const [f, fstat] of pkgIndex.files)` threw `files is not iterable`. That surfaced in CI as `ERR_PNPM_READ_FROM_STORE` the first time a benchmark run picked up a shared pnpm-v11 cache with pacquet-written rows in it — every downstream job's `pnpm install` crashed trying to decode them.

#251 taught pacquet how to **read** pnpm's records; b0bed43 fixed the `checkedAt` encoding (float64 vs uint64-as-BigInt) and called the pnpm-reads-pacquet direction done — but the container shape was still wrong, and nothing in the test suite actually put a pacquet-written row back through msgpackr, so it slipped past.

## Fix

New `encode_package_files_index` in `msgpackr_records.rs` that emits msgpackr-records bytes. Behaviour matches what msgpackr 1.11.8 itself emits for the same logical input (verified against the version pinned in pnpm v11's `catalog:`):

- **Slot allocation is shape-aware.** Slot `0x40` is reserved for the top-level `PackageFilesIndex`. Inner slots in `0x41..=0x7f` are allocated lazily, one per distinct record shape, in first-seen order. A `CafsFileInfo` carrying `checkedAt` and one without it land in separate slots; same-shape instances downstream collapse to a single bare-slot byte, which is the record-compression win records exist for.
- **Optional fields are omitted from the schema when `None`**, not encoded as `nil`. A `SideEffectsDiff { added: Some, deleted: None }` decodes on the pnpm side to `{ added: Map }`, not `{ added: Map, deleted: null }` — the same JS shape msgpackr itself would produce for the same Rust input, so downstream checks like `diff.added != null` / `diff.deleted != null` behave identically regardless of which tool wrote the row.
- **`CafsFileInfo.checkedAt`** is written as `float 64` when `Some` (uint 64 would decode to a JS `BigInt` and crash pnpm's `mtimeMs - (checkedAt ?? 0)`), omitted from the schema when `None`.
- **`manifest: Some(_)` returns `EncodeError::ManifestNotSupported`.** Pacquet doesn't populate it today; encoding `serde_json::Value` as records is follow-up work.

The encoder also **promotes integer values in `0x40..=0x7f` to `uint 8`** instead of positive fixints. Inside a records stream those bytes are slot-reference markers — a bare `0x7b` would be parsed as a ref to slot `0x7b` rather than the integer `123`. msgpackr does the same promotion for the same reason.

State is a tight `[Option<u8>; 2]` for `CafsFileInfo` slots and `[Option<u8>; 4]` for `SideEffectsDiff` slots, indexed by a shape bitmask — 2 and 4 are the respective maxima, so lookup is a cache-local array index with no `HashMap` allocation on the hot per-file path.

`StoreIndex::set` now calls this encoder. `StoreIndex::get` goes through the existing transcoder which already handled records, so the read path needed no behaviour change — just a doc update to enumerate the three row flavours it now tolerates (pnpm-written records, current pacquet-written records, legacy pacquet plain-msgpack from rows that predate this PR).

## Why not a bespoke format?

Considered writing pacquet's rows in a different shape (positional tuple, or keep plain msgpack and switch pnpm's side). Ruled out: the whole point of #244 and #251 is a shared `index.db` the two tools read each other's entries from. Any shape other than "what pnpm's msgpackr emits" breaks that promise.

## Performance

Faster **and** smaller than `rmp_serde::to_vec_named`. Measured on a release build, 10k iterations per row-size, with a realistic `CafsFileInfo` shape (128-hex digest, millisecond timestamp):

**Serialize time per row**

| files | `rmp_serde` (before) | records encoder (after) | speedup |
|---|---:|---:|---:|
| 1 | 304 ns | 164 ns | **1.9×** |
| 10 | 1.23 µs | 838 ns | **1.5×** |
| 50 | 3.71 µs | 2.25 µs | **1.6×** |
| 200 | 11.9 µs | 8.11 µs | **1.5×** |
| 1000 | 55.9 µs | 27.7 µs | **2.0×** |

**Row size on disk**

| files | `rmp_serde` | records encoder | delta |
|---|---:|---:|---:|
| 1 | 214 B | 220 B | +6 B |
| 10 | 1960 B | 1723 B | **−12%** |
| 50 | 9722 B | 8405 B | **−14%** |
| 200 | 38822 B | 33455 B | **−14%** |
| 1000 | 194022 B | 167055 B | **−14%** |

The 6-byte regression at `n=1` is the outer record header (`d4 72 40`) plus a slightly longer field-name array; from `n≥4` records come out smaller because each subsequent `CafsFileInfo` instance collapses to a single slot byte instead of re-emitting `"digest"`, `"mode"`, `"size"`, `"checkedAt"` (~20 bytes of field names). Why it's faster: no serde runtime indirection (no trait dispatch, no generic-over-Serializer code paths), single pre-sized `Vec`, direct byte writes, zero-alloc slot lookup via the fixed-size arrays.

`StoreIndex::get` is unchanged, so read cost is identical to before. Tarball writes call `encode` once per package, so this shows up at ~10–50 µs/package — drowned out by the SQLite `INSERT` (~100–500 µs) either way, but the payload-size shrink compounds across thousands of packages in the shared v11 store.

## Test plan

- [x] `cargo test -p pacquet-store-dir --lib` — **37 passing, 12 new** in `msgpackr_records::tests`:
  - `encode_emits_record_header_for_top_level_struct` — outer `0xd4 0x72 0x40` record-def header is present.
  - `encode_roundtrips_single_file` / `encode_roundtrips_many_files_sharing_one_slot` — single-file round-trip; N uniform-shape `CafsFileInfo`s produce exactly 2 record defs (outer + one inner shape), the rest are bare slot refs.
  - `encode_handles_fixint_in_slot_range_safely` — `size: 0x7b` round-trips without triggering `UnknownSlot`.
  - `encode_omits_checked_at_when_none` — `None` `checkedAt` isn't written as `nil`; the string `"checkedAt"` doesn't appear anywhere in the output.
  - `encode_allocates_separate_slots_for_distinct_cafs_shapes` — mixed `CafsFileInfo` shapes (some with `checkedAt`, some without) land in separate slots; same shape reuses.
  - `encode_requires_build_when_set` / `encode_omits_requires_build_when_none` — outer-record optional-field handling.
  - `encode_side_effects_roundtrip` — `SideEffectsDiff` with both fields populated.
  - `encode_side_effects_with_only_added_omits_deleted_field` — exact shape the reviewer flagged: `deleted: None` must not appear as a field name in the output.
  - `encode_allocates_separate_slots_for_distinct_side_effects_shapes` — two `SideEffectsDiff`s with distinct shapes land in separate slots.
  - `encode_rejects_manifest_some` — error path for the currently-unimplemented branch.
- [x] `cargo test -p pacquet-cli --test pnpm_compatibility` — new `pnpm_reads_pacquet_written_rows` passes. pacquet installs into a store, `node_modules` is wiped, then `pnpm install --ignore-scripts` reinstalls against the same store. If the record encoding is wrong this reproduces the benchmark's `ERR_PNPM_READ_FROM_STORE`. Complements the existing `same_file_structure` / `same_index_file_contents` which clear the store between the pacquet and pnpm halves and therefore don't exercise this path.
- [x] `cargo clippy --workspace --all-targets -- -D warnings`
- [x] `cargo fmt --all -- --check`
- [x] `cargo doc -p pacquet-store-dir --no-deps` — no warnings.

Mirror the tracing change from the index-open call sites: when the
blocking cache-lookup task returns `Err(JoinError)`, emit a
`tracing::warn!` with the error and the `cache_key` before degrading
to `None`. Panic / cancellation stays diagnosable; the caller still
falls through to a fresh download.

Also drop the stale "msgpackr-records decoding is a follow-up"
wording from the cache-lookup rationale — `StoreIndex::get` has
decoded pnpm-written msgpackr-records since 0ea1252 (#251), and
`transcode_to_plain_msgpack` has handled it on the read side since
90be4f6 (#252). Simplify to "an undecodable entry … falls through
to the download path" so the comment reflects today's decoder.

…261)

* perf(store-dir): share one read-only StoreIndex across cache lookups

`DownloadTarballToStore::load_cached_cas_paths` previously opened a
fresh SQLite `Connection::open_with_flags` + `PRAGMA busy_timeout` on
every snapshot, so a 1352-package frozen-lockfile install paid 1352
connection-opens just to read cache hints — all serialized on tokio's
blocking pool because `spawn_blocking` can't parallelize opens against
one file.

Open a `StoreIndex` once per install, wrap it in `Arc<Mutex<…>>`
(rusqlite `Connection` is `Send` but not `Sync`), and thread the handle
through `CreateVirtualStore`, `InstallWithoutLockfile`, and
`DownloadTarballToStore`. Each lookup now takes the mutex just for the
sub-millisecond `SELECT`, releases it before the per-file validation
pass, and frees the Arc when the install finishes.

Measured on the #260 repro (1352 packages, release build, macOS APFS,
warm CAS store): `rm -rf node_modules && pacquet install --frozen-
lockfile` drops from ~17.4 s to ~13.9 s, with CPU utilization rising
from ~135 % to ~160 % as the serialized-open bottleneck clears.

* address review feedback

* `create_virtual_store.rs`: bind `store_index.as_ref()` to a local and
  use `async move` so the closure's copy of the `Option<&…>` is moved
  into each future, rather than borrowing from the closure's
  per-iteration scope.
* `load_cached_cas_paths`: treat a poisoned index mutex as a cache miss
  rather than propagating the panic. The `SELECT` is stateless, so the
  prior panic can't have left the index in an inconsistent shape, and
  cache lookups are a best-effort hint — falling through to a fresh
  download is strictly safer than crashing every subsequent snapshot.

* park StoreIndex open on the blocking pool

Copilot flagged that `StoreIndex::shared_readonly_in` does synchronous
SQLite work (`Connection::open_with_flags` + `PRAGMA busy_timeout`) and
calling it directly in an `async fn` stalls the tokio reactor thread
for the duration of the open. It's once per install, not a hot spot,
but matches the pattern the store-index writer already uses and costs
us nothing to keep consistent.

Wrap both call sites (frozen-lockfile and no-lockfile install paths)
in `tokio::task::spawn_blocking` and await the result before kicking
off concurrent package work. Also moves `tokio` from `dev-dependencies`
into `dependencies` in package-manager's `Cargo.toml` since it's now a
runtime dep for the library code, not just tests.

* degrade JoinError on StoreIndex open to a cache miss

`spawn_blocking(...).await` returns `Result<_, JoinError>` which fires
on blocking-task panic or on cancellation during runtime shutdown.
Panicking on that turns a transient runtime issue into a hard crash
for an install that could otherwise succeed — cache lookups just miss.

Drop the `.expect(...)` and degrade to `None` via `.ok().flatten()`.
`shared_readonly_in` already yields `None` for a first-time install
against an empty store, and every downstream caller already handles
that shape, so this is the same failure mode the code already copes
with — just reached from a different source.

* use async move in install_dependencies_from_registry map closure

Makes the recursive no-lockfile closure's captures explicit: bind
`&node_modules_path` to a local `node_modules_path_ref` (Copy) and
switch to `async move` so every captured variable is moved (copied,
for all the `&T` and `Option<&T>` captures) into the future instead
of borrowing the closure's per-iteration stack frame.

Matches the pattern already used in `create_virtual_store.rs` and in
the outer no-lockfile closure — keeps all three parallel-map sites
consistent.

* log JoinError before degrading to cache miss on StoreIndex open

The silent `.ok().flatten()` hid two failure modes behind a cache miss:
a panic inside the blocking task, and task cancellation during runtime
shutdown. Neither is a crash-worthy error, but a blocking-task panic
during install really should be visible somewhere.

Promote the two call sites to a `match` and emit a `tracing::warn!` on
the `Err(JoinError)` branch before falling back to `None`. Uses the
`pacquet::install` target in common with the rest of this crate's
tracing. Degradation path is unchanged.

* chore: cargo fmt

* log JoinError in tarball cache lookup, refresh stale comment

Mirror the tracing change from the index-open call sites: when the
blocking cache-lookup task returns `Err(JoinError)`, emit a
`tracing::warn!` with the error and the `cache_key` before degrading
to `None`. Panic / cancellation stays diagnosable; the caller still
falls through to a fresh download.

Also drop the stale "msgpackr-records decoding is a follow-up"
wording from the cache-lookup rationale — `StoreIndex::get` has
decoded pnpm-written msgpackr-records since 0ea1252 (#251), and
`transcode_to_plain_msgpack` has handled it on the read side since
90be4f6 (#252). Simplify to "an undecodable entry … falls through
to the download path" so the comment reflects today's decoder.