Skip to content

Bound or restructure releases/manifest.json growth (append-forever history) #1310

@Aaronontheweb

Description

@Aaronontheweb

Summary

feeds/scripts/generate-release-manifest.sh preserves every prior release in the releases[] array forever. The live manifest.json is already 68 KB across 64 releases (~1 KB/release) and grows monotonically. This isn't urgent, but it's append-forever by design and worth a deliberate strategy before it gets large.

Surfaced while designing the beta channel (#1027); we deliberately left it as-is there to keep that PR focused. The beta channel will accelerate growth (multiple x.y.z-beta.n entries per stable), which is why it's worth tracking now.

Why it matters (eventually, not yet)

  • install.sh no-jq fallback slurps the entire manifest (tr '\n' ' ' + grep) — gets slower/more fragile as the file grows.
  • C# update-check (UpdateCheckService.FetchVerifiedManifestAsync) downloads + signature-verifies + deserializes the whole file on each check (24h-cached, so low impact — but still the whole blob).
  • Monotonic growth is a smell; better to pick a bounded model on our terms.

Current numbers

  • 64 releases → 68 KB. Projected: ~200 releases ≈ 0.2 MB, ~500 ≈ 0.5 MB. With betas, the entry count climbs faster.

Ideas (not yet decided)

  1. Bound releases[] to a recent window (e.g. keep last N≈25), always retaining whatever latest and latestPrerelease reference even if they'd age out. Cheapest — fits inside the existing generator's Python. Binaries stay in R2 (netclaw-releases/{version}/…) and GitHub Releases permanently, so nothing is lost; the only effect is an old NETCLAW_VERSION=<ancient> pin can't resolve its sha256 from the hot manifest.
  2. Split hot vs cold: a small manifest.json (pointers + referenced entries + recent window) for the install/update hot path, plus a cold manifest-history.json (or per-version sidecars) fetched only on explicit old-version pins.
  3. Per-version checksum sidecars ({version}/checksums.json) so a pinned install verifies against the sidecar and the main manifest shrinks to pointers + minimal recent entries.
  4. Harden the no-jq parse path regardless of which option, so it doesn't depend on whole-file slurping.

Constraints to preserve

  • Keep schemaVersion: 1 backward-compatible (older clients ignore additive fields).
  • Keep minisign signature verification intact.
  • Don't break explicit pins for recent versions; immutable binaries remain available.

Related: #1027

Metadata

Metadata

Assignees

No one assigned

    Labels

    cleanupCode quality improvements and tech debt reductionenhancementNew feature or request

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions