Skip to content

fix(pack): bundle download and host packages in Linux AppImage assembly#2845

Merged
Siri-Ray merged 1 commit into
nexu-io:mainfrom
youcefzemmar:fix/linux-appimage-bundle-download-host
May 29, 2026
Merged

fix(pack): bundle download and host packages in Linux AppImage assembly#2845
Siri-Ray merged 1 commit into
nexu-io:mainfrom
youcefzemmar:fix/linux-appimage-bundle-download-host

Conversation

@youcefzemmar

@youcefzemmar youcefzemmar commented May 24, 2026

Copy link
Copy Markdown
Contributor

Fixes #728

Why

pnpm exec tools-pack linux build --to appimage fails on main with a 404 from the public npm registry, so a Linux desktop AppImage cannot be produced from source. #728 (help wanted) explicitly asks contributors to run exactly this command on latest main and report whether the AppImage builds — this is the next blocker after #2276.

Root cause. The Linux path assembles INTERNAL_PACKAGES as file: tarballs and runs npm install --omit=dev in an isolated app directory. pnpm pack rewrites each tarball's workspace:* refs to a concrete version, so the install resolves each package's runtime @open-design/* deps. Any such dep not itself in INTERNAL_PACKAGES is fetched from the registry and 404s (these packages are workspace-only).

Linux ships webOutputMode: "server" and tarball-installs every INTERNAL_PACKAGES entry, including @open-design/desktop and @open-design/web. @open-design/host (dep of web + desktop, added in #2246) and @open-design/download (dep of desktop, added in #2677) landed after the Linux list was written and were never added:

npm error code E404
npm error 404 Not Found - GET https://registry.npmjs.org/@open-design%2fdownload

Why this is Linux-only (scope rationale). mac/win default to webOutputMode: "standalone", where desktop/web/packaged/daemon are prebundled with esbuild and excluded from the tarball install (shouldInstallInternalPackageFor{Mac,Win}PrebundleMAC/WIN_STANDALONE_PREBUNDLE_EXCLUDED_INTERNAL_PACKAGES). The packages mac/win do tarball-install (contracts, registry-protocol, agui-adapter, plugin-runtime, diagnostics) have no download/host dependency, so those lanes correctly omit them. This fix therefore stays scoped to linux.ts and touches no mac/win constants and no workspace-build.ts cache.

What users will see

pnpm exec tools-pack linux build --to appimage completes and produces a working Open Design-<namespace>.AppImage from source on Linux. No change to packaged app behavior.

Surface area

  • None — internal build tooling (tools/pack) + a regression test. No UI, API/contract, CLI flag, dependency, or default-behavior change.

Screenshots

N/A — no UI change.

Bug fix verification

  • Test: tools/pack/tests/internal-packages-closure.test.ts — for each pack lane it derives the set the lane actually installs (honoring the standalone prebundle exclusion on mac/win) and asserts that set is closed under its runtime @open-design/* dependencies.
  • Red on main, green on branch? yes — and lane-precise. Reverting the two INTERNAL_PACKAGES additions makes only the linux case fail, naming the missing edges, while mac and win still pass:
    × linux: every installed package's runtime @open-design deps are installed
      [ { dependency: '@open-design/host',     dependent: '@open-design/web' },
        { dependency: '@open-design/download', dependent: '@open-design/desktop' },
        { dependency: '@open-design/host',     dependent: '@open-design/desktop' } ]
    ✓ mac:  ...
    ✓ win:  ...
    
    This encodes why only Linux needs these packages and guards every lane against the next workspace dependency silently breaking its assembly.

Validation

Native (non-containerized) build, Ubuntu / Node 24.15.0 / pnpm 10.33.2:

  • pnpm exec tools-pack linux build --to appimage → produces Open Design-default.AppImage (~277 MB); assembled node_modules/@open-design/ contains all 14 packages incl. download + host.
  • Runtime smoke (headless, no display server): tools-pack linux install --headless + start --headless → packaged daemon + web boot, GET / and GET /api/health return 200, stop --headless clean.
  • Extracted AppImage contains the Electron binary, resources/app, and all extraResources + open-design-config.json.
  • pnpm --filter @open-design/tools-pack test132 passed (incl. the new closure test, all 3 lanes).
  • pnpm --filter @open-design/tools-pack typecheck → clean. pnpm guard → pass.

@lefarcen lefarcen requested a review from nettee May 24, 2026 22:05
@lefarcen lefarcen added size/S PR changes 20-100 lines risk/medium Medium risk: regular code changes type/bugfix Bug fix labels May 24, 2026
@lefarcen lefarcen mentioned this pull request May 24, 2026
@lefarcen

Copy link
Copy Markdown
Contributor

Heads-up: PR #2837 is also open against this same Linux AppImage packaging blocker. Both PRs touch tools/pack/src/linux.ts and add @open-design/download / @open-design/host so the isolated Linux assembly stops resolving those local runtime packages from npm.

You and @Echo9k may want to compare approaches; the maintainer team will decide which path lands. Sharing this so neither effort gets wasted.

The Linux AppImage path assembles INTERNAL_PACKAGES as `file:` tarballs
and runs `npm install --omit=dev` in an isolated app directory. `pnpm
pack` rewrites each tarball's `workspace:*` refs to a concrete version,
so any runtime @open-design/* dependency missing from INTERNAL_PACKAGES
is resolved from the public npm registry and 404s.

Linux ships webOutputMode "server" and tarball-installs every
INTERNAL_PACKAGES entry, including @open-design/desktop and
@open-design/web. @open-design/host (dep of web + desktop, added in
nexu-io#2246) and @open-design/download (dep of desktop, added in nexu-io#2677) landed
after the Linux package list was written and were never added to it, so
`pnpm exec tools-pack linux build --to appimage` fails with:

  npm error 404 Not Found - GET .../@open-design%2fdownload

mac/win default to "standalone", where desktop/web/packaged/daemon are
prebundled with esbuild and excluded from the tarball install
(shouldInstallInternalPackageFor{Mac,Win}Prebundle). The packages they
do install have no download/host dependency, so those lanes correctly
omit them and need no change — this fix stays scoped to linux.ts and
touches no mac/win or workspace-build code.

Add both packages to the Linux INTERNAL_PACKAGES and build them in
buildWorkspaceArtifacts (download depends on platform). Add a cross-lane
regression test that, for each lane, derives the set it actually installs
(honoring the standalone prebundle exclusion) and asserts that set is
closed under its runtime @open-design/* dependencies. The test is red on
the linux lane without this fix and green with it, while mac/win pass
either way — encoding why only Linux needs these packages.
@youcefzemmar youcefzemmar force-pushed the fix/linux-appimage-bundle-download-host branch from 0be2414 to 1139446 Compare May 24, 2026 22:42
@youcefzemmar

Copy link
Copy Markdown
Contributor Author

Thanks @lefarcen for flagging the overlap with #2837, and credit to @Echo9k for independently hitting the same root cause. Sharing some analysis on the scope question, since it's the main difference between the two PRs and it also explains the blocking review on #2837.

Do mac/win actually need download/host in INTERNAL_PACKAGES? I dug into the lane internals — the answer is no, and it's load-bearing:

  • mac/win default to webOutputMode: "standalone". In that mode shouldInstallInternalPackageFor{Mac,Win}Prebundle excludes @open-design/{daemon,desktop,packaged,platform,sidecar,sidecar-proto,web} from the tarball install (MAC/WIN_STANDALONE_PREBUNDLE_EXCLUDED_INTERNAL_PACKAGES) — they're prebundled with esbuild instead.
  • desktop (→ download, host) and web (→ host) are in that excluded set, so their package.json deps are never npm-resolved on mac/win. The packages mac/win do tarball-install — contracts, registry-protocol, agui-adapter, plugin-runtime, diagnostics — have no download/host dependency.
  • Linux ships webOutputMode: "server", so the exclusion doesn't apply and it tarball-installs desktop/web → it is the only lane that needs download/host.

A consequence worth calling out: because mac/win don't tarball-install these, adding download/host to their INTERNAL_PACKAGES also pulls in the shared workspace-build.ts cache (its WORKSPACE_BUILD_PACKAGES/output-file/artifact lists), which is exactly the blocking gap @nettee found on #2837. The Linux lane uses its own buildWorkspaceArtifacts and never touches that cache, so the Linux-only fix sidesteps it entirely.

I encoded this as a cross-lane invariant in internal-packages-closure.test.ts: for each lane it derives the set actually installed (honoring the standalone exclusion) and asserts it's closed under its runtime @open-design/* deps. Reverting the fix makes only the linux case go red (web→host, desktop→download, desktop→host) while mac and win stay green — which is the drift-prevention coverage the review asked for, scoped correctly per lane.

Net: this PR keeps the change to tools/pack/src/linux.ts + one test, with no mac/win or cache churn. Totally happy to defer to whichever path the team prefers — just wanted the lane analysis on record so the decision is easy and neither effort is wasted. 🙌

@nettee nettee left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for narrowing the Linux AppImage failure and adding regression coverage. I found one blocking gap: the new closure test hard-codes mac/win to standalone mode, but server mode is still accepted on those lanes and remains broken for the same missing @open-design/host / @open-design/download closure.

🔁 Powered by Looper · runner=reviewer · agent=opencode · An autonomous AI dev team for your GitHub repos.

Comment thread tools/pack/tests/internal-packages-closure.test.ts
@youcefzemmar

Copy link
Copy Markdown
Contributor Author

Good catch @nettee — you're right. mac/win can be forced into server mode (OD_WEB_OUTPUT_MODE=server), and in that mode they tarball-install desktop/web and would hit the same missing-closure (download/host) this PR fixes for Linux. My test asserts closure only under each lane's shipping default (linux=server, mac/win=standalone), so it doesn't surface the mac/win+server case.

For scope, worth noting that gap is pre-existing — mac/win ship standalone; server is the transitional mode only Linux still uses (per the comment in config.ts), not something this PR introduces. Closing it is a real design decision rather than a one-liner, and it splits two ways:

  1. Support mac/win+server → add @open-design/download + @open-design/host to the mac/win INTERNAL_PACKAGES and to workspace-build.ts (WORKSPACE_BUILD_PACKAGES / workspaceBuildOutputFiles / workspaceBuildArtifacts). On mac/win those two aren't in the prebundle exclusion list, so they'd then be tarball-installed even in standalone — which is exactly why the shared workspace-build cache has to come along (the same gap you flagged as blocking on fix(pack): add missing download and host packages to Linux INTERNAL_PACKAGES #2837), and it changes the shipping mac/win packaging.
  2. Treat server as Linux-only → guard resolveToolPackWebOutputMode to reject server on mac/win, eliminating the broken-but-accepted config.

I'd rather not guess between those: option 1 changes the mac/win build path, and I can't validate a mac/win build in my environment, so I don't want to land it unverified (and it pulls this well-scoped Linux fix into a cross-platform refactor). Happy to implement whichever direction the team prefers, or keep this PR scoped to the verified Linux fix and track mac/win+server hardening separately. Just point me at the path and I'll turn it around.

@youcefzemmar youcefzemmar requested a review from nettee May 26, 2026 01:40
@lefarcen

Copy link
Copy Markdown
Contributor

Hey @nettee, quick nudge when you have a moment — @youcefzemmar has laid out the two possible directions after your blocking review: either support mac/win server mode by extending the package/build closure, or treat server as Linux-only and reject it for mac/win at config time.

Since the PR is currently blocked on that scope call rather than another author change, could you confirm which path you prefer?

@Siri-Ray Siri-Ray left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@youcefzemmar I reviewed the Linux AppImage package list/build-step changes and the new internal package closure regression coverage. The added @open-design/download and @open-design/host entries line up with the runtime workspace dependencies pulled in by the Linux server-mode assembly, and the build order covers their dist outputs before desktop/web packaging. I did not find any changed-range issues that need follow-up here. Thanks for the clear root-cause write-up and the focused regression test. 🙌

🔁 Powered by Looper · runner=reviewer · agent=codex · An autonomous AI dev team for your GitHub repos.

@nettee nettee left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@youcefzemmar I’m approving this Linux-scoped fix. The added @open-design/download and @open-design/host Linux tarballs line up with the runtime workspace deps pulled in by the AppImage server-mode assembly, and the explicit build steps cover their dist outputs before desktop/web packaging.

The mac/win OD_WEB_OUTPUT_MODE=server closure gap that @nettee flagged is still real, but I agree with your scope call: it predates this PR, affects a non-default path, and needs an explicit maintainer decision rather than being folded into the Linux AppImage unblocker. I opened #3259 to track that follow-up: either reject server mode on mac/win or fully support its package/build closure.

Thanks for keeping the fix focused and for spelling out the lane behavior clearly.

@Siri-Ray Siri-Ray added this pull request to the merge queue May 29, 2026
@github-merge-queue github-merge-queue Bot removed this pull request from the merge queue due to failed status checks May 29, 2026
@Siri-Ray Siri-Ray added this pull request to the merge queue May 29, 2026
Merged via the queue into nexu-io:main with commit d6d42c3 May 29, 2026
13 checks passed
@open-design-bot

Copy link
Copy Markdown
Contributor

🎉 📡 You just leveled up to Giotto

Giotto card for @youcefzemmar

📡 ✨ Sending steady signals.

🙌 Your contributions send a clear signal across the network: you care about making Open Design better.

💛 Thanks for helping Open Design move forward. Keep building in the open. 🚀


📊 Rank #198 among 198 contributors.

🔗 Share on X (English) · 分享到 X(中文)

AmyShang-alt pushed a commit that referenced this pull request May 29, 2026
…ly (#2845)

The Linux AppImage path assembles INTERNAL_PACKAGES as `file:` tarballs
and runs `npm install --omit=dev` in an isolated app directory. `pnpm
pack` rewrites each tarball's `workspace:*` refs to a concrete version,
so any runtime @open-design/* dependency missing from INTERNAL_PACKAGES
is resolved from the public npm registry and 404s.

Linux ships webOutputMode "server" and tarball-installs every
INTERNAL_PACKAGES entry, including @open-design/desktop and
@open-design/web. @open-design/host (dep of web + desktop, added in
#2246) and @open-design/download (dep of desktop, added in #2677) landed
after the Linux package list was written and were never added to it, so
`pnpm exec tools-pack linux build --to appimage` fails with:

  npm error 404 Not Found - GET .../@open-design%2fdownload

mac/win default to "standalone", where desktop/web/packaged/daemon are
prebundled with esbuild and excluded from the tarball install
(shouldInstallInternalPackageFor{Mac,Win}Prebundle). The packages they
do install have no download/host dependency, so those lanes correctly
omit them and need no change — this fix stays scoped to linux.ts and
touches no mac/win or workspace-build code.

Add both packages to the Linux INTERNAL_PACKAGES and build them in
buildWorkspaceArtifacts (download depends on platform). Add a cross-lane
regression test that, for each lane, derives the set it actually installs
(honoring the standalone prebundle exclusion) and asserts that set is
closed under its runtime @open-design/* dependencies. The test is red on
the linux lane without this fix and green with it, while mac/win pass
either way — encoding why only Linux needs these packages.
@youcefzemmar

Copy link
Copy Markdown
Contributor Author

hey @leilei926524-tech check #3387

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

risk/medium Medium risk: regular code changes size/S PR changes 20-100 lines type/bugfix Bug fix

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Linux version

4 participants