fix(pack): bundle download and host packages in Linux AppImage assembly#2845
Conversation
|
Heads-up: PR #2837 is also open against this same Linux AppImage packaging blocker. Both PRs touch You and @Echo9k may want to compare approaches; the maintainer team will decide which path lands. Sharing this so neither effort gets wasted. |
The Linux AppImage path assembles INTERNAL_PACKAGES as `file:` tarballs and runs `npm install --omit=dev` in an isolated app directory. `pnpm pack` rewrites each tarball's `workspace:*` refs to a concrete version, so any runtime @open-design/* dependency missing from INTERNAL_PACKAGES is resolved from the public npm registry and 404s. Linux ships webOutputMode "server" and tarball-installs every INTERNAL_PACKAGES entry, including @open-design/desktop and @open-design/web. @open-design/host (dep of web + desktop, added in nexu-io#2246) and @open-design/download (dep of desktop, added in nexu-io#2677) landed after the Linux package list was written and were never added to it, so `pnpm exec tools-pack linux build --to appimage` fails with: npm error 404 Not Found - GET .../@open-design%2fdownload mac/win default to "standalone", where desktop/web/packaged/daemon are prebundled with esbuild and excluded from the tarball install (shouldInstallInternalPackageFor{Mac,Win}Prebundle). The packages they do install have no download/host dependency, so those lanes correctly omit them and need no change — this fix stays scoped to linux.ts and touches no mac/win or workspace-build code. Add both packages to the Linux INTERNAL_PACKAGES and build them in buildWorkspaceArtifacts (download depends on platform). Add a cross-lane regression test that, for each lane, derives the set it actually installs (honoring the standalone prebundle exclusion) and asserts that set is closed under its runtime @open-design/* dependencies. The test is red on the linux lane without this fix and green with it, while mac/win pass either way — encoding why only Linux needs these packages.
0be2414 to
1139446
Compare
|
Thanks @lefarcen for flagging the overlap with #2837, and credit to @Echo9k for independently hitting the same root cause. Sharing some analysis on the scope question, since it's the main difference between the two PRs and it also explains the blocking review on #2837. Do mac/win actually need
A consequence worth calling out: because mac/win don't tarball-install these, adding I encoded this as a cross-lane invariant in Net: this PR keeps the change to |
nettee
left a comment
There was a problem hiding this comment.
Thanks for narrowing the Linux AppImage failure and adding regression coverage. I found one blocking gap: the new closure test hard-codes mac/win to standalone mode, but server mode is still accepted on those lanes and remains broken for the same missing @open-design/host / @open-design/download closure.
|
Good catch @nettee — you're right. mac/win can be forced into For scope, worth noting that gap is pre-existing — mac/win ship
I'd rather not guess between those: option 1 changes the mac/win build path, and I can't validate a mac/win build in my environment, so I don't want to land it unverified (and it pulls this well-scoped Linux fix into a cross-platform refactor). Happy to implement whichever direction the team prefers, or keep this PR scoped to the verified Linux fix and track mac/win+ |
|
Hey @nettee, quick nudge when you have a moment — @youcefzemmar has laid out the two possible directions after your blocking review: either support mac/win Since the PR is currently blocked on that scope call rather than another author change, could you confirm which path you prefer? |
Siri-Ray
left a comment
There was a problem hiding this comment.
@youcefzemmar I reviewed the Linux AppImage package list/build-step changes and the new internal package closure regression coverage. The added @open-design/download and @open-design/host entries line up with the runtime workspace dependencies pulled in by the Linux server-mode assembly, and the build order covers their dist outputs before desktop/web packaging. I did not find any changed-range issues that need follow-up here. Thanks for the clear root-cause write-up and the focused regression test. 🙌
nettee
left a comment
There was a problem hiding this comment.
@youcefzemmar I’m approving this Linux-scoped fix. The added @open-design/download and @open-design/host Linux tarballs line up with the runtime workspace deps pulled in by the AppImage server-mode assembly, and the explicit build steps cover their dist outputs before desktop/web packaging.
The mac/win OD_WEB_OUTPUT_MODE=server closure gap that @nettee flagged is still real, but I agree with your scope call: it predates this PR, affects a non-default path, and needs an explicit maintainer decision rather than being folded into the Linux AppImage unblocker. I opened #3259 to track that follow-up: either reject server mode on mac/win or fully support its package/build closure.
Thanks for keeping the fix focused and for spelling out the lane behavior clearly.
🎉 📡 You just leveled up to Giotto
🙌 Your contributions send a clear signal across the network: you care about making Open Design better. 💛 Thanks for helping Open Design move forward. Keep building in the open. 🚀 📊 Rank #198 among 198 contributors. |
…ly (#2845) The Linux AppImage path assembles INTERNAL_PACKAGES as `file:` tarballs and runs `npm install --omit=dev` in an isolated app directory. `pnpm pack` rewrites each tarball's `workspace:*` refs to a concrete version, so any runtime @open-design/* dependency missing from INTERNAL_PACKAGES is resolved from the public npm registry and 404s. Linux ships webOutputMode "server" and tarball-installs every INTERNAL_PACKAGES entry, including @open-design/desktop and @open-design/web. @open-design/host (dep of web + desktop, added in #2246) and @open-design/download (dep of desktop, added in #2677) landed after the Linux package list was written and were never added to it, so `pnpm exec tools-pack linux build --to appimage` fails with: npm error 404 Not Found - GET .../@open-design%2fdownload mac/win default to "standalone", where desktop/web/packaged/daemon are prebundled with esbuild and excluded from the tarball install (shouldInstallInternalPackageFor{Mac,Win}Prebundle). The packages they do install have no download/host dependency, so those lanes correctly omit them and need no change — this fix stays scoped to linux.ts and touches no mac/win or workspace-build code. Add both packages to the Linux INTERNAL_PACKAGES and build them in buildWorkspaceArtifacts (download depends on platform). Add a cross-lane regression test that, for each lane, derives the set it actually installs (honoring the standalone prebundle exclusion) and asserts that set is closed under its runtime @open-design/* dependencies. The test is red on the linux lane without this fix and green with it, while mac/win pass either way — encoding why only Linux needs these packages.
|
hey @leilei926524-tech check #3387 |
Fixes #728
Why
pnpm exec tools-pack linux build --to appimagefails onmainwith a 404 from the public npm registry, so a Linux desktop AppImage cannot be produced from source. #728 (help wanted) explicitly asks contributors to run exactly this command on latestmainand report whether the AppImage builds — this is the next blocker after #2276.Root cause. The Linux path assembles
INTERNAL_PACKAGESasfile:tarballs and runsnpm install --omit=devin an isolated app directory.pnpm packrewrites each tarball'sworkspace:*refs to a concrete version, so the install resolves each package's runtime@open-design/*deps. Any such dep not itself inINTERNAL_PACKAGESis fetched from the registry and 404s (these packages are workspace-only).Linux ships
webOutputMode: "server"and tarball-installs everyINTERNAL_PACKAGESentry, including@open-design/desktopand@open-design/web.@open-design/host(dep of web + desktop, added in #2246) and@open-design/download(dep of desktop, added in #2677) landed after the Linux list was written and were never added:Why this is Linux-only (scope rationale). mac/win default to
webOutputMode: "standalone", wheredesktop/web/packaged/daemonare prebundled with esbuild and excluded from the tarball install (shouldInstallInternalPackageFor{Mac,Win}Prebundle→MAC/WIN_STANDALONE_PREBUNDLE_EXCLUDED_INTERNAL_PACKAGES). The packages mac/win do tarball-install (contracts,registry-protocol,agui-adapter,plugin-runtime,diagnostics) have nodownload/hostdependency, so those lanes correctly omit them. This fix therefore stays scoped tolinux.tsand touches no mac/win constants and noworkspace-build.tscache.What users will see
pnpm exec tools-pack linux build --to appimagecompletes and produces a workingOpen Design-<namespace>.AppImagefrom source on Linux. No change to packaged app behavior.Surface area
tools/pack) + a regression test. No UI, API/contract, CLI flag, dependency, or default-behavior change.Screenshots
N/A — no UI change.
Bug fix verification
tools/pack/tests/internal-packages-closure.test.ts— for each pack lane it derives the set the lane actually installs (honoring the standalone prebundle exclusion on mac/win) and asserts that set is closed under its runtime@open-design/*dependencies.main, green on branch? yes — and lane-precise. Reverting the twoINTERNAL_PACKAGESadditions makes only the linux case fail, naming the missing edges, while mac and win still pass:Validation
Native (non-containerized) build, Ubuntu / Node 24.15.0 / pnpm 10.33.2:
pnpm exec tools-pack linux build --to appimage→ producesOpen Design-default.AppImage(~277 MB); assemblednode_modules/@open-design/contains all 14 packages incl.download+host.tools-pack linux install --headless+start --headless→ packaged daemon + web boot,GET /andGET /api/healthreturn 200,stop --headlessclean.resources/app, and allextraResources+open-design-config.json.pnpm --filter @open-design/tools-pack test→ 132 passed (incl. the new closure test, all 3 lanes).pnpm --filter @open-design/tools-pack typecheck→ clean.pnpm guard→ pass.