Skip to content

ci(release): build artifacts on macos-latest to fix darwin-x64 signing#11415

Merged
zkochan merged 2 commits into
mainfrom
segment-error
May 1, 2026
Merged

ci(release): build artifacts on macos-latest to fix darwin-x64 signing#11415
zkochan merged 2 commits into
mainfrom
segment-error

Conversation

@zkochan

@zkochan zkochan commented Apr 30, 2026

Copy link
Copy Markdown
Member

Summary

Fixes EXC_BAD_ACCESS at startup of the darwin-x64 release binary (reported against v11.0.2/v11.0.3 on Intel Macs).

The root cause is in the cross-signing step, not pnpm code. The release flow:

  1. On Linux CI, pnpm pack-app downloads the official darwin-x64 Node.js 25.9.0 and uses postject to splice the SEA blob into a Mach-O segment.
  2. The modified binary is then ad-hoc signed with ldid -S from the (saurik fork, pinned to c2f8abf01…).

Node.js 25 binaries use LC_DYLD_CHAINED_FIXUPS. When postject mutates the binary and the saurik fork of ldid recomputes the ad-hoc signature, the resulting LC_CODE_SIGNATURE page hashes don't match what the kernel sees on macOS 11+. dyld validates pages, skips fixup application on the offending pages, and a C++ global initializer reads a raw chained-fixup chain entry as a pointer — landing on address=0x3, which is exactly the low-bit "next-offset" pattern of an unprocessed chain entry.

A reporter's lldb trace makes this unambiguous:

EXC_BAD_ACCESS (code=1, address=0x3)
frame #0: pnpm`__cxx_global_var_init
frame #1: dyld`...findAndRunAllInitializers
frame #2: dyld`...forEachInitializer

Fix

Move the entire release job onto macos-latest. pack-app's adHocSignMacBinary (releasing/commands/src/pack-app/packApp.ts:555-557) already uses native codesign --sign - when running on darwin, which knows how to compute correct page hashes against post-postject Mach-O including chained-fixup data. No application code changes are required.

Side-effects:

  • The entire Install ldid step (apt-get + GitLab clone + g++ build) is removed.
  • macos-latest is Apple Silicon, so buildFullMatrix in build-artifacts.ts:16 evaluates true — same 8-target matrix as today.
  • Linux/Windows artifacts cross-build the same way (postject is byte-level and platform-neutral; those targets need no signing).
  • verify-binary.mjs's execution check shifts from linux-x64 to darwin-arm64. Existence verification still runs for all targets.
  • macOS runners are slower than Ubuntu, so the release job will take longer. Acceptable cost.

Test plan

  • Tag a vX.Y.Z-rc.N to exercise the workflow end-to-end on macos-latest.
  • Download the resulting pnpm-darwin-x64.tar.gz and confirm ./pnpm --version runs without segfaulting on an Intel Mac.
  • codesign -dv --verbose=4 ./pnpm shows a valid ad-hoc signature with hashes matching the file.
  • Confirm pnpm-linux-x64.tar.gz, pnpm-linux-arm64.tar.gz, pnpm-linux-x64-musl.tar.gz, pnpm-linux-arm64-musl.tar.gz, pnpm-darwin-arm64.tar.gz, pnpm-win32-x64.zip, pnpm-win32-arm64.zip are all produced.
  • OIDC provenance attestations are still attached to the published npm packages.

Summary by CodeRabbit

  • Chores
    • Updated release workflow to optimize build and signing processes for improved artifact distribution.

zkochan added 2 commits May 1, 2026 00:45
Cross-signing darwin Mach-O binaries on Linux with the saurik fork of
ldid produces an ad-hoc signature whose page hashes don't match the
post-postject layout for Node.js 25's chained fixups, leaving fixups
unapplied at load and crashing the binary in __cxx_global_var_init
(EXC_BAD_ACCESS at 0x3 — the unprocessed chain-entry tag).

Running the release on macos-latest lets pack-app's adHocSignMacBinary
use native codesign, which understands chained fixups. Drops the entire
ldid build step.
@zkochan

zkochan commented May 1, 2026

Copy link
Copy Markdown
Member Author

@coderabbitai full review

@coderabbitai

coderabbitai Bot commented May 1, 2026

Copy link
Copy Markdown
✅ Actions performed

Full review triggered.

@coderabbitai

coderabbitai Bot commented May 1, 2026

Copy link
Copy Markdown

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 87ae492d-28d9-4b37-931b-bf54c2458277

📥 Commits

Reviewing files that changed from the base of the PR and between 4bf61ba and 0a72e0b.

📒 Files selected for processing (1)
  • .github/workflows/release.yml

📝 Walkthrough

Walkthrough

The release workflow's runner environment switches from Ubuntu to macOS with inline comments explaining Darwin artifact signing. The Linux-specific ldid installation step is removed, while other build and publishing steps remain unaffected.

Changes

Cohort / File(s) Summary
GitHub Workflows Configuration
.github/workflows/release.yml
Changed CI runner from ubuntu-latest to macos-latest, removed Linux-only ldid tooling step, and added inline documentation for Darwin signing behavior.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~12 minutes

Poem

🐰 The workflow hops from Linux land,
To Apple's macOS, shiny and grand,
No more ldid in sight,
Just Darwin signing, done right,
A rabbit-approved release so planned! 🍎

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately describes the main change: migrating the release job from ubuntu-latest to macos-latest specifically to fix darwin-x64 signing issues.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch segment-error

Review rate limit: 9/10 reviews remaining, refill in 6 minutes.

Comment @coderabbitai help to get the list of available commands and usage tips.

@zkochan zkochan marked this pull request as ready for review May 1, 2026 19:28
Copilot AI review requested due to automatic review settings May 1, 2026 19:28

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Updates the GitHub Release workflow to run on macOS so the darwin artifacts are re-signed using native codesign, addressing the reported darwin-x64 startup crash caused by invalid ad-hoc signatures produced during Linux cross-signing.

Changes:

  • Switch the release job runner from ubuntu-latest to macos-latest.
  • Remove the Linux-only ldid installation/build step and document the signing rationale inline.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@zkochan zkochan merged commit d374e33 into main May 1, 2026
17 checks passed
@zkochan zkochan deleted the segment-error branch May 1, 2026 19:54
@zkochan zkochan added this to the v11.0 milestone May 2, 2026
zkochan added a commit that referenced this pull request May 2, 2026
#11415)

* ci(release): build artifacts on macos-latest to fix darwin-x64 signing

Cross-signing darwin Mach-O binaries on Linux with the saurik fork of
ldid produces an ad-hoc signature whose page hashes don't match the
post-postject layout for Node.js 25's chained fixups, leaving fixups
unapplied at load and crashing the binary in __cxx_global_var_init
(EXC_BAD_ACCESS at 0x3 — the unprocessed chain-entry tag).

Running the release on macos-latest lets pack-app's adHocSignMacBinary
use native codesign, which understands chained fixups. Drops the entire
ldid build step.

* ci(release): document why release runs on macos-latest
zkochan added a commit that referenced this pull request May 4, 2026
## Summary

Closes #11423.

`pnpm-darwin-x64.tar.gz` and `@pnpm/macos-x64` are removed because the binaries they contain segfault at startup on Intel Macs and the underlying bug is upstream and unfixed.

## Why this isn't a fix in code

The crash happens in `__cxx_global_var_init` with `EXC_BAD_ACCESS (code=1, address=0x3)` — the unprocessed-chain-entry tag — in dyld's chained-fixup processing. PR #11415's hypothesis was that `ldid`'s page hashes were the cause, but switching to native `codesign` in #11415 didn't fix it: the upstream minimal repro in [nodejs/node#62893](nodejs/node#62893) is `node --build-sea` + `codesign --sign -` + run, with no pnpm and no `ldid`, and it still crashes. The corruption is in LIEF's Mach-O surgery during `--build-sea` for x64 — chained-fixup chain entries get rewritten incorrectly when the SEA segment is inserted, and re-signing produces a valid signature over the broken bytes.

The Node.js team is not going to fix this:

- [nodejs/node#60250](nodejs/node#60250) (merged) — *"It's unlikely that anyone would invest in fixing them on x64 macOS in the near future, now that x64 macOS is being phased out."* They skipped the SEA tests on x64 macOS rather than chase the bug.
- [nodejs/node#59553](nodejs/node#59553) (open) — long-running test failures on macOS x64 with the same root cause (sometimes surfacing as `unsupported thread-local, larger than 4GB`).

`@yao-pkg/pkg` works around it by appending the JS payload to the file tail and using a custom-patched Node binary that reads from the tail at startup; this avoids Mach-O surgery entirely. We can't reuse pack-app for that because vanilla Node from nodejs.org doesn't read tail-appended payloads — only pkg-fetch's patched binaries do — so adopting that path would mean re-implementing pkg-fetch for one target. For now we're dropping the broken artifact rather than introducing a second build mechanism.

## Changes

- **`pnpm/artifacts/exe/package.json`** — remove `@pnpm/macos-x64` from `optionalDependencies`; remove `darwin-x64` from `pnpm.app.targets`.
- **`.meta-updater/src/index.ts`** — remove `@pnpm/macos-x64` from the enforced `optionalDependencies` list (otherwise `meta-updater` would put it back).
- **`pnpm/artifacts/exe/scripts/build-artifacts.ts`** — drop `darwin-x64` from `narrowTargets` so dev-local builds match the published matrix; comment explains why.
- **`__utils__/scripts/src/copy-artifacts.ts`** — stop creating `pnpm-darwin-x64.tar.gz` so the GitHub release page no longer ships it.
- **`pnpm/artifacts/darwin-x64/`** — deleted (was the workspace source for `@pnpm/macos-x64`).
- **`pnpm/artifacts/exe/setup.js`** — wraps the `import.meta.resolve('${pkgName}/package.json')` lookup in `try`/`catch`. On Intel Mac specifically, prints a clear message pointing at this issue, the upstream Node.js issue, and the two workarounds (`npm install -g pnpm` to use the system Node.js, or stay on pnpm 10.x). Other unsupported hosts get a generic message in the same shape. Exits non-zero so the install fails loudly instead of silently leaving a broken `pnpm`.
- **`pnpm-lock.yaml`** — regenerated.
- **`.changeset/drop-darwin-x64-broken-sea.md`** — patch bumps for `@pnpm/exe` and `pnpm` with user-facing explanation and pointers.

Docs side already lists this limitation under `pack-app` Known limitations: pnpm/pnpm.io@36d962f6 / pnpm/pnpm.io@91f45632.

## Compat

- Intel Mac users on existing `@pnpm/exe` (≤ 11.0.4) keep working with the (broken) old binary they already have.
- `pnpm self-update` from an Intel Mac on an older `@pnpm/exe` will hit the new `setup.js` error path with a clear pointer to the workarounds.
- New Intel Mac installs via `npm install -g @pnpm/exe` will fail loudly with the same pointer.
- Install via `npm install -g pnpm` (the JS-only package, uses system Node) is unaffected and remains the recommended path.
- The `install.sh` from `get.pnpm.io` will fail with a 404 on the missing `pnpm-darwin-x64.tar.gz`. That's a separate repo and a follow-up — happy to do that as a second PR.
zkochan added a commit that referenced this pull request May 4, 2026
## Summary

Closes #11423.

`pnpm-darwin-x64.tar.gz` and `@pnpm/macos-x64` are removed because the binaries they contain segfault at startup on Intel Macs and the underlying bug is upstream and unfixed.

## Why this isn't a fix in code

The crash happens in `__cxx_global_var_init` with `EXC_BAD_ACCESS (code=1, address=0x3)` — the unprocessed-chain-entry tag — in dyld's chained-fixup processing. PR #11415's hypothesis was that `ldid`'s page hashes were the cause, but switching to native `codesign` in #11415 didn't fix it: the upstream minimal repro in [nodejs/node#62893](nodejs/node#62893) is `node --build-sea` + `codesign --sign -` + run, with no pnpm and no `ldid`, and it still crashes. The corruption is in LIEF's Mach-O surgery during `--build-sea` for x64 — chained-fixup chain entries get rewritten incorrectly when the SEA segment is inserted, and re-signing produces a valid signature over the broken bytes.

The Node.js team is not going to fix this:

- [nodejs/node#60250](nodejs/node#60250) (merged) — *"It's unlikely that anyone would invest in fixing them on x64 macOS in the near future, now that x64 macOS is being phased out."* They skipped the SEA tests on x64 macOS rather than chase the bug.
- [nodejs/node#59553](nodejs/node#59553) (open) — long-running test failures on macOS x64 with the same root cause (sometimes surfacing as `unsupported thread-local, larger than 4GB`).

`@yao-pkg/pkg` works around it by appending the JS payload to the file tail and using a custom-patched Node binary that reads from the tail at startup; this avoids Mach-O surgery entirely. We can't reuse pack-app for that because vanilla Node from nodejs.org doesn't read tail-appended payloads — only pkg-fetch's patched binaries do — so adopting that path would mean re-implementing pkg-fetch for one target. For now we're dropping the broken artifact rather than introducing a second build mechanism.

## Changes

- **`pnpm/artifacts/exe/package.json`** — remove `@pnpm/macos-x64` from `optionalDependencies`; remove `darwin-x64` from `pnpm.app.targets`.
- **`.meta-updater/src/index.ts`** — remove `@pnpm/macos-x64` from the enforced `optionalDependencies` list (otherwise `meta-updater` would put it back).
- **`pnpm/artifacts/exe/scripts/build-artifacts.ts`** — drop `darwin-x64` from `narrowTargets` so dev-local builds match the published matrix; comment explains why.
- **`__utils__/scripts/src/copy-artifacts.ts`** — stop creating `pnpm-darwin-x64.tar.gz` so the GitHub release page no longer ships it.
- **`pnpm/artifacts/darwin-x64/`** — deleted (was the workspace source for `@pnpm/macos-x64`).
- **`pnpm/artifacts/exe/setup.js`** — wraps the `import.meta.resolve('${pkgName}/package.json')` lookup in `try`/`catch`. On Intel Mac specifically, prints a clear message pointing at this issue, the upstream Node.js issue, and the two workarounds (`npm install -g pnpm` to use the system Node.js, or stay on pnpm 10.x). Other unsupported hosts get a generic message in the same shape. Exits non-zero so the install fails loudly instead of silently leaving a broken `pnpm`.
- **`pnpm-lock.yaml`** — regenerated.
- **`.changeset/drop-darwin-x64-broken-sea.md`** — patch bumps for `@pnpm/exe` and `pnpm` with user-facing explanation and pointers.

Docs side already lists this limitation under `pack-app` Known limitations: pnpm/pnpm.io@36d962f6 / pnpm/pnpm.io@91f45632.

## Compat

- Intel Mac users on existing `@pnpm/exe` (≤ 11.0.4) keep working with the (broken) old binary they already have.
- `pnpm self-update` from an Intel Mac on an older `@pnpm/exe` will hit the new `setup.js` error path with a clear pointer to the workarounds.
- New Intel Mac installs via `npm install -g @pnpm/exe` will fail loudly with the same pointer.
- Install via `npm install -g pnpm` (the JS-only package, uses system Node) is unaffected and remains the recommended path.
- The `install.sh` from `get.pnpm.io` will fail with a 404 on the missing `pnpm-darwin-x64.tar.gz`. That's a separate repo and a follow-up — happy to do that as a second PR.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants