Skip to content

semrel-extra/zx-bulk-release

Repository files navigation

zx-bulk-release

A zx-based tool for conventional-commits-driven secure release workflow for monorepos. Inspired by multi-semantic-release.

CI Maintainability Code Coverage npm (tag)

Key features

  • Conventional commits trigger semantic releases.
  • Automated cross-pkg version bumping.
  • Predictable toposort-driven flow.
  • No default branch blocking (no release commits).
  • Pkg changelogs go to changelog branch (configurable).
  • Docs are published to gh-pages branch (configurable).
  • No extra builds. The required deps are fetched from the pkg registry (npmFetch config opt).
  • Flexible pipeline: run all phases in one command, split into two (pack/deliver), or use all four (receive/pack/verify/deliver) for maximum supply chain security.
  • Coordinated delivery: multiple release agents can safely serve the same monorepo concurrently via git-tag-based semaphores.

Roadmap

  • Store release metrics to meta.
  • Two-phase pipeline (pack / deliver).
  • Semaphore. Let several release agents serve the monorepo at the same time.
  • Multistack. Add support for java/kt/py.

Requirements

  • macOS / linux
  • Node.js >= 16.0.0
  • npm >=7 / yarn >= 3
  • tar
  • git

Usage

Install

yarn add zx-bulk-release

CLI

GH_TOKEN=ghtoken GH_USER=username NPM_TOKEN=npmtoken npx zx-bulk-release [opts]
Flag Description Default
--receive Consume rebuild signal, analyze, preflight. Writes zbr-context.json. Run before deps install.
--pack [dir] Pack only: build, test, and write delivery tars to dir. No credentials needed. parcels
--verify [in:out] Verify untrusted parcels against context, copy valid ones to output dir. Use --context for path. parcels:parcels
--context <path> Path to trusted zbr-context.json (used with --verify). zbr-context.json
--deliver [dir] Deliver only: read tars from dir and run delivery channels. No source code needed. parcels
--ignore Packages to ignore: a, b
--include-private Include private packages false
--concurrency build/publish threads limit os.cpus.length
--no-build Skip buildCmd invoke
--no-test Disable testCmd run
--no-npm-fetch Disable npm artifacts fetching
--only-workspace-deps Recognize only workspace: deps as graph edges
--dry-run / --no-publish Disable any publish logic
--report Persist release state to file
--snapshot Publish only to npm snapshot channel and run publishCmd (if defined), skip everything else
--debug Enable zx verbose mode
--version / -v Print own version

Pipeline modes

zbr supports three deployment schemes — pick the one that matches your security requirements.

All-in-one

A single command runs all phases in one process. Simple but requires all credentials at build time.

jobs:
  release:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with: { fetch-depth: 0 }
      - run: yarn install
      - run: npx zx-bulk-release
        env:
          GH_TOKEN: ${{ secrets.GH_TOKEN }}
          NPM_TOKEN: ${{ secrets.NPM_TOKEN }}

Two phases: pack + deliver

Build and delivery run as separate jobs. The pack phase needs no credentials — tars contain ${{ENV_VAR}} templates resolved at delivery time.

jobs:
  pack:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with: { fetch-depth: 0 }
      - run: yarn install
      - run: npx zx-bulk-release --pack
      - uses: actions/upload-artifact@v4
        with:
          name: parcels-${{ github.run_id }}
          path: parcels/
          retention-days: 1
          if-no-files-found: ignore

  deliver:
    needs: pack
    runs-on: ubuntu-latest
    steps:
      - uses: actions/download-artifact@v4
        id: download
        with:
          name: parcels-${{ github.run_id }}
          path: parcels/
        continue-on-error: true
      - if: steps.download.outcome == 'success'
        run: npx zx-bulk-release --deliver
        env:
          GH_TOKEN: ${{ secrets.GH_TOKEN }}
          NPM_TOKEN: ${{ secrets.NPM_TOKEN }}
          GIT_COMMITTER_NAME: Semrel Extra Bot
          GIT_COMMITTER_EMAIL: semrel-extra-bot@hotmail.com

Four phases: receive + pack + verify + deliver

Maximum supply chain security. Each phase has strict trust boundaries. See SECURITY.md for the threat model.

  • receive — runs before yarn install, consumes rebuild signals, analyzes, writes trusted context
  • pack — runs after deps install with zero credentials, builds and packs tars
  • verify — validates untrusted parcels against the trusted context
  • deliver — delivers only verified parcels
on:
  push:
    branches: [master]
    tags: ['zbr-rebuild.*']

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with: { fetch-depth: 0 }

      # receive — runs BEFORE deps install, safe to use GH_TOKEN
      - run: npx zx-bulk-release --receive
        id: receive
        env:
          GH_TOKEN: ${{ secrets.GH_TOKEN }}

      # Trust anchor: context uploaded before any third-party code
      - uses: actions/upload-artifact@v4
        with:
          name: context-${{ github.run_id }}
          path: zbr-context.json

      # pack — deps installed, hostile code may run, zero credentials
      - if: steps.receive.outputs.status == 'proceed'
        run: |
          yarn install
          npx zx-bulk-release --pack

      - if: steps.receive.outputs.status == 'proceed'
        uses: actions/upload-artifact@v4
        with:
          name: parcels-${{ github.run_id }}
          path: parcels/

  deliver:
    needs: build
    runs-on: ubuntu-latest
    steps:
      # Download trusted context and untrusted parcels separately
      - uses: actions/download-artifact@v4
        with:
          name: context-${{ github.run_id }}
          path: .
      - uses: actions/download-artifact@v4
        with:
          name: parcels-${{ github.run_id }}
          path: parcels-unverified/

      # verify — validate untrusted parcels against trusted context
      - run: npx zx-bulk-release --verify parcels-unverified/:parcels/

      # deliver — only verified parcels
      - run: npx zx-bulk-release --deliver
        env:
          GH_TOKEN: ${{ secrets.GH_TOKEN }}
          NPM_TOKEN: ${{ secrets.NPM_TOKEN }}
          GIT_COMMITTER_NAME: Semrel Extra Bot
          GIT_COMMITTER_EMAIL: semrel-extra-bot@hotmail.com

After delivery, each tar is replaced with a marker (released, skip, conflict, or orphan). In the four-phase mode, parcels are verified against the trusted context (uploaded before deps install) before any delivery begins.

Recovery is simply re-running the deliver job. Only undelivered tars (not yet replaced with markers) will be processed. No rebuild required.

JS API

import { run } from 'zx-bulk-release'

const cwd = '/foo/bar'
const env = {GH_TOKEN: 'foo', NPM_TOKEN: 'bar'}
const flags = {dryRun: true}

await run({
  cwd,    // Defaults to process.cwd()
  flags,  // Defaults to process.env
  env     // Defaults to minimist-parsed `process.argv.slice(2)`
})

Config

Config files

cosmiconfig-compatible lookup: .releaserc, .release.json, .release.yaml, .releaserc.js, release.config.js, or release key in package.json. Searched from the package root up to the repo root.

{
  "buildCmd": "yarn && yarn build",
  "testCmd": "yarn test",
  "npmFetch": true,
  "changelog": "changelog",
  "ghPages": "gh-pages",
  "diffTagUrl": "${repoPublicUrl}/compare/${prevTag}...${newTag}",
  "diffCommitUrl": "${repoPublicUrl}/commit/${hash}"
}

Command templating

buildCmd, testCmd and publishCmd support ${{ variable }} interpolation. The template context includes all pkg fields and the release context (flags, git, env, etc):

{
  "buildCmd": "yarn build --pkg=${{name}} --ver=${{version}}",
  "testCmd": "yarn test --scope=${{name}}",
  "publishCmd": "echo releasing ${{name}}@${{version}}"
}

Available variables include: name, version, absPath, relPath, and anything from pkg.ctx (e.g. git.sha, git.root, flags.*).

Changelog diff URLs

By default, changelog entries link to GitHub compare/commit pages. Override diffTagUrl and diffCommitUrl to customize for other platforms (e.g. Gerrit):

{
  "diffTagUrl": "https://gerrit.foo.com/plugins/gitiles/${repoName}/+/refs/tags/${newTag}",
  "diffCommitUrl": "https://gerrit.foo.com/plugins/gitiles/${repoName}/+/${hash}%5E%21"
}

Available variables: repoName, repoPublicUrl, prevTag, newTag, name, version, hash, short.

GitHub Enterprise

Set ghUrl to point to your GHE instance. API URL (ghApiUrl) is derived automatically.

{
  "ghUrl": "https://ghe.corp.com"
}

Or via env: GH_URL=https://ghe.corp.com / GITHUB_URL=https://ghe.corp.com.

env vars

export const parseEnv = (env = process.env) => {
  const {GH_USER, GH_USERNAME, GITHUB_USER, GITHUB_USERNAME, GH_TOKEN, GITHUB_TOKEN, GH_URL, GITHUB_URL, NPM_TOKEN, NPM_REGISTRY, NPMRC, NPM_USERCONFIG, NPM_CONFIG_USERCONFIG, NPM_PROVENANCE, NPM_OIDC, ACTIONS_ID_TOKEN_REQUEST_URL, GIT_COMMITTER_NAME, GIT_COMMITTER_EMAIL} = env

  return {
    ghUser:             GH_USER || GH_USERNAME || GITHUB_USER || GITHUB_USERNAME || 'x-access-token',
    ghToken:            GH_TOKEN || GITHUB_TOKEN,
    ghUrl:              GH_URL || GITHUB_URL || 'https://github.com',
    npmToken:           NPM_TOKEN,
    // npmConfig suppresses npmToken
    npmConfig:          NPMRC || NPM_USERCONFIG || NPM_CONFIG_USERCONFIG,
    npmRegistry:        NPM_REGISTRY || 'https://registry.npmjs.org',
    npmProvenance:      NPM_PROVENANCE,
    // OIDC trusted publishing: https://docs.npmjs.com/trusted-publishers/
    npmOidc:            NPM_OIDC || (!NPM_TOKEN && ACTIONS_ID_TOKEN_REQUEST_URL),
    gitCommitterName:   GIT_COMMITTER_NAME || 'Semrel Extra Bot',
    gitCommitterEmail:  GIT_COMMITTER_EMAIL || 'semrel-extra-bot@hotmail.com',
  }
}

OIDC Trusted Publishing

npm now supports OIDC trusted publishing as a replacement for long-lived access tokens. To enable:

  1. Configure a trusted publisher for each package on npmjs.com (link your GitHub repo and workflow)
  2. Set NPM_OIDC=true in your workflow environment
  3. Ensure your GitHub Actions workflow has permissions: { id-token: write }
  4. Make sure each package.json includes the repository field matching your GitHub repo URL

OIDC mode is also auto-detected when NPM_TOKEN is not set and ACTIONS_ID_TOKEN_REQUEST_URL is present (GitHub Actions with id-token: write permission).

When OIDC is active, NPM_TOKEN and NPMRC are ignored for publishing and --provenance is enabled automatically.

Snapshot releases from PRs

The --snapshot flag publishes packages to the snapshot npm dist-tag with a pre-release version like 1.2.1-snap.a3f0c12. This is useful for testing changes from a feature branch before merging.

What snapshot does differently:

  • Version gets a -snap.<short-sha> suffix instead of a clean bump
  • Git release tags are not pushed
  • Only npm and publishCmd channels run (no gh-release, no changelog, no gh-pages, no meta)
  • npm tag is snapshot instead of latest

Workflow example (.github/workflows/snapshot.yml):

name: Snapshot
on:
  pull_request:
    types: [labeled]

jobs:
  snapshot:
    if: github.event.label.name == 'snapshot'
    runs-on: ubuntu-latest
    permissions:
      contents: read
      id-token: write    # for OIDC trusted publishing
    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 0
      - uses: actions/setup-node@v4
        with:
          node-version: 20
      - run: yarn install
      - run: npx zx-bulk-release --snapshot
        env:
          GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
          NPM_TOKEN: ${{ secrets.NPM_TOKEN }}

How to use:

  1. Push a feature branch and open a PR.
  2. Add the snapshot label to the PR.
  3. The workflow publishes snapshot versions to npm.
  4. Consumers install snapshots via npm install yourpkg@snapshot.
  5. After merge, the regular release flow on master publishes clean versions to latest.

Selective testing along the change graph

In a monorepo, --dry-run combined with --no-build lets you run tests only for packages affected by the current changes — following the dependency graph, without publishing anything. This gives you a precise CI check scoped to what actually changed:

npx zx-bulk-release --dry-run --no-build

See antongolub/misc for a real-world example of this pattern.

Demo

Implementation notes

Architecture

The release pipeline is split into three subsystems under post/:

post/
  modes/     — pipeline entry points: receive, pack, verify, deliver
  depot/     — preparation: analysis, versioning, building, testing, tar packing
  courier/   — sealed delivery: receives self-describing tars and delivers through channels
  api/       — shared infrastructure wrappers (git, npm, gh)

This separation ensures that courier never touches the project directory — it works only with pre-packed tars and credentials resolved at delivery time. The two subsystems can run in separate CI jobs with different privilege levels.

Flow

receive:  topo -> contextify -> analyze -> preflight -> context.json
pack:     build -> test -> pack -> directive
verify:   validate parcels against context -> copy to output
deliver:  deliver parcels

all-in-one: topo -> contextify -> analyze -> preflight -> build -> test -> pack -> publish -> clean

@semrel-extra/topo resolves the release queue respecting dependency graphs. The graph allows parallel execution where the dependency tree permits; memoizeBy prevents duplicate work when a package is reached by multiple paths.

By default, packages marked as private are omitted. Override with --include-private.

Preflight runs between analyze and build. It checks tag availability on git remote and eliminates version conflicts before wasting build/test time. If a tag is already taken by a newer commit, preflight re-resolves the version. If taken by the same or an older commit, the package is skipped.

Steps

Each step has a uniform signature (pkg, ctx):

  • contextify — resolves per-package config, latest release metadata, and git context.
  • analyze — determines semantic changes, release type, and next version.
  • preflight — checks tag availability on remote, re-resolves version on conflict, skips duplicates.
  • build — runs buildCmd (with dep traversal and optional npm artifact fetch).
  • test — runs testCmd.
  • pack — stages delivery artifacts into self-describing tar containers (npm pack, docs copy, assets, release notes). Each tar is named parcel.{sha7}.{channel}.{name}.{version}.{hash6}.tar and contains a manifest.json with channel name, delivery instructions, and template credentials (${{ENV_VAR}}). A directive meta-parcel is also generated, listing all parcels and their delivery steps. After this step, everything the courier needs is outside the project dir.
  • publish — hands tars to courier's deliver(), runs cmd channel separately. Tag push is handled by the git-tag channel.
  • clean — restores package.json files and unsets git user config.

Set config.releaseRules to override the default rules preset:

[
  {group: 'Features', releaseType: 'minor', prefixes: ['feat']},
  {group: 'Fixes & improvements', releaseType: 'patch', prefixes: ['fix', 'perf', 'refactor', 'docs', 'patch']},
  {group: 'BREAKING CHANGES', releaseType: 'major', keywords: ['BREAKING CHANGE', 'BREAKING CHANGES']},
]

Tar containers

Each delivery artifact is a self-describing tar archive:

parcel.{sha7}.{channel}.{name}.{version}.{hash6}.tar
  manifest.json    — channel name, delivery params, template credentials
  package.tgz     — (npm channel) npm tarball
  assets/          — (gh-release channel) release assets
  docs/            — (gh-pages channel) documentation files

The sha7 prefix groups all parcels of one commit. name is the sanitized package name (@scope/pkgscope-pkg). The hash6 suffix is a content hash for deduplication — two builds of the same commit producing identical content yield the same filename (last-writer-wins), while different content gets a different hash.

A directive meta-parcel (parcel.{sha7}.directive.{ts}.tar) is generated alongside regular parcels. It contains the complete delivery map: package queue, per-package channel steps, and an authoritative list of parcel filenames. The directive enables coordinated delivery — see DELIVER_SPEC.md.

The manifest contains ${{ENV_VAR}} placeholders that are resolved by the courier at delivery time via resolveManifest(). This ensures credentials never touch the build phase.

Channels

Delivery channels are a registry of {name, when, prepare?, run, requires?, snapshot?, transport?} objects:

  • git-tag — pushes the release tag to git remote. Runs before other channels. Returns conflict if the tag already exists. Skipped in snapshot mode.
  • meta — pushes release metadata to the meta branch (or as a GH release asset). Checks docs seniority before committing.
  • npm — publishes to the npm registry. Returns duplicate on EPUBLISHCONFLICT / 403.
  • gh-release — creates a GitHub release with optional file assets (requires tag to exist). Returns duplicate on 422.
  • gh-pages — pushes docs to a gh-pages branch. Checks docs seniority before committing.
  • changelog — pushes a changelog entry to a changelog branch. Checks docs seniority before committing.
  • cmd — runs a custom publishCmd (depot-side, not through courier; transport: false).

Each channel declares requires — a list of manifest fields that must be present after credential resolution. Courier validates before delivery; missing credentials produce a warning and skip marker.

Channels return one of: ok (success), duplicate (already published, goal achieved), or conflict (version collision, needs rebuild). Unrecoverable errors are thrown.

Coordinated delivery

When multiple zbr processes target the same monorepo concurrently, delivery is coordinated via git-tag-based semaphores:

  1. Directive — each build produces a meta-parcel listing all packages and their delivery steps.
  2. Semaphore lock — before delivering, a process pushes zbr-deliver.{sha7} as an annotated tag. Atomic push = ownership. Failure = another process is working.
  3. Orphan cleanup — the directive invalidates parcels not in its authoritative list (stale artifacts from previous builds).
  4. Conflict resolution — if git-tag returns conflict, all parcels of that package are marked, and a zbr-rebuild.{sha7} tag signals CI to rebuild with fresh versions.
  5. Partial delivery — skipped parcels (missing credentials) remain valid tarballs. Another process with the right credentials can pick them up later.

See DELIVER_SPEC.md for the full protocol specification.

Credential flow

depot (build phase)                          courier (deliver phase)
  manifest: { token: '${{NPM_TOKEN}}' }  ->   resolveManifest(manifest, env)
                                                 -> { token: 'actual-secret' }

Template credentials (${{ENV_VAR}}) are written into manifests at pack time. Courier resolves them from process.env at delivery time. This means the build phase never sees real secrets — including git push credentials (GH_TOKEN, GIT_COMMITTER_NAME, GIT_COMMITTER_EMAIL) which are now resolved by the git-tag channel at delivery time.

Tags

Lerna tags (like @pkg/name@v1.0.0-beta.0) are suitable for monorepos, but they don't follow semver spec. Therefore, we propose another contract:

'2022.6.13-optional-org.pkg-name.v1.0.0-beta.1+sha.1-f0'
// date    name                  version             format

Note, npm-package-name charset is wider than semver, so we need a pinch of base64url magic for some cases.

'2022.6.13-examplecom.v1.0.0.ZXhhbXBsZS5jb20-f1'
// date    name       ver    b64             format

Anyway, it's still possible to override the default config by tagFormat option:

tagFormat Example
f0 2022.6.22-qiwi.pijma-native.v1.0.0-beta.0+foo.bar-f0
f1 2022.6.13-examplecom.v1.0.0.ZXhhbXBsZS5jb20-f1
lerna @qiwi/pijma-ssr@1.1.12
pure 1.2.3-my.package

Meta

Each release gathers its own meta. It is recommended to store the data somehow to ensure flow reliability.:

  • Set meta: {type: 'asset'} to persist as gh asset.
  • If set meta: {type: null} the required data will be fetched from the npm artifact.
  • Otherwise, it will be pushed as a regular git commit to the meta branch (default behaviour).

2022-6-26-semrel-extra-zxbr-test-c-1-3-1-f0.json

{
  "META_VERSION": "1",
  "name": "@semrel-extra/zxbr-test-c",
  "hash": "07b7df33f0159f674c940bd7bbb2652cdaef5207",
  "version": "1.3.1",
  "dependencies": {
    "@semrel-extra/zxbr-test-a": "^1.4.0",
    "@semrel-extra/zxbr-test-d": "~1.2.0"
  }
}

Report

Release process state is reported to the console and to a file if --report flag is set to /some/path/release-report.json, for example.

{
  status: 'success',            // 'sucess' | 'failure' | 'pending'
  error: null,                  // null or Error
  queue: ['a', 'b', 'c', 'd']   // release queue
  packages: [{
    name: 'a',
    version: '1.1.0',
    path: '/pkg/abs/path',
    relPath: 'pkg/rel/path',
    config: {                   // pkg config
      changelog: 'changelog',
      npmFetch: true
    },                 
    changes: [{                 // semantic changes
      group: 'Features',
      releaseType: 'minor',
      change: 'feat: add feat',
      subj: 'feat: add feat',
      body: '',
      short: '792512c',
      hash: '792512cccd69c6345d9d32d3d73e2591ea1776b5'
    }],                  
    tag: {
      version: 'v1.1.0',
      name: 'a',
      ref: '2022.6.22-a.v1.1.0-f0'
    },
    releaseType: 'minor',       // 'major' | 'minor' | 'patch'
    prevVersion: '1.0.0'        // previous version or null
  }, {
    name: 'b',
    // ...
  }],
  events: [
    {msg: ['zx-bulk-release'], scope:'~', date: 1665839585488, level: 'info'},
    {msg: ['queue:',['a','b']], scope:'~', date: 1665839585493, level: 'info'},
    {msg: ["run buildCmd 'yarn && yarn build && yarn test'"], scope: 'a', date: 1665839585719, level:'info'},
    // ...
  ]
}

Output

Compact and clear logs

Run npm_config_yes=true npx zx-bulk-release
zx-bulk-release
[@semrel-extra/zxbr-test-a] semantic changes [
  {
    group: 'Fixes & improvements',
    releaseType: 'patch',
    change: 'fix(a): random',
    subj: 'fix(a): random',
    body: '',
    short: '6ff25bd',
    hash: '6ff25bd421755b929ef2b58f35c727670fd93849'
  }
]
[@semrel-extra/zxbr-test-a] run cmd 'yarn && yarn build && yarn test'
[@semrel-extra/zxbr-test-a] push release tag 2022.6.27-semrel-extra.zxbr-test-a.1.8.1-f0
[@semrel-extra/zxbr-test-a] push artifact to branch 'meta'
[@semrel-extra/zxbr-test-a] push changelog
[@semrel-extra/zxbr-test-a] publish npm package @semrel-extra/zxbr-test-a 1.8.1 to https://registry.npmjs.org
[@semrel-extra/zxbr-test-a] create gh release
[@semrel-extra/zxbr-test-b] semantic changes [
  {
    group: 'Dependencies',
    releaseType: 'patch',
    change: 'perf',
    subj: 'perf: @semrel-extra/zxbr-test-a updated to 1.8.1'
  }
]
[@semrel-extra/zxbr-test-b] run cmd 'yarn && yarn build && yarn test'
[@semrel-extra/zxbr-test-b] push release tag 2022.6.27-semrel-extra.zxbr-test-b.1.3.5-f0
[@semrel-extra/zxbr-test-b] push artifact to branch 'meta'
[@semrel-extra/zxbr-test-b] push changelog
[@semrel-extra/zxbr-test-b] publish npm package @semrel-extra/zxbr-test-b 1.3.5 to https://registry.npmjs.org
[@semrel-extra/zxbr-test-b] create gh release
[@semrel-extra/zxbr-test-d] semantic changes [

References

License

MIT

About

zx-based alternative for multi-semantic-release

Topics

Resources

License

Security policy

Stars

Watchers

Forks

Packages

 
 
 

Contributors