Skip to content

Commit 7d3bc4d

Browse files
vincentkocvincentkoc
authored
fix(ci): bundle test shards and right-size runners
Bundles only isolated low-risk Node shards, keeps stateful suites isolated, and right-sizes Blacksmith runners to reduce runner-registration bursts. Exact-head CI run 27932220655 passed the changed CI planning lanes; its two remaining agent test reds match latest main baseline run 27929063460 and are unrelated to this CI-only diff.
1 parent 5fbb5f7 commit 7d3bc4d

6 files changed

Lines changed: 220 additions & 20 deletions

File tree

‎.github/workflows/ci.yml‎

Lines changed: 23 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -197,7 +197,7 @@ jobs:
197197
node --input-type=module <<'EOF'
198198
import { appendFileSync } from "node:fs";
199199
import {
200-
createNodeTestShards,
200+
createNodeTestShardBundles,
201201
} from "./scripts/lib/ci-node-test-plan.mjs";
202202
import {
203203
createChannelContractTestShards,
@@ -273,7 +273,7 @@ jobs:
273273
}
274274
275275
const nodeTestShards = runNodeFull
276-
? createNodeTestShards({
276+
? createNodeTestShardBundles({
277277
includeReleaseOnlyPluginShards: false,
278278
}).map((shard) => ({
279279
check_name: shard.checkName,
@@ -320,7 +320,14 @@ jobs:
320320
run_checks_windows: runWindows,
321321
checks_windows_matrix: createMatrix(
322322
runWindows
323-
? [{ check_name: "checks-windows-node-test", runtime: "node", task: "test" }]
323+
? [
324+
{
325+
check_name: "checks-windows-node-test",
326+
runtime: "node",
327+
task: "test",
328+
runner: "blacksmith-8vcpu-windows-2025",
329+
},
330+
]
324331
: [],
325332
),
326333
run_macos_node: runMacos,
@@ -558,7 +565,7 @@ jobs:
558565
contents: read
559566
needs: [preflight]
560567
if: needs.preflight.outputs.run_build_artifacts == 'true'
561-
runs-on: ${{ github.event_name == 'workflow_dispatch' && 'ubuntu-24.04' || (github.repository == 'openclaw/openclaw' && 'blacksmith-32vcpu-ubuntu-2404' || 'ubuntu-24.04') }}
568+
runs-on: ${{ github.event_name == 'workflow_dispatch' && 'ubuntu-24.04' || (github.repository == 'openclaw/openclaw' && 'blacksmith-16vcpu-ubuntu-2404' || 'ubuntu-24.04') }}
562569
timeout-minutes: 20
563570
outputs:
564571
channels-result: ${{ steps.built_artifact_checks.outputs['channels-result'] }}
@@ -1139,7 +1146,7 @@ jobs:
11391146
name: ${{ matrix.check_name }}
11401147
needs: [preflight]
11411148
if: needs.preflight.outputs.run_checks_node_core_nondist == 'true'
1142-
runs-on: ${{ github.event_name == 'workflow_dispatch' && 'ubuntu-24.04' || (github.repository == 'openclaw/openclaw' && (matrix.runner || 'blacksmith-8vcpu-ubuntu-2404') || 'ubuntu-24.04') }}
1149+
runs-on: ${{ github.event_name == 'workflow_dispatch' && 'ubuntu-24.04' || (github.repository == 'openclaw/openclaw' && (matrix.runner || 'blacksmith-4vcpu-ubuntu-2404') || 'ubuntu-24.04') }}
11431150
timeout-minutes: 60
11441151
strategy:
11451152
fail-fast: false
@@ -1269,7 +1276,7 @@ jobs:
12691276
runner: blacksmith-16vcpu-ubuntu-2404
12701277
- check_name: check-dependencies
12711278
task: dependencies
1272-
runner: blacksmith-8vcpu-ubuntu-2404
1279+
runner: blacksmith-4vcpu-ubuntu-2404
12731280
- check_name: check-test-types
12741281
task: test-types
12751282
runner: blacksmith-4vcpu-ubuntu-2404
@@ -1390,7 +1397,7 @@ jobs:
13901397
name: ${{ matrix.check_name }}
13911398
needs: [preflight]
13921399
if: ${{ !cancelled() && always() && needs.preflight.outputs.run_check_additional == 'true' }}
1393-
runs-on: ${{ github.event_name == 'workflow_dispatch' && 'ubuntu-24.04' || (github.repository == 'openclaw/openclaw' && 'blacksmith-8vcpu-ubuntu-2404' || 'ubuntu-24.04') }}
1400+
runs-on: ${{ github.event_name == 'workflow_dispatch' && 'ubuntu-24.04' || (github.repository == 'openclaw/openclaw' && (matrix.runner || 'blacksmith-4vcpu-ubuntu-2404') || 'ubuntu-24.04') }}
13941401
timeout-minutes: 20
13951402
strategy:
13961403
fail-fast: false
@@ -1400,21 +1407,29 @@ jobs:
14001407
- check_name: check-additional-boundaries-a
14011408
group: boundaries
14021409
boundary_shard: 1/4
1410+
runner: blacksmith-8vcpu-ubuntu-2404
14031411
- check_name: check-additional-boundaries-bcd
14041412
group: boundaries
14051413
boundary_shard: 2/4,3/4,4/4
1414+
runner: blacksmith-8vcpu-ubuntu-2404
14061415
- check_name: check-session-accessor-boundary
14071416
group: session-accessor-boundary
1417+
runner: blacksmith-4vcpu-ubuntu-2404
14081418
- check_name: check-session-transcript-reader-boundary
14091419
group: session-transcript-reader-boundary
1420+
runner: blacksmith-4vcpu-ubuntu-2404
14101421
- check_name: check-additional-extension-channels
14111422
group: extension-channels
1423+
runner: blacksmith-8vcpu-ubuntu-2404
14121424
- check_name: check-additional-extension-bundled
14131425
group: extension-bundled
1426+
runner: blacksmith-8vcpu-ubuntu-2404
14141427
- check_name: check-additional-extension-package-boundary
14151428
group: extension-package-boundary
1429+
runner: blacksmith-8vcpu-ubuntu-2404
14161430
- check_name: check-additional-runtime-topology-architecture
14171431
group: runtime-topology-architecture
1432+
runner: blacksmith-4vcpu-ubuntu-2404
14181433
steps:
14191434
- name: Checkout
14201435
shell: bash
@@ -1757,7 +1772,7 @@ jobs:
17571772
name: ${{ matrix.check_name }}
17581773
needs: [preflight]
17591774
if: needs.preflight.outputs.run_checks_windows == 'true'
1760-
runs-on: ${{ github.event_name == 'workflow_dispatch' && 'windows-2025' || (github.repository == 'openclaw/openclaw' && 'blacksmith-16vcpu-windows-2025' || 'windows-2025') }}
1775+
runs-on: ${{ github.event_name == 'workflow_dispatch' && 'windows-2025' || (github.repository == 'openclaw/openclaw' && (matrix.runner || 'blacksmith-8vcpu-windows-2025') || 'windows-2025') }}
17611776
timeout-minutes: 60
17621777
env:
17631778
NODE_OPTIONS: --max-old-space-size=8192

‎docs/ci.md‎

Lines changed: 10 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -74,7 +74,7 @@ Scope logic lives in `scripts/ci-changed-scope.mjs` and is covered by unit tests
7474
- **CI routing-only edits, selected cheap core-test fixture edits, and narrow plugin contract helper/test-routing edits** use a fast Node-only manifest path: `preflight`, security, and a single `checks-fast-core` task. That path skips build artifacts, Node 22 compatibility, channel contracts, full core shards, bundled-plugin shards, and additional guard matrices when the change is limited to the routing or helper surfaces the fast task exercises directly.
7575
- **Windows Node checks** are scoped to Windows-specific process/path wrappers, npm/pnpm/UI runner helpers, package manager config, and the CI workflow surfaces that execute that lane; unrelated source, plugin, install-smoke, and test-only changes stay on the Linux Node lanes.
7676

77-
The slowest Node test families are split or balanced so each job stays small without over-reserving runners: plugin contracts and channel contracts each run as two weighted Blacksmith-backed shards with the standard GitHub runner fallback, core unit fast/support lanes run separately, core runtime infra is split between state, process/config, shared, and three cron domain shards, auto-reply runs as balanced workers (with the reply subtree split into agent-runner, dispatch, and commands/state-routing shards), and agentic gateway/server configs are split across chat/auth/model/http-plugin/runtime/startup lanes instead of waiting on built artifacts. Broad browser, QA, media, and miscellaneous plugin tests use their dedicated Vitest configs instead of the shared plugin catch-all. Include-pattern shards record timing entries using the CI shard name, so `.artifacts/vitest-shard-timings.json` can distinguish a whole config from a filtered shard. `check-additional-*` keeps package-boundary compile/canary work together and separates runtime topology architecture from gateway watch coverage; the boundary guard list is striped into one prompt-heavy shard and one combined shard for the remaining guard stripes, each running selected independent guards concurrently and printing per-check timings. The expensive Codex happy-path prompt snapshot drift check runs as its own additional job for manual CI and for prompt-affecting changes only, so normal unrelated Node changes do not wait behind cold prompt snapshot generation and the boundary shards stay balanced while prompt drift is still pinned to the PR that caused it; the same flag skips prompt snapshot Vitest generation inside the built-artifact core support-boundary shard. Gateway watch, channel tests, and the core support-boundary shard run concurrently inside `build-artifacts` after `dist/` and `dist-runtime/` are already built.
77+
The slowest Node test families are split or balanced so each job stays small without over-reserving runners: plugin contracts and channel contracts each run as two weighted Blacksmith-backed shards with the standard GitHub runner fallback, core unit fast/support lanes run separately, core runtime infra is split between state, process/config, shared, and three cron domain shards, auto-reply runs as balanced workers (with the reply subtree split into agent-runner, dispatch, and commands/state-routing shards), and agentic gateway/server configs are split across chat/auth/model/http-plugin/runtime/startup lanes instead of waiting on built artifacts. Normal CI then packs only isolated infra include-pattern shards into deterministic bundles of at most 64 test files, reducing the Node matrix without merging non-isolated command/cron, stateful agents-core, or gateway/server suites; heavy fixed suites stay on 8 vCPU while the bundled and lower-weight lanes use 4 vCPU. Broad browser, QA, media, and miscellaneous plugin tests use their dedicated Vitest configs instead of the shared plugin catch-all. Include-pattern shards record timing entries using the CI shard name, so `.artifacts/vitest-shard-timings.json` can distinguish a whole config from a filtered shard. `check-additional-*` keeps package-boundary compile/canary work together and separates runtime topology architecture from gateway watch coverage; the boundary guard list is striped into one prompt-heavy shard and one combined shard for the remaining guard stripes, each running selected independent guards concurrently and printing per-check timings. The expensive Codex happy-path prompt snapshot drift check runs as its own additional job for manual CI and for prompt-affecting changes only, so normal unrelated Node changes do not wait behind cold prompt snapshot generation and the boundary shards stay balanced while prompt drift is still pinned to the PR that caused it; the same flag skips prompt snapshot Vitest generation inside the built-artifact core support-boundary shard. Gateway watch, channel tests, and the core support-boundary shard run concurrently inside `build-artifacts` after `dist/` and `dist-runtime/` are already built.
7878

7979
Android CI runs both `testPlayDebugUnitTest` and `testThirdPartyDebugUnitTest` and then builds the Play debug APK. The third-party flavor has no separate source set or manifest; its unit-test lane still compiles the flavor with the SMS/call-log BuildConfig flags, while avoiding a duplicate debug APK packaging job on every Android-relevant push.
8080

@@ -111,15 +111,15 @@ gh workflow run full-release-validation.yml --ref main -f ref=<branch-or-sha>
111111

112112
## Runners
113113

114-
| Runner | Jobs |
115-
| -------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
116-
| `ubuntu-24.04` | Manual CI dispatch and non-canonical repository fallbacks, workflow-sanity, labeler, auto-response, docs workflows outside CI, and install-smoke preflight so the Blacksmith matrix can queue earlier |
117-
| `blacksmith-4vcpu-ubuntu-2404` | `CodeQL Critical Quality`, `preflight`, `security-fast`, lower-weight extension shards, `checks-fast-core`, plugin/channel contract shards, `checks-node-compat-node22`, `check-guards`, `check-prod-types`, and `check-test-types` |
118-
| `blacksmith-8vcpu-ubuntu-2404` | Linux Node test shards, bundled plugin test shards, `check-additional-*` shards, `check-dependencies`, and `android` |
119-
| `blacksmith-16vcpu-ubuntu-2404` | `build-artifacts`, `check-lint` (CPU-sensitive enough that 8 vCPU cost more than they saved); install-smoke Docker builds (32-vCPU queue time cost more than it saved) |
120-
| `blacksmith-16vcpu-windows-2025` | `checks-windows` |
121-
| `blacksmith-6vcpu-macos-15` | `macos-node` on `openclaw/openclaw`; forks fall back to `macos-15` |
122-
| `blacksmith-12vcpu-macos-26` | `macos-swift` on `openclaw/openclaw`; forks fall back to `macos-26` |
114+
| Runner | Jobs |
115+
| ------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
116+
| `ubuntu-24.04` | Manual CI dispatch and non-canonical repository fallbacks, workflow-sanity, labeler, auto-response, docs workflows outside CI, and install-smoke preflight so the Blacksmith matrix can queue earlier |
117+
| `blacksmith-4vcpu-ubuntu-2404` | `CodeQL Critical Quality`, `preflight`, `security-fast`, lower-weight extension shards, `checks-fast-core`, plugin/channel contract shards, most bundled/lower-weight Linux Node shards, `check-guards`, `check-prod-types`, `check-test-types`, selected `check-additional-*` shards, and `check-dependencies` |
118+
| `blacksmith-8vcpu-ubuntu-2404` | Retained heavy Linux Node suites, boundary/extension-heavy `check-additional-*` shards, and `android` |
119+
| `blacksmith-16vcpu-ubuntu-2404` | `build-artifacts`, `check-lint` (CPU-sensitive enough that 8 vCPU cost more than they saved); install-smoke Docker builds (32-vCPU queue time cost more than it saved) |
120+
| `blacksmith-8vcpu-windows-2025` | `checks-windows` |
121+
| `blacksmith-6vcpu-macos-15` | `macos-node` on `openclaw/openclaw`; forks fall back to `macos-15` |
122+
| `blacksmith-12vcpu-macos-26` | `macos-swift` on `openclaw/openclaw`; forks fall back to `macos-26` |
123123

124124
Canonical-repo CI keeps Blacksmith as the default runner path for normal push and pull-request runs. `workflow_dispatch` and non-canonical repository runs use GitHub-hosted runners, but normal canonical runs do not currently probe Blacksmith queue health or automatically fall back to GitHub-hosted labels when Blacksmith is unavailable.
125125

0 commit comments

Comments
 (0)