Skip to content

perf: use advanced serialization#792

Merged
9aoy merged 2 commits intoweb-infra-dev:mainfrom
claneo:advanced-serialization
Feb 27, 2026
Merged

perf: use advanced serialization#792
9aoy merged 2 commits intoweb-infra-dev:mainfrom
claneo:advanced-serialization

Conversation

@claneo
Copy link
Copy Markdown
Collaborator

@claneo claneo commented Dec 23, 2025

Requires tinylibs/tinypool#128 to be merged.

Summary

Problem

When sending large files to worker, json serialization cost too much time.

image

It will even throw an error when the message is larger.

node:buffer:1154
      data[i] = this[i];
              ^

RangeError: Invalid array length
    at Uint8Array.toJSON (node:buffer:1154:15)
    at stringify (<anonymous>)
    at writeChannelMessage (node:internal/child_process/serialization:164:20)
    at target._send (node:internal/child_process:851:17)
    at target.send (node:internal/child_process:751:19)
    at ProcessWorker.send (file:///project/node_modules/.pnpm/tinypool@1.1.1/node_modules/tinypool/dist/index.js:140:41)
    at EventEmitter.<anonymous> (file:///project/node_modules/.pnpm/tinypool@1.1.1/node_modules/tinypool/dist/index.js:136:9)
    at EventEmitter.emit (node:events:519:28)
    at post (file:///project/node_modules/.pnpm/@rstest+core@0.7.4_patch_hash=2ca3ffb9c737907cda764b112b1056468ffdef8a8e3d1652736072afc_3e469dd1c273e9c5e41855799fa57aa8/node_modules/@rstest/core/dist/0%7E89.js:45:21)
    at EventEmitter.onMessage (file:///project/node_modules/.pnpm/@rstest+core@0.7.4_patch_hash=2ca3ffb9c737907cda764b112b1056468ffdef8a8e3d1652736072afc_3e469dd1c273e9c5e41855799fa57aa8/node_modules/@rstest/core/dist/216.js:155:27)

After switching to advanced serialization, the duration has significantly decreased.

image

Breaking changes

tinypool v2.0.0 dropped Node 18 support.

Changes

  • Update tinypool to v2.1.0
  • Set serialization to 'advanced'
  • Remove all manual serialization

Related Links

Checklist

  • Tests updated (or not required).
  • Documentation updated (or not required).

@netlify
Copy link
Copy Markdown

netlify bot commented Dec 23, 2025

Deploy Preview for rstest-dev ready!

Name Link
🔨 Latest commit 8a85aa5
🔍 Latest deploy log https://app.netlify.com/projects/rstest-dev/deploys/6949feb50ef6810008241403
😎 Deploy Preview https://deploy-preview-792--rstest-dev.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

@9aoy
Copy link
Copy Markdown
Collaborator

9aoy commented Dec 23, 2025

Rstest will drop Node 18 support in Rspack v2.0. Before that, a preview version can be released if needed.
web-infra-dev/rspack#9270

@claneo
Copy link
Copy Markdown
Collaborator Author

claneo commented Dec 23, 2025

a preview version can be released if needed

Which one do you mean? rstest or rspack?

@9aoy
Copy link
Copy Markdown
Collaborator

9aoy commented Dec 23, 2025

Which one do you mean? rstest or rspack?

Rstest

@claneo claneo force-pushed the advanced-serialization branch from 8a85aa5 to 386b111 Compare February 26, 2026 16:08
@claneo claneo marked this pull request as ready for review February 27, 2026 02:51
@claneo claneo requested a review from Copilot February 27, 2026 02:52
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR switches the core worker pool IPC to Tinypool’s advanced serialization to reduce overhead and avoid JSON serialization limits when transferring large payloads between the main process and workers.

Changes:

  • Enable Tinypool serialization: 'advanced' for worker pools and remove manual v8.serialize/v8.deserialize plumbing.
  • Stop manually “serializing” runtime config; adjust test collection/listing to pass TestInfo (and include runMode) instead of full Test objects.
  • Adjust coverage payload typing/shape to use istanbul-lib-coverage’s FileCoverageData-based structure.

Reviewed changes

Copilot reviewed 14 out of 14 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
packages/core/src/types/testSuite.ts Move runMode into info types and adjust coverage type for worker payloads.
packages/core/src/types/core.ts Update list command result to return TestInfo[].
packages/core/src/runtime/worker/rpc.ts Remove custom v8 (de)serialization from Birpc options.
packages/core/src/runtime/worker/index.ts Remove config “undo” step and adjust coverage attachment shape.
packages/core/src/runtime/runner/runner.ts Include runMode in suite/case hook payloads.
packages/core/src/runtime/runner/index.ts Centralize Test -> TestInfo mapping and include runMode.
packages/core/src/runtime/api/index.ts Update runtime collectTests typing to TestInfo[].
packages/core/src/pool/index.ts Stop wrapping config for JSON and change collect results typing to TestInfo[].
packages/core/src/pool/forks.ts Enable Tinypool advanced serialization and remove v8 (de)serialization.
packages/core/src/core/listTests.ts Traverse TestInfo for listing and use runMode filtering.
packages/core/src/core/globalSetup.ts Enable advanced serialization for the global setup pool.
packages/core/src/browserRuntime.ts Export TestInfo from browser-runtime types.
packages/browser/src/protocol.ts Update browser protocol collect-result payload to TestInfo[].
.vscode/tasks.json Remove shared problem matcher block (VS Code task config change).
Comments suppressed due to low confidence (1)

packages/core/src/pool/index.ts:165

  • createPool now declares collectTests returns TestInfo[], but createForksPool (packages/core/src/pool/forks.ts) still types collectTests as returning Test[]. Test is not assignable to TestInfo (notably TestSuite.tests is Test[], not TestInfo[]), so this should cause a TypeScript type mismatch and also contradicts the worker-side change to return TestInfo[]. Update the forks pool return type (and any related imports/annotations) to use TestInfo[] for collect results to keep types consistent end-to-end.
  }) => Promise<
    {
      tests: TestInfo[];
      testPath: string;
      errors?: FormattedError[];
      project: string;
    }[]
  >;

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@claneo claneo requested a review from 9aoy February 27, 2026 06:08
@9aoy 9aoy merged commit 69e91b3 into web-infra-dev:main Feb 27, 2026
13 checks passed
@9aoy
Copy link
Copy Markdown
Collaborator

9aoy commented Feb 27, 2026

👍

@claneo claneo deleted the advanced-serialization branch February 27, 2026 07:01
@fi3ework fi3ework mentioned this pull request Mar 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants