Skip to content

perf(core): lazy load runtime diff formatting and fake timers#1079

Merged
fi3ework merged 3 commits intomainfrom
perf/lazy-runtime-loaders
Mar 20, 2026
Merged

perf(core): lazy load runtime diff formatting and fake timers#1079
fi3ework merged 3 commits intomainfrom
perf/lazy-runtime-loaders

Conversation

@fi3ework
Copy link
Copy Markdown
Member

@fi3ework fi3ework commented Mar 19, 2026

bench diff

~19% improvement for part of E2E testing in Rstest codebase

hyperfine --warmup 8 --runs 20 \
  --command-name 'dev1-e2e-slice' "cd '/Users/fi3ework/OSS/worktrees/wtp/dev1/e2e' && pnpm test test-api lifecycle projects" \
  --command-name 'rstest-e2e-slice' "cd '/Users/fi3ework/OSS/rstest/e2e' && pnpm test test-api lifecycle projects"
Benchmark 1: dev1-e2e-slice
  Time (mean ± σ):      8.828 s ±  0.339 s    [User: 45.502 s, System: 13.569 s]
  Range (min … max):    8.569 s … 10.194 s    20 runs
 
  Warning: Statistical outliers were detected. Consider re-running this benchmark on a quiet system without any interferences from other programs. It might help to use the '--warmup' or '--prepare' options.
 
Benchmark 2: rstest-e2e-slice
  Time (mean ± σ):     10.539 s ±  0.448 s    [User: 48.182 s, System: 14.123 s]
  Range (min … max):    9.968 s … 11.870 s    20 runs
 
Summary
  dev1-e2e-slice ran
    1.19 ± 0.07 times faster than rstest-e2e-slice

Summary

Background

Worker startup eagerly loads diff-formatting and fake-timer internals even when a test run never uses them, which adds avoidable runtime setup work.

Implementation

  • switch runtime diff formatting to async lazy imports and await error formatting at each call site
  • lazy load @sinonjs/fake-timers when FakeTimers is constructed so unused workers skip that module initialization
  • keep the public runtime behavior unchanged while preserving merged error metadata and sync fake-timer APIs

User Impact

Test runs that do not exercise diff output or fake timers pay less worker startup overhead.

Checklist

  • Tests updated (or not required).
  • Documentation updated (or not required).

@fi3ework fi3ework marked this pull request as ready for review March 19, 2026 15:41
@fi3ework fi3ework marked this pull request as draft March 19, 2026 15:43
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 88cb488e7c

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Capture error.expected and error.actual into local variables before
the await point in formatTestError. This prevents microtask callbacks
from mutating the values between the yield and the diff/format calls.
@fi3ework fi3ework marked this pull request as ready for review March 20, 2026 06:40
@fi3ework fi3ework merged commit 9c6b40f into main Mar 20, 2026
20 of 26 checks passed
@fi3ework fi3ework deleted the perf/lazy-runtime-loaders branch March 20, 2026 06:41
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 37fa4f9971

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

parentNames: test.parentNames,
name: test.name,
errors: formatTestError(error, test),
errors: await formatTestError(error, test),
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Keep failure formatting out of the test-hook critical path

Even though formatTestError now snapshots expected/actual, the new await here still yields before runTestsCase reaches afterEachFns, test.context.task.result, and test.onFailed. If a failing test queued a microtask/promise callback before throwing, that callback can now run before cleanup and failure hooks, so afterEach/onTestFailed handlers may observe side effects or extra rejections that happened after the assertion failed. This changes hook ordering for ordinary test failures in packages/core/src/runtime/runner/runner.ts and can make cleanup logic flaky.

Useful? React with 👍 / 👎.

err: any,
test?: Test,
): Promise<FormattedError[]> => {
const errors = Array.isArray(err) ? err : [err];
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Snapshot mutable error arrays before awaiting diff imports

formatTestError now keeps a live reference to any array passed in, but callers like packages/core/src/runtime/worker/index.ts pass the shared unhandledErrors array while the process-level error listeners stay installed until teardown(). Once loadDiffModules() awaits, any late unhandledRejection/uncaughtException pushed into that same array during the import window is omitted from the formatted result, so a test file can log multiple late errors but only report the ones present before formatting started.

Useful? React with 👍 / 👎.

@9aoy 9aoy mentioned this pull request Mar 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant