feat(vscode): report test run progress by claneo · Pull Request #720 · web-infra-dev/rstest

claneo · 2025-11-27T02:24:19Z

Changes Summary

core
1. Support pass logger to default reporter
2. Add two new reporter hooks: onTestSuiteStart and onTestSuiteResult
3. Support match test case with testNamePattern exactly (This is still a temporary solution that cannot handle cases where the name contains >. I plan to support matching by arrays instead of strings in the future.)
4. Stop stringify actual and expected value in error object
vscode extension
1. New features, see below
2. Simplified updateFromContents, now ancestors stack is poped when exit astnode
3. Enqueued all test items before run them

New Features

Realtime test result
Pipe default reporter to test results panel
Place error message at right location
Display error stack if exist
Display expected and actual value if it is a mismatch error
Support update snapshot if it is a snapshot mismatch error

20251127-095853.mp4

Problems

Sourcemap does not work if debugger attached, causes error stack is wrong
Is it ok to import core source code from vscode extension directly?

Checklist

Tests updated (or not required).
Documentation updated (or not required).

netlify · 2025-11-27T02:24:25Z

✅ Deploy Preview for rstest-dev ready!

Name	Link
🔨 Latest commit	`55e11a1`
🔍 Latest deploy log	https://app.netlify.com/projects/rstest-dev/deploys/69364ceed194cf00085b7a4d
😎 Deploy Preview	https://deploy-preview-720--rstest-dev.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

Copilot

Pull request overview

This PR implements real-time test run progress reporting for the VSCode extension, enabling features like live test results, error location highlighting, and snapshot updating. The changes span both the core testing framework (adding new reporter hooks onTestSuiteStart and onTestSuiteResult) and the VSCode extension (implementing a comprehensive test run reporter with error formatting and stack trace parsing).

Key changes include:

Added suite-level reporter hooks to track test suite execution lifecycle
Implemented exact test name pattern matching with configurable delimiter
Refactored VSCode extension to report progress in real-time via new TestRunReporter class
Enhanced error reporting with actual/expected value formatting using pretty-format
Added support for updating snapshots directly from VSCode test results

Reviewed changes

Copilot reviewed 43 out of 44 changed files in this pull request and generated 9 comments.

Show a summary per file

File	Description
packages/core/src/types/reporter.ts	Added `onTestSuiteStart` and `onTestSuiteResult` hooks, logger option for default reporter
packages/core/src/types/testSuite.ts	Extracted `TestSuiteInfo` type, added `expected`/`actual` fields to `FormattedError`
packages/core/src/types/config.ts	Added `exact` flag for test name pattern matching
packages/core/src/runtime/runner/runner.ts	Implemented suite result tracking and hook invocations
packages/core/src/runtime/runner/task.ts	Added exact matching logic for test name patterns
packages/core/src/utils/error.ts	Made `getSourcemap` parameter optional
packages/vscode/src/testRunReporter.ts	New class implementing real-time test progress reporting with error formatting
packages/vscode/src/worker/reporter.ts	Refactored to use new progress-based reporter architecture
packages/vscode/src/worker/index.ts	Updated to pass reporter and logger instances to rstest
packages/vscode/src/master.ts	Added `onTestProgress` method to handle worker progress events
packages/vscode/src/testTree.ts	Simplified test tree parsing with exit callbacks, removed old result handling
packages/vscode/src/extension.ts	Added snapshot update command and improved test run workflow
packages/vscode/package.json	Added `pretty-format` dependency and snapshot update command configuration
packages/vscode/tests/suite/progress.test.ts	New integration test for progress reporting functionality
e2e/reporter/rstest.customReporterConfig.ts	Added suite hooks to test reporter
pnpm-lock.yaml	Updated dependencies (@rslib/core, @rsbuild/core, @rspack versions)

Files not reviewed (1)

pnpm-lock.yaml: Language not supported

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2025-11-28T14:55:28Z

packages/vscode/tsconfig.json

    "strict": true,
-    "skipLibCheck": true
+    "skipLibCheck": true,
+    "types": ["../core/src/env.d.ts"]


The types compiler option expects an array of package names or paths to type definition files, not a path to a .d.ts file. This should likely be removed and handled differently, or you should use typeRoots instead. The current configuration will likely not work as intended.

packages/vscode/src/master.ts

packages/vscode/tests/suite/progress.test.ts

packages/core/src/types/config.ts

packages/vscode/src/testRunReporter.ts

Copilot · 2025-11-28T14:55:30Z

packages/vscode/src/testRunReporter.ts

+import { ROOT_SUITE_NAME } from '../../core/src/utils/constants';
+import { parseErrorStacktrace } from '../../core/src/utils/error';


This import uses a relative path '../../core/src/utils/constants' to access core package source directly. While the PR description mentions this as a known concern ("Is it ok to import core source code from vscode extension directly?"), this creates tight coupling and build order dependencies. Consider either:

Exporting these constants from @rstest/core package's main entry point

Duplicating the constant in the vscode extension

Creating a shared constants package

The same applies to the import on line 12 (parseErrorStacktrace).

Copilot · 2025-11-28T14:55:31Z

packages/vscode/src/master.ts

+  async onTestProgress<T extends keyof TestRunReporter>(
+    runId: string,
+    event: T,
+    param: Parameters<NonNullable<TestRunReporter[T]>>[0],
+  ) {
+    const reporter = this.testRunReporters.get(runId);
+    // @ts-expect-error


The @ts-expect-error comment suppresses type checking without explanation. Consider adding a comment explaining why this is necessary, or better yet, fixing the type issue. The problem likely stems from TypeScript's inability to ensure param matches the expected parameter type for the specific event method.

Suggested change

async onTestProgress<T extends keyof TestRunReporter>(

runId: string,

event: T,

param: Parameters<NonNullable<TestRunReporter[T]>>[0],

) {

const reporter = this.testRunReporters.get(runId);

// @ts-expect-error

// Map each method name to its first parameter type

type TestRunReporterEventParams = {

[K in keyof TestRunReporter]: TestRunReporter[K] extends (arg: infer P, ...args: any[]) => any ? P : never;

};

async onTestProgress<T extends keyof TestRunReporter>(

runId: string,

event: T,

param: TestRunReporterEventParams[T],

) {

const reporter = this.testRunReporters.get(runId);

Copilot · 2025-11-28T14:55:31Z

e2e/rstest.config.ts

 export default defineConfig({
  setupFiles: ['../scripts/rstest.setup.ts'],
-  testTimeout: process.env.CI ? 10_000 : 5_000,
+  testTimeout: 10_000,


The change removes the conditional timeout based on CI environment (process.env.CI ? 10_000 : 5_000) and sets a fixed 10-second timeout. This may slow down local development since tests will now wait up to 10 seconds even in non-CI environments. Consider keeping the conditional logic to maintain faster feedback in local development.

Suggested change

testTimeout: 10_000,

testTimeout: process.env.CI ? 10_000 : 5_000,

packages/core/src/types/config.ts

fi3ework

generally LGTM, cc @9aoy, PTAL at the core package modification.

packages/core/src/runtime/runner/runner.ts

9aoy · 2025-12-09T06:56:17Z

👍

github-actions bot added the change: feat label Nov 27, 2025

claneo force-pushed the vscode-progress branch 3 times, most recently from eb7033a to fc55505 Compare November 28, 2025 14:44

claneo marked this pull request as ready for review November 28, 2025 14:49

Copilot AI review requested due to automatic review settings November 28, 2025 14:49

Copilot started reviewing on behalf of claneo November 28, 2025 14:50 View session

Copilot finished reviewing on behalf of claneo November 28, 2025 14:53

Copilot AI reviewed Nov 28, 2025

View reviewed changes

fi3ework reviewed Dec 3, 2025

View reviewed changes

packages/core/src/types/config.ts Outdated Show resolved Hide resolved

claneo force-pushed the vscode-progress branch from 6a0f62f to 2023755 Compare December 3, 2025 09:41

claneo mentioned this pull request Dec 5, 2025

feat(vscode): support run tests with debug or coverage, list test items in tree structure #738

Merged

2 tasks

claneo added 4 commits December 7, 2025 17:51

feat(vscode): report test run progress

8e424aa

fix copilot review

b4cd295

replace exact option with regexp match

8f79f41

move pretty-format into core to solve serialize issue

bda78b3

claneo force-pushed the vscode-progress branch from 7cfaa6e to bda78b3 Compare December 7, 2025 09:53

dedupe

0d24f4b

fi3ework reviewed Dec 8, 2025

View reviewed changes

packages/core/src/runtime/runner/runner.ts Outdated Show resolved Hide resolved

fix: RealDate

55e11a1

9aoy merged commit 37d7cc8 into web-infra-dev:main Dec 9, 2025
16 checks passed

9aoy mentioned this pull request Dec 10, 2025

release: 0.7.2 #751

Merged

claneo deleted the vscode-progress branch December 10, 2025 10:56

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat(vscode): report test run progress#720

feat(vscode): report test run progress#720
9aoy merged 6 commits intoweb-infra-dev:mainfrom
claneo:vscode-progress

claneo commented Nov 27, 2025 •

edited

Loading

Uh oh!

netlify bot commented Nov 27, 2025 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Nov 28, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI Nov 28, 2025

Uh oh!

Copilot AI Nov 28, 2025

Uh oh!

Copilot AI Nov 28, 2025

Uh oh!

Uh oh!

fi3ework left a comment

Uh oh!

Uh oh!

9aoy commented Dec 9, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

		import { ROOT_SUITE_NAME } from '../../core/src/utils/constants';
		import { parseErrorStacktrace } from '../../core/src/utils/error';

	testTimeout: 10_000,
	testTimeout: process.env.CI ? 10_000 : 5_000,

Uh oh!

Conversation

claneo commented Nov 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changes Summary

New Features

Problems

Related Links

Checklist

Uh oh!

netlify bot commented Nov 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ Deploy Preview for rstest-dev ready!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Nov 28, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI Nov 28, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 28, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 28, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

fi3ework left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

9aoy commented Dec 9, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

claneo commented Nov 27, 2025 •

edited

Loading

netlify bot commented Nov 27, 2025 •

edited

Loading