feat(wdio-runner): automatically include SoftAssertionService by JustasMonkev · Pull Request #14546 · webdriverio/webdriverio

JustasMonkev · 2025-06-05T15:52:46Z

Proposed changes

This change integrates the SoftAssertionService from expect-webdriverio
directly into the wdio-runner package, making soft assertions available
to all WebdriverIO users without requiring manual configuration.

Users can now use expect.soft() to collect assertion failures without
halting test execution, improving the testing experience by providing
more comprehensive test reports.

With minimal performance impact, adding this feature by default eliminates
an unnecessary configuration step and reduces potential setup errors.

Related to: webdriverio/expect-webdriverio#1836

Types of changes

Polish (an improvement for an existing feature)
Bugfix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Breaking change (fix or feature that would cause existing functionality to not work as expected)
Documentation update (improvements to the project's docs)
Specification changes (updates to WebDriver command specifications)
Internal updates (everything related to internal scripts, governance documentation and CI files)

Checklist

I have read the CONTRIBUTING doc
I have added tests that prove my fix is effective or that my feature works
I have added the necessary documentation (if appropriate)
I have added proper type definitions for new commands (if appropriate)

Backport Request

//: # (The current main branch is the development branch for WebdriverIO v9. If your change should be released to the current major version of WebdriverIO (v8), please raise another PR with the same changes against the v8 branch.)

This change is solely for v9 and doesn't need to be back-ported
Back-ported PR at #XXXXX

Further comments

Reviewers: @webdriverio/project-committers

packages/wdio-runner/src/index.ts

website/docs/Assertion.md

…entation

pkg-pr-new · 2025-06-06T20:24:32Z

Open in StackBlitz

eslint-plugin-wdio

npm i https://pkg.pr.new/webdriverio/webdriverio/eslint-plugin-wdio@14546

@wdio/allure-reporter

npm i https://pkg.pr.new/webdriverio/webdriverio/@wdio/allure-reporter@14546

@wdio/appium-service

npm i https://pkg.pr.new/webdriverio/webdriverio/@wdio/appium-service@14546

@wdio/browser-runner

npm i https://pkg.pr.new/webdriverio/webdriverio/@wdio/browser-runner@14546

@wdio/browserstack-service

npm i https://pkg.pr.new/webdriverio/webdriverio/@wdio/browserstack-service@14546

@wdio/cli

npm i https://pkg.pr.new/webdriverio/webdriverio/@wdio/cli@14546

@wdio/concise-reporter

npm i https://pkg.pr.new/webdriverio/webdriverio/@wdio/concise-reporter@14546

@wdio/config

npm i https://pkg.pr.new/webdriverio/webdriverio/@wdio/config@14546

@wdio/cucumber-framework

npm i https://pkg.pr.new/webdriverio/webdriverio/@wdio/cucumber-framework@14546

@wdio/dot-reporter

npm i https://pkg.pr.new/webdriverio/webdriverio/@wdio/dot-reporter@14546

@wdio/firefox-profile-service

npm i https://pkg.pr.new/webdriverio/webdriverio/@wdio/firefox-profile-service@14546

@wdio/globals

npm i https://pkg.pr.new/webdriverio/webdriverio/@wdio/globals@14546

@wdio/jasmine-framework

npm i https://pkg.pr.new/webdriverio/webdriverio/@wdio/jasmine-framework@14546

@wdio/json-reporter

npm i https://pkg.pr.new/webdriverio/webdriverio/@wdio/json-reporter@14546

@wdio/junit-reporter

npm i https://pkg.pr.new/webdriverio/webdriverio/@wdio/junit-reporter@14546

@wdio/lighthouse-service

npm i https://pkg.pr.new/webdriverio/webdriverio/@wdio/lighthouse-service@14546

@wdio/local-runner

npm i https://pkg.pr.new/webdriverio/webdriverio/@wdio/local-runner@14546

@wdio/logger

npm i https://pkg.pr.new/webdriverio/webdriverio/@wdio/logger@14546

@wdio/mocha-framework

npm i https://pkg.pr.new/webdriverio/webdriverio/@wdio/mocha-framework@14546

@wdio/protocols

npm i https://pkg.pr.new/webdriverio/webdriverio/@wdio/protocols@14546

@wdio/repl

npm i https://pkg.pr.new/webdriverio/webdriverio/@wdio/repl@14546

@wdio/reporter

npm i https://pkg.pr.new/webdriverio/webdriverio/@wdio/reporter@14546

@wdio/runner

npm i https://pkg.pr.new/webdriverio/webdriverio/@wdio/runner@14546

@wdio/sauce-service

npm i https://pkg.pr.new/webdriverio/webdriverio/@wdio/sauce-service@14546

@wdio/shared-store-service

npm i https://pkg.pr.new/webdriverio/webdriverio/@wdio/shared-store-service@14546

@wdio/smoke-test-cjs-service

npm i https://pkg.pr.new/webdriverio/webdriverio/@wdio/smoke-test-cjs-service@14546

@wdio/smoke-test-reporter

npm i https://pkg.pr.new/webdriverio/webdriverio/@wdio/smoke-test-reporter@14546

@wdio/smoke-test-service

npm i https://pkg.pr.new/webdriverio/webdriverio/@wdio/smoke-test-service@14546

@wdio/spec-reporter

npm i https://pkg.pr.new/webdriverio/webdriverio/@wdio/spec-reporter@14546

@wdio/static-server-service

npm i https://pkg.pr.new/webdriverio/webdriverio/@wdio/static-server-service@14546

@wdio/sumologic-reporter

npm i https://pkg.pr.new/webdriverio/webdriverio/@wdio/sumologic-reporter@14546

@wdio/testingbot-service

npm i https://pkg.pr.new/webdriverio/webdriverio/@wdio/testingbot-service@14546

@wdio/types

npm i https://pkg.pr.new/webdriverio/webdriverio/@wdio/types@14546

@wdio/utils

npm i https://pkg.pr.new/webdriverio/webdriverio/@wdio/utils@14546

@wdio/webdriver-mock-service

npm i https://pkg.pr.new/webdriverio/webdriverio/@wdio/webdriver-mock-service@14546

webdriver

npm i https://pkg.pr.new/webdriverio/webdriverio/webdriver@14546

webdriverio

npm i https://pkg.pr.new/webdriverio/webdriverio@14546

commit: c4324c0

christian-bromann

Minor updates but overall 👍 LGTM

website/docs/Configuration.md

website/docs/Assertion.md

christian-bromann

LGTM 👍

christian-bromann · 2025-06-09T16:32:44Z

@JustasMonkev in order to get the unit tests pass we have to extend the mock for it we use across all unit tests: https://github.com/webdriverio/webdriverio/blob/main/__mocks__/expect-webdriverio.ts

JustasMonkev · 2025-06-09T16:33:47Z

@JustasMonkev in order to get the unit tests pass we have to extend the mock for it we use across all unit tests: https://github.com/webdriverio/webdriverio/blob/main/__mocks__/expect-webdriverio.ts

@christian-bromann I will check it today, and try to fix it

christian-bromann · 2025-06-09T18:29:59Z

@JustasMonkev I wonder if these snapshot issues are related to an updated to the expect package, thoughts?

JustasMonkev · 2025-06-09T18:38:31Z

@JustasMonkev I wonder if these snapshot issues are related to an updated to the expect package, thoughts?

probably checking it, rigth now

JustasMonkev · 2025-06-09T18:41:08Z

I think the issue is related to this: vitest-dev/vitest#7322
The expect library had many dependency updates in this release: @christian-bromann
https://github.com/webdriverio/expect-webdriverio/releases/tag/v5.2.1

christian-bromann · 2025-06-09T19:16:12Z

Let's just adjust the snapshot for now and move on. At some point I would love to replace the jest expect library with the Vitest one.

JustasMonkev · 2025-06-10T05:47:22Z

Let's just adjust the snapshot for now and move on. At some point I would love to replace the jest expect library with the Vitest one.

I commented out the failing part. When running the snapshot test, it passes, but it doesn't start the other tests it just exits after the first run is completed.

[0-0] This is the global teardown script speaking!
[0-0] PASSED in chrome - webdriverio/e2e/browser-runner/react.test.tsx

 "dot" Reporter:
..

 "spec" Reporter:
------------------------------------------------------------------
[chrome 139.0.7229.0 windows #0-0] Running: chrome (v139.0.7229.0) on windows
[chrome 139.0.7229.0 windows #0-0] Session ID: 5b1a76f00d5369145bc7030032408534
[chrome 139.0.7229.0 windows #0-0]
[chrome 139.0.7229.0 windows #0-0] React Component Testing
[chrome 139.0.7229.0 windows #0-0]    ✓ Test theme button toggle
[chrome 139.0.7229.0 windows #0-0]    ✓ supports snapshot tests
[chrome 139.0.7229.0 windows #0-0]
[chrome 139.0.7229.0 windows #0-0] 2 passing (16.8s)


Snapshot Summary:
1 snapshot(s) unchecked.

Spec Files:      1 passed, 1 total (100% completed) in 00:01:03

---------------------|---------|----------|---------|---------|-------------------
File                 | % Stmts | % Branch | % Funcs | % Lines | Uncovered Line #s 
---------------------|---------|----------|---------|---------|-------------------
All files            |     100 |    83.33 |     100 |     100 | 
 __fixtures__        |     100 |      100 |     100 |     100 | 
  setup.js           |     100 |      100 |     100 |     100 | 
 components          |     100 |    83.33 |     100 |     100 | 
  ReactComponent.jsx |     100 |    83.33 |     100 |     100 | 7
---------------------|---------|----------|---------|---------|-------------------

> e2e@ test:browser:react:verifyCoverage \webdriverio\e2e
> run-s verifyCoverage


> e2e@ verifyCoverage \webdriverio\e2e
> test -d ./browser-runner/coverage

 ELIFECYCLE  Command failed with exit code 1.
ERROR: "verifyCoverage" exited with 1.
 ELIFECYCLE  Command failed with exit code 1.
ERROR: "test:browser:react:verifyCoverage" exited with 1.
 ELIFECYCLE  Command failed with exit code 1.
ERROR: "test:browser:react" exited with 1.
 ELIFECYCLE  Command failed with exit code 1.
 ELIFECYCLE  Command failed with exit code 1.
ERROR: "test:component:run" exited with 1.
 ELIFECYCLE  Command failed with exit code 1.

christian-bromann · 2025-06-10T18:12:46Z

1 snapshot(s) unchecked.

I think because of this ☝️

…nent tests

JustasMonkev · 2025-06-11T18:26:31Z

1 snapshot(s) unchecked.

I think because of this ☝️

same is also in main branch 😄
https://github.com/webdriverio/webdriverio/actions/runs/15588961026/job/43902809841#step:6:51

christian-bromann

LGTM 👍

I will fix the e2e test issue on main.

wdio-bot · 2025-06-11T22:27:11Z

Hey JustasMonkev 👋

Thank you for your contribution to WebdriverIO! Your pull request has been marked as an "Expensable" contribution.

We've sent you an email with further instructions on how to claim your expenses from our development fund.
Please make sure to check your spam folder as well. If you have any questions, feel free to reach out to us at expense@webdriver.io or in the contributing channel on Discord.

We are looking forward to more contributions from you in the future 🙌

Have a nice day,
The WebdriverIO Team 🤖

…verio#14546) * feat: automatically include SoftAssertionService in the core * fix(docs): clarify default inclusion of SoftAssertionService in documentation * feat: add autoAssertOnTestEnd option to control soft assertion behavior * Update website/docs/Configuration.md * Update website/docs/Assertion.md * refactor(docs): reorganize Soft Assertions section for clarity * feat: add SoftAssertionService to expect-webdriverio mock * fix(tests): adjust snapshot formatting for consistency in React component tests --------- Co-authored-by: Christian Bromann <git@bromann.dev>

dsufriawan · 2025-06-22T11:39:57Z

Hi, when can we expect this to be published in NPM?
WDIO version in NPM is still 9.1.5

christian-bromann · 2025-06-23T16:28:49Z

We've just published v9.16.0 that includes these changes.

feat: automatically include SoftAssertionService in the core

d17875e

JustasMonkev mentioned this pull request Jun 5, 2025

feat: add soft assertions feature webdriverio/expect-webdriverio#1836

Merged

christian-bromann reviewed Jun 6, 2025

View reviewed changes

packages/wdio-runner/src/index.ts Show resolved Hide resolved

website/docs/Assertion.md Outdated Show resolved Hide resolved

JustasMonkev and others added 3 commits June 6, 2025 08:50

fix(docs): clarify default inclusion of SoftAssertionService in docum…

02660d6

…entation

Merge branch 'webdriverio:main' into feat/auto-enable-soft-assertions

c703b20

feat: add autoAssertOnTestEnd option to control soft assertion behavior

66edea4

JustasMonkev requested a review from christian-bromann June 6, 2025 19:28

christian-bromann approved these changes Jun 6, 2025

View reviewed changes

website/docs/Configuration.md Outdated Show resolved Hide resolved

website/docs/Assertion.md Outdated Show resolved Hide resolved

christian-bromann added 2 commits June 6, 2025 13:25

Update website/docs/Configuration.md

771427a

Update website/docs/Assertion.md

e43d3c3

christian-bromann added PR: Polish 💅 PRs that contain improvements on existing features PR: New Feature 🚀 PRs that contain new features and removed PR: Polish 💅 PRs that contain improvements on existing features labels Jun 6, 2025

christian-bromann requested changes Jun 6, 2025

View reviewed changes

website/docs/Assertion.md Outdated Show resolved Hide resolved

refactor(docs): reorganize Soft Assertions section for clarity

236a727

JustasMonkev requested a review from christian-bromann June 8, 2025 04:26

christian-bromann approved these changes Jun 8, 2025

View reviewed changes

feat: add SoftAssertionService to expect-webdriverio mock

f48f28e

fix(tests): adjust snapshot formatting for consistency in React compo…

068d5c6

…nent tests

Merge branch 'main' into feat/auto-enable-soft-assertions

c4324c0

christian-bromann approved these changes Jun 11, 2025

View reviewed changes

christian-bromann merged commit 10cddfd into webdriverio:main Jun 11, 2025
6 checks passed

wdio-bot added the Expensable $25 💸 label Jun 11, 2025

mato533 mentioned this pull request Jun 13, 2025

fix(wdio-utils):fix to support windows path string #14565

Merged

13 tasks

Uh oh!

Conversation

JustasMonkev commented Jun 5, 2025

Proposed changes

Types of changes

Checklist

Backport Request

Further comments

Reviewers: @webdriverio/project-committers

Uh oh!

Uh oh!

Uh oh!

pkg-pr-new bot commented Jun 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

christian-bromann left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

christian-bromann left a comment

Choose a reason for hiding this comment

Uh oh!

christian-bromann commented Jun 9, 2025

Uh oh!

JustasMonkev commented Jun 9, 2025

Uh oh!

christian-bromann commented Jun 9, 2025

Uh oh!

JustasMonkev commented Jun 9, 2025

Uh oh!

JustasMonkev commented Jun 9, 2025

Uh oh!

christian-bromann commented Jun 9, 2025

Uh oh!

JustasMonkev commented Jun 10, 2025

Uh oh!

christian-bromann commented Jun 10, 2025

Uh oh!

JustasMonkev commented Jun 11, 2025

Uh oh!

christian-bromann left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

wdio-bot commented Jun 11, 2025

Uh oh!

dsufriawan commented Jun 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

christian-bromann commented Jun 23, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

pkg-pr-new bot commented Jun 6, 2025 •

edited

Loading

dsufriawan commented Jun 22, 2025 •

edited

Loading