Jasmine afterTest() wdio hook returns wrong passed value for failed spec as passed=true #11684
Description
Have you read the Contributing Guidelines on issues?
- I have read the Contributing Guidelines on issues.
WebdriverIO Version
8.20.2
Node.js Version
8.15.0
Mode
WDIO Testrunner
Which capabilities are you using?
browserName: chrome,
browserVersion: 'latest',
platformName: 'Windows 10'
operatingSystem: Windows 10
dependencies in package.json
"@babel/core": "^7.23.2",
"@babel/preset-env": "^7.23.2",
"@babel/register": "^7.22.15",
"@wdio/allure-reporter": "^8.20.0",
"@wdio/cli": "^8.20.2",
"@wdio/jasmine-framework": "^8.23.0",
"@wdio/local-runner": "^8.20.0",
"@wdio/spec-reporter": "^8.20.0",
"allure-commandline": "^2.24.1"What happened?
Hi Team, We are using the WebdriverIO with the Jasmine framework for our test automation project. We observed the afterTest() hook in the WebdriverIO shows the passed value as true for the spec that failed with an expectation. Due to this issue, we cannot take a screenshot of the failed spec and attach it to the allure report. Please find the below framework details.
WebdriverIO version: 8.20.2
Framework: Jasmine
Language: Javascript
Mode: [WDIO Testrunner]
If WDIO Testrunner, running sync/async: [async]
Node.js version: [18.15.0]
NPM version: [9.5.0]
Browser name and version: [Chrome 119]
Platform name and version: [Windows 10
We even tried to use the below check in afterTest() hook, but still, the screenshot is not captured in the allure report failed spec.
afterTest: async function (test, context, { error, result, duration, passed, retries }) {
if (!passed || error) {
await browser.takeScreenshot()
}
}
Attached is the allure report screenshot
What is your expected behavior?
The afterTest() hook should return the correct passed value as true/false for the spec passed/failed due to expectation/assertion
How to reproduce the bug.
Attached is the sample project created to reproduce this issue. The project contains two specs, one will pass and the other will fail due to expectations.
Relevant log output
Execution of 1 workers started at 2023-11-19T14:42:21.622Z
[0-0] RUNNING in chrome - file:///C:/Users/User1/Desktop/JasmineProject/test/specs/test.e2e.js
DevTools listening on ws://127.0.0.1:56785/devtools/browser/8fcda011-3409-4c9b-afb6-7b7d4dff81c6
[26656:30380:1119/201242.995:ERROR:policy_logger.cc(157)] :components\policy\core\browser\policy_conversions_client.cc(317) DeveloperToolsDisabled has an error of type: Ignored because it was overridden by DeveloperToolsAvailability.
[0-0] Error in "My Login application should login with valid credentials and expectation failure"
at <Jasmine>
at UserContext.<anonymous> (C:\Users\User1\Desktop\JasmineProject\test\specs\/test.e2e.js:24:50)
at processTicksAndRejections (node:internal/process/task_queues:95:5)
[0-0] FAILED in chrome - file:///C:/Users/User1/Desktop/JasmineProject/test/specs/test.e2e.js
"spec" Reporter:
------------------------------------------------------------------
[chrome 119.0.6045.160 windows #0-0] Running: chrome (v119.0.6045.160) on windows
[chrome 119.0.6045.160 windows #0-0] Session ID: 6dc13b4a034ab8e2c836fc5a6d043a9f
[chrome 119.0.6045.160 windows #0-0] » \test\specs\/test.e2e.js
[chrome 119.0.6045.160 windows #0-0] My Login application
[chrome 119.0.6045.160 windows #0-0] ✓ should login with valid credentials
[chrome 119.0.6045.160 windows #0-0] ✖ should login with valid credentials and expectation failure
[chrome 119.0.6045.160 windows #0-0]
[chrome 119.0.6045.160 windows #0-0] 1 passing (37.8s)
[chrome 119.0.6045.160 windows #0-0] 1 failing
[chrome 119.0.6045.160 windows #0-0]
[chrome 119.0.6045.160 windows #0-0] 1) My Login application should login with valid credentials and expectation failure
[chrome 119.0.6045.160 windows #0-0] Expect $(`//a/i[text()=' Logout']`) to be disabled
Expected: "disabled"
Received: "not disabled"
[chrome 119.0.6045.160 windows #0-0] at <Jasmine>
[chrome 119.0.6045.160 windows #0-0] at UserContext.<anonymous> (C:\Users\User1\Desktop\JasmineProject\test\specs\/test.e2e.js:24:50)
[chrome 119.0.6045.160 windows #0-0] at processTicksAndRejections (node:internal/process/task_queues:95:5)
Spec Files: 0 passed, 1 failed, 1 total (100% completed) in 00:00:55Code of Conduct
- I agree to follow this project's Code of Conduct
Is there an existing issue for this?
- I have searched the existing issues