[🐛 Bug]: afterTest hook contains wrong test object but correct error in a specific scenario #11495
Description
Have you read the Contributing Guidelines on issues?
- I have read the Contributing Guidelines on issues.
WebdriverIO Version
8.20.2
Node.js Version
18.6.1
Mode
Standalone Mode
Which capabilities are you using?
No response
What happened?
The afterTest hook is called containing the wrong test object but correct error in a specific scenario
Using the example spec from the documentation I have added two tests to demonstrate the issue. The first one fails due to a time out because the last line does not contain the text it is expecting ('d' removed from 'logged'). The second test case fails because of an exception (k not defined).
- When running with Mocha test framework - the afterTest hook is called correctly.
- When running with the Jasmine test framework - the afterTest hook is called incorrectly. The test object in both cases is the second test case, but the error is different (and correct) in each invocation.
import { expect } from '@wdio/globals'
import LoginPage from '../pageobjects/login.page.js'
import SecurePage from '../pageobjects/secure.page.js'
describe('My Login application', () => {
it('timeout failure', async () => {
await LoginPage.open()
await LoginPage.login('tomsmith', 'SuperSecretPassword!')
await expect(true).toBe(true);
await expect(SecurePage.flashAlert).toBeExisting()
await expect(SecurePage.flashAlert).toHaveTextContaining(
'You logge into a secure area!')
})
it('exception fail rather than assert', async () => {
await LoginPage.open()
await LoginPage.login('tomsmith', 'SuperSecretPassword!')
k++; // reference error, k not defined
await expect(true).toBe(true);
await expect(SecurePage.flashAlert).toBeExisting()
await expect(SecurePage.flashAlert).toHaveTextContaining(
'You logged into a secure area!')
})
})
If you change the order of the test cases (in this case reverse them) so that the timed out test is last then the afterTest hook is called correctly for both Mocha and Jasmin:
import { expect } from '@wdio/globals'
import LoginPage from '../pageobjects/login.page.js'
import SecurePage from '../pageobjects/secure.page.js'
describe('My Login application', () => {
it('exception fail rather than assert', async () => {
await LoginPage.open()
await LoginPage.login('tomsmith', 'SuperSecretPassword!')
k++; // reference error, k not defined
await expect(true).toBe(true);
await expect(SecurePage.flashAlert).toBeExisting()
await expect(SecurePage.flashAlert).toHaveTextContaining(
'You logged into a secure area!')
})
it('timeout failure', async () => {
await LoginPage.open()
await LoginPage.login('tomsmith', 'SuperSecretPassword!')
await expect(true).toBe(true);
await expect(SecurePage.flashAlert).toBeExisting()
await expect(SecurePage.flashAlert).toHaveTextContaining(
'You logge into a secure area!')
})
})
What is your expected behavior?
The afterTest hook should contain the correct test object.
How to reproduce the bug.
- Create a new wdio project
- Use the example spec from the documentation but modified in the way shown above in the 'What happened' section.
- Set test framework to Jasmine (the issue does not occur with Mocha)
- Add a console.log of the test object return in the afterTest hook (or use debugger/breakdpoint).
- You will see the same test object in the afterTest hook but the error will be different (and correct) in the two invocations.
Relevant log output
Follow steps in How to reproduce the bug. This can be reproduced with a new project quickly.Code of Conduct
- I agree to follow this project's Code of Conduct
Is there an existing issue for this?
- I have searched the existing issues