Skipping a test in beforeTest global hook is not preventing the test runner from executing the entire test code #6645
Description
Environment (please complete the following information):
- WebdriverIO version: 7.3.0
- Mode: Wdio
- If WDIO Testrunner, running sync/async: sync
- Node.js version: 12.18.2
- NPM version: 6.14.5
- Browser name and version: Chrome 89 (but it is not browser related)
- Platform name and version: Ubuntu 20.04.1
- Additional wdio packages used (if applicable): @wdio/mocha-framework, @wdio/local-runner, @wdio/sync, @wdio/spec-reporter
Config of WebdriverIO
exports.config = {
runner: 'local',
specs: [
'./test/specs/**/*.js'
],
maxInstances: 10,
capabilities: [{
maxInstances: 5,
browserName: 'chrome',
acceptInsecureCerts: true
}],
logLevel: 'debug',
bail: 0,
baseUrl: 'http://localhost',
waitforTimeout: 10000,
connectionRetryTimeout: 120000,
connectionRetryCount: 3,
services: ['chromedriver'],
framework: 'mocha',
reporters: ['spec'],
mochaOpts: {
ui: 'bdd',
timeout: 60000
},
beforeTest: function (test, context) {
console.log('... Skipping test in beforeTest hoook wdio.conf.js');
context.skip();
},
}
Spec Example
describe('Skip Example suite', () => {
it('should be skipped', function () {
browser.url(`https://google.com`);
console.log('THIS TEXT SHOULD NOT BE PRINTED');
});
});
Describe the bug
When I skip a test programatically using the beforeTest function, the reporter prints the test as skipped accordingly, but the test execution is not aborted, therefore the entire code within it function is executed even though the test was skipped before initializing.
If I skip programatically using this.skip() inside it or within a beforeEach hook in the spec file it works, but I need this logic at global level, that's why I need it to work correctly.
To Reproduce
Run the test with the spec and config provided.
Report generated:
Execution of 1 spec files started at 2021-04-01T01:26:49.245Z
Starting ChromeDriver 88.0.4324.96 (68dba2d8a0b149a1d3afac56fa74648032bcf46b-refs/branch-heads/4324@{#1784}) on port 9515
Only local connections are allowed.
Please see https://chromedriver.chromium.org/security-considerations for suggestions on keeping ChromeDriver safe.
ChromeDriver was started successfully.
[0-0] RUNNING in chrome - /test/specs/skip-example.js
[0-0] ... Skipping test in beforeTest hoook wdio.conf.js
[0-0] 2021-04-01T01:26:50.295Z ERROR @wdio/sync: undefined
[0-0] THIS TEXT SHOULD NOT BE PRINTED
[0-0] PASSED in chrome - /test/specs/skip-example.js
"spec" Reporter:
------------------------------------------------------------------
[chrome 88.0.4324.96 linux #0-0] Running: chrome (v88.0.4324.96) on linux
[chrome 88.0.4324.96 linux #0-0] Session ID: 376b61a6729d0b1caf50a78292a535c5
[chrome 88.0.4324.96 linux #0-0]
[chrome 88.0.4324.96 linux #0-0] » /test/specs/skip-example.js
[chrome 88.0.4324.96 linux #0-0] Skip Example suite
[chrome 88.0.4324.96 linux #0-0] ? should be skipped
[chrome 88.0.4324.96 linux #0-0] - should be skipped
[chrome 88.0.4324.96 linux #0-0]
[chrome 88.0.4324.96 linux #0-0] 1 skipped (1.3s)
Spec Files: 1 passed, 1 total (100% completed) in 00:00:02
The scenario was displayed as skipped, but the code inside it block was executed, otherwise the string THIS TEXT SHOULD NOT BE PRINTED wouldn't be printed
Expected behavior
The code inside it block should not be executed, same behavior when I use it.skip or this.skip() inside it
Log
https://gist.github.com/roggerfe/d0927cdf383c5488c7335e3a9ff4651c