[🐛 Bug]: Cucumber profiles are not being loaded with WebDriverIO v7 and Cucumber 7 #7692
Description
Have you read the Contributing Guidelines on issues?
- I have read the Contributing Guidelines on issues.
WebdriverIO Version
7.16.5
Node.js Version
12.20.1
Mode
WDIO Testrunner
Which capabilities are you using?
{
"hostname": "localhost",
"port": 4444,
"path": "/",
"user": "webdriverio",
"key": "xxxxxxxxxxxxxxxx-xxxxxx-xxxxx-xxxxxxxxx",
"region": "us",
"specs": [
"main/features/**"
],
"suites": {
"android": [
"main/features/android/**/*.feature"
],
"ios": [
"main/features/ios/**/*.feature"
],
"localizations_android": [
"main/features/android/localizations/**/*.feature"
],
"localizations_ios": [
"main/features/ios/localizations/**/*.feature"
]
},
"exclude": [],
"logLevel": "info",
"outputDir": "log",
"baseUrl": "http://localhost:8080",
"framework": "cucumber",
"cucumberOpts": {
"timeout": 30000,
"profile": [
"dry-run"
],
"tagExpression": "@android and @run and not @skip and not @wip and not @blocked"
},
"reporters": [
"spec"
],
"runner": "local",
"specFileRetries": 0,
"specFileRetriesDelay": 2,
"specFileRetriesDeferred": false,
"sync": true,
"deprecationWarnings": true,
"bail": 0,
"waitforTimeout": 10000,
"connectionRetryTimeout": 90000,
"connectionRetryCount": 3,
"services": [
[
"appium",
{
"args": {
"relaxedSecurity": true,
"address": "localhost",
"basePath": "/wd/hub",
"port": 4723
},
"command": "appium"
}
]
],
"capabilities": [
{
"appium:locale": "US",
"appium:language": "en",
"appium:deviceName": "pixel_3a",
"appium:avd": "Pixel_3a_API_30",
"appium:platformVersion": "11.0",
"appium:app": "apps/android/my-app.apk",
"appium:printPageSourceOnFindFailure": true,
"excludeDriverLogs": [
"bugreport",
"performance"
],
"platformName": "Android",
"maxInstances": 1,
"appium:orientation": "PORTRAIT",
"appium:automationName": "UiAutomator2",
"appium:noReset": false,
"appium:fullReset": true,
"appium:allowTestPackages": true,
"appium:newCommandTimeout": 240,
"goog:chromeOptions": {
"args": [
"no-first-run",
"disable-fre",
"no-default-browser-check"
]
}
}
]
}What happened?
It looks like using a Cucumber profile does not load in correctly when specified in the capabilities. No matter what, it is ultimately skipped, and the Cucumber options must be directly specified with cucumberOpts, rendering the cucumberOpts.profile option as completely irrelevant.
I have a cucumber.js file defined at the root of my project directory as defined by here but even that does not do anything for me, even when I require the profiles before I finish loading my capabilities.
With the expanded capabilities above, I've tried to set a profile in two ways:
- Specifying the profile I want to use directly, ie.
const caps = {
cucumberOpts: {
profile: ['dry-run'],
timeout: 30000
}
}- By expanding the profile string completely within the
profileoption:
const profiles = require('../../cucumber.js')
const caps = {
cucumberOpts: {
//--require '/Users/zachhamm/Documents/GitHub/automation-suite/main/support/constants.js' --require '/Users/zachhamm/Documents/GitHub/automation-suite/main/step_definitions/**/*.js' --require '/Users/zachhamm/Documents/GitHub/automation-suite/main/support/hooks/*.hooks.js' --tags '@android and @run and not @skip and not @wip and not @blocked' --format html:./cucumber-report.html --dry-run
profile: [profiles['dry-run']],
timeout: 30000
}
}Notice that I am not requiring any steps in the main cucumberOpts, but instead, requiring them through the profile. When I do that, it will fail because it can't find any of the steps to require.
If I select the dry-run profile using the cucumber-js command line option, it will load those required steps before running the feature and it works:
cucumber-js main/features/android/zachtest.feature -p dry-run
------------------------
3 scenarios (3 skipped)
9 steps (9 skipped)
0m00.036s (executing steps: 0m00.000s)
I've also tried where I've required the files in the cucumberOpts.require and then tried with the dry-run profile, hoping for a dry run, but alas, it still would not execute it as a dry run.
My conclusion is ultimately that the cucumberOpts.profile is not being loaded.
What is your expected behavior?
If loading a Cucumber profile in the cucumber.js file into the cucumberOpts.profile array, then the capabilities shall merge those options to the Cucumber framework as it does with the cucumberOpts object.
How to reproduce the bug.
- Make sure your step definitions and hooks exist in a directory structure like such:
` cucumber.js
* apps
* android
` my-app.apk
* main
* config
` environment.js
* features
* support
* step_definitions
` *.steps.js
* hooks
` *.hooks.js
- Create a profile in the
cucumber.jsfile as such:
// cucumber.js
const path = require('path');
const dryRun = [
`--require '${path.join(__dirname, 'main/support/step_definitions/**/*.js'}'`,
`--require '${path.join(__dirname, 'main/support/hooks/**/*.js'}'`
`--format progress-bar`
`--dry-run`
].concat(' ');
modules.export = {
'dry-run': dryRun
}- Use the capabilities (replacing your device and feature paths as needed):
main/config/environment.js
{
"hostname": "localhost",
"port": 4444,
"path": "/",
"user": "webdriverio",
"key": "xxxxxxxxxxxxxxxx-xxxxxx-xxxxx-xxxxxxxxx",
"region": "us",
"specs": [
"main/features/**"
],
"logLevel": "info",
"outputDir": "log",
"baseUrl": "http://localhost:8080",
"framework": "cucumber",
"cucumberOpts": {
"timeout": 30000,
"profile": [
"dry-run"
],
"tagExpression": "@android and @run and not @skip and not @wip and not @blocked"
},
"reporters": [
"spec"
],
"runner": "local",
"specFileRetries": 0,
"specFileRetriesDelay": 2,
"specFileRetriesDeferred": false,
"sync": true,
"bail": 0,
"waitforTimeout": 10000,
"connectionRetryTimeout": 90000,
"connectionRetryCount": 3,
"services": [
[
"appium",
{
"args": {
"relaxedSecurity": true,
"address": "localhost",
"basePath": "/wd/hub",
"port": 4723
},
"command": "appium"
}
]
],
"capabilities": [
{
"appium:deviceName": "pixel_3a",
"appium:avd": "Pixel_3a_API_30",
"appium:platformVersion": "11.0",
"appium:app": "apps/android/my-app.apk",
"platformName": "Android",
"maxInstances": 1,
"appium:automationName": "UiAutomator2",
"appium:allowTestPackages": true,
"appium:newCommandTimeout": 240,
}
]
}- Tag a feature to runwith
@android @run - Run the client:
node wdio main/config/environment.js
Relevant log output
"spec" Reporter:
------------------------------------------------------------------
[emulator-5554 Android 11 #0-0] Running: emulator-5554 on Android 11 executing apps/android/my-app.apk
[emulator-5554 Android 11 #0-0] Session ID: 5724913e-9201-4739-9e5e-bf4ff5023607
[emulator-5554 Android 11 #0-0]
[emulator-5554 Android 11 #0-0] » /main/features/android/zachtest.feature
[emulator-5554 Android 11 #0-0] Android | TEST
[emulator-5554 Android 11 #0-0] Making sure stuff runs
[emulator-5554 Android 11 #0-0] ✖ Given the user is on the login screen
[emulator-5554 Android 11 #0-0] ✖ When the user taps the forgotPasswordLink element
[emulator-5554 Android 11 #0-0] ✖ Then the user is on the forgotPassword screen
[emulator-5554 Android 11 #0-0] ✖ And the following elements are displayed:
[emulator-5554 Android 11 #0-0] │ cancelButton │
[emulator-5554 Android 11 #0-0]
[emulator-5554 Android 11 #0-0] 4 failing (1.3s)
[emulator-5554 Android 11 #0-0]
[emulator-5554 Android 11 #0-0] 1) Making sure stuff runs Given the user is on the login screen
[emulator-5554 Android 11 #0-0] Step "the user is on the login screen" is not defined. You can ignore this error by setting cucumberOpts.ignoreUndefinedDefinitions as true.
[emulator-5554 Android 11 #0-0] Step "the user is on the login screen" is not defined. You can ignore this error by setting cucumberOpts.ignoreUndefinedDefinitions as true.
[emulator-5554 Android 11 #0-0] at Feature(/Users/zachhamm/Documents/GitHub/automation-suite/main/features/android/zachtest.feature):1:1
[emulator-5554 Android 11 #0-0] at Scenario(Making sure stuff runs):35:3
[emulator-5554 Android 11 #0-0] at Step(the user is on the login screen):38:5
[emulator-5554 Android 11 #0-0]
[emulator-5554 Android 11 #0-0]
[emulator-5554 Android 11 #0-0] 2) Making sure stuff runs When the user taps the forgotPasswordLink element
[emulator-5554 Android 11 #0-0] Step "the user taps the forgotPasswordLink element" is not defined. You can ignore this error by setting cucumberOpts.ignoreUndefinedDefinitions as true.
[emulator-5554 Android 11 #0-0] Step "the user taps the forgotPasswordLink element" is not defined. You can ignore this error by setting cucumberOpts.ignoreUndefinedDefinitions as true.
[emulator-5554 Android 11 #0-0] at Feature(/Users/zachhamm/Documents/GitHub/automation-suite/main/features/android/zachtest.feature):1:1
[emulator-5554 Android 11 #0-0] at Scenario(Making sure stuff runs):35:3
[emulator-5554 Android 11 #0-0] at Step(the user taps the forgotPasswordLink element):39:5
[emulator-5554 Android 11 #0-0]
[emulator-5554 Android 11 #0-0]
[emulator-5554 Android 11 #0-0] 3) Making sure stuff runs Then the user is on the forgotPassword screen
[emulator-5554 Android 11 #0-0] Step "the user is on the forgotPassword screen" is not defined. You can ignore this error by setting cucumberOpts.ignoreUndefinedDefinitions as true.
[emulator-5554 Android 11 #0-0] Step "the user is on the forgotPassword screen" is not defined. You can ignore this error by setting cucumberOpts.ignoreUndefinedDefinitions as true.
[emulator-5554 Android 11 #0-0] at Feature(/Users/zachhamm/Documents/GitHub/automation-suite/main/features/android/zachtest.feature):1:1
[emulator-5554 Android 11 #0-0] at Scenario(Making sure stuff runs):35:3
[emulator-5554 Android 11 #0-0] at Step(the user is on the forgotPassword screen):40:5
[emulator-5554 Android 11 #0-0]
[emulator-5554 Android 11 #0-0]
[emulator-5554 Android 11 #0-0] 4) Making sure stuff runs And the following elements are displayed:
[emulator-5554 Android 11 #0-0] Step "the following elements are displayed:" is not defined. You can ignore this error by setting cucumberOpts.ignoreUndefinedDefinitions as true.
[emulator-5554 Android 11 #0-0] Step "the following elements are displayed:" is not defined. You can ignore this error by setting cucumberOpts.ignoreUndefinedDefinitions as true.
[emulator-5554 Android 11 #0-0] at Feature(/Users/zachhamm/Documents/GitHub/automation-suite/main/features/android/zachtest.feature):1:1
[emulator-5554 Android 11 #0-0] at Scenario(Making sure stuff runs):35:3
[emulator-5554 Android 11 #0-0] at Step(the following elements are displayed:):41:5
[emulator-5554 Android 11 #0-0]
Spec Files: 0 passed, 1 failed, 1 total (100% completed) in 00:01:08Code of Conduct
- I agree to follow this project's Code of Conduct
Is there an existing issue for this?
- I have searched the existing issues