Skip to content

[🐛 Bug]: Wdio can hangs generating allure-report when combining afterAll hook and a leak of startStep #13915

@dprevost-LMI

Description

@dprevost-LMI

Have you read the Contributing Guidelines on issues?

WebdriverIO Version

8.39.0

Node.js Version

v18.20.4

Mode

WDIO Testrunner

Which capabilities are you using?

config {
  "runner": "local",
  "waitforTimeout": 5000,
  "connectionRetryTimeout": 10000,
  "connectionRetryCount": 3,
  "framework": "mocha",
  "specs": [
    "../../src/tests/**/*.test.ts"
  ],
  "mochaOpts": {
    "ui": "bdd",
    "timeout": 240000,
    "retries": 0
  },
  "reporters": [
    [
      "allure",
      {
        "outputDir": "logs/allure-results",
        "addConsoleLogs": true
      }
    ],
    [
      "spec",
      {
        "showPreface": false,
        "realtimeReporting": false
      }
    ]
  ],
  "services": [
    [
      "appium",
      {
        "args": {
          "port": 4723,
          "relaxedSecurity": true,
          "log": "./logs/appium.log",
          "logFilters": "./appiumPasswordLogFilters.json"
        }
      }
    ]
  ],
  "capabilities": [
    {
      "platformName": "Android",
      "appium:udid": "MASKED",
      "appium:automationName": "UiAutomator2",
      "appium:appPackage": "com.mypackage",
      "appium:appActivity": "MainActivity",
      "appium:newCommandTimeout": 240,
      "appium:noReset": true,
      "goto:caps": {
        "isLocalRun": true,
        "iosEnterpriseBuild": false,
        "testUserEmail": "",
        "isDevMode": true
      },
      "appium:settings[allowInvisibleElements]": true,
      "appium:locale": "en_US",
      "appium:language": ""
    }
  ]
}

What happened?

When a precise test structure (see code in next section) combines an afterAll hook with an allure startStep that was never ended with endStep, the allure-reporter throws an error on the on:suite:end, and since the WDIOReporter does not seem to handle it gracefully, the reporter hangs indefinitely.

This case happened to us precisely because we used the waitForEnabled method on a selector, which starts steps without finishing them. Switching to waitForExist fixed the startStep leaks, and as a consequence, it stopped throwing on the onEndTestSuite, and therefore, the report stopped hanging.

On my side, I added a patch with a try-catch on this line (on both v8 and v9) with a console log in case we ever get that problem again.

What is your expected behavior?

Even if a reporter throws on the on:suite:end, the WDIOReporter should be able to report the error and stop instead of hanging.

How to reproduce the bug.

To reproduce, create a test file using the information below.
Note: Our real-case scenario is way more complex than the one below, but I was able to simplify it and reach the below.

import { startStep } from '@wdio/allure-reporter';

describe('A', function () {
  context('B', () => {
    before(async () => {
      console.log('Before B step');

      this.afterAll(async () => {
        console.log('After all B step');
      });

      try {
        startStep('Before B step');
        console.log('waitForEnabled-before');
        await $('element-no-enalbed').waitForEnabled();
      } catch (error) {
        console.log('waitForEnabled-error');
        // silence error
      }
    });

    context('C', () => {
      it('D', async () => {
        console.log('Test D step');
        expect(true).toBe(true);
      });
    });
  });
});

To get the same log output as I set below, a try/catch will be required in the following file /node_modules/@wdio/reporter/build/index.js#ln165

        this.on('suite:end', /* istanbul ignore next */ (suite) => {
            const suiteStat = this.suites[suite.uid];
            suiteStat.complete();
            this.currentSuites.pop();
            /* added try-catch below */
            try {
                this.onSuiteEnd(suiteStat);
            } catch(e) {
                console.error("@wdio/reporter: Error in onSuiteEnd, potential stepStart not ended. Happens with waitForEnabled use waitForExists instead", e)
            }
        });

Relevant log output

The log below shows that the test hangs forever without the try/cath.

RUNNING in Android - file:///src/tests/example/template.test.ts
Before B step
waitForEnabled-before
waitForEnabled-error
Test D step
After all B step <==== last log ever output

With the try/catch in /node_modules/@wdio/reporter/build/index.js#ln165

RUNNING in Android - file:///src/tests/example/template.test.ts
Before B step
waitForEnabled-before
waitForEnabled-error
Test D step
After all B step
@wdio/reporter: Error in onSuiteEnd, potential stepStart not ended. Happens with waitForEnabled use waitForExists instead Error: There isn't any active suite!
    at AllureReporter._endSuite (node_modules/@wdio/allure-reporter/build/reporter.js:80:19)
    at AllureReporter.onSuiteEnd (node_modules/@wdio/allure-reporter/build/reporter.js:307:14)
    at AllureReporter.<anonymous> (node_modules/@wdio/reporter/build/index.js:166:22)
    at AllureReporter.emit (node:events:517:28)
    at AllureReporter.emit (node:domain:489:12)
    at node_modules/@wdio/runner/build/reporter.js:47:56
    at Array.forEach (<anonymous>)
    at BaseReporter.emit (node_modules/@wdio/runner/build/reporter.js:47:25)
    at MochaAdapter.emit (node_modules/@wdio/mocha-framework/build/index.js:162:24)
    at Runner.emit (node:events:529:35)
PASSED in Android - file:///src/tests/example/template.test.ts
Report successfully generated to logs/allure-report

Code of Conduct

  • I agree to follow this project's Code of Conduct

Is there an existing issue for this?

  • I have searched the existing issues

Metadata

Metadata

Assignees

No one assigned

    Labels

    Bug 🐛help wantedIssues that are free to take by anyone interested

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions