Skip to content

JUnit formatter incorrectly reports the total number of tests run #6772

@AlexHowansky

Description

@AlexHowansky

Bug report

Using v1.4.8

When logging via the JUnit formatter, the tests attribute in the testsuite element gets set to the number of failures. It should get set to the total number of tests performed.

Code snippet that reproduces the problem

vendor/bin/phpstan --error-format=junit

This is the log for a run with 50 tests, 1 of which fails:

<?xml version="1.0" encoding="UTF-8"?>
<testsuite failures="1" name="phpstan" tests="1" xmlns:xsi="...">

Expected output

Via this JUnit doc page:

Child elements: <testsuite>
Attributes:
  id: The ID of the scan.
  name: The label of the scan.
  tests: The total number of rules that were applied.
  failures: The total number of rule violations.
  time The time that was required to process all the rules.

Note the number of tests is known to be 50 because of the progress meter:

50/50 [▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓] 100%

The expected output is thus:

<?xml version="1.0" encoding="UTF-8"?>
<testsuite failures="1" name="phpstan" tests="50" xmlns:xsi="...">

The main side effect of this issue is that when all tests pass, the log contains tests="0" and a CI/CD pipeline scanning that log reports that nothing was done. For example, here is BitBucket showing the "Static Analysis" step which should have "50 tests passed":

image

Did PHPStan help you today? Did it make you happy in any way?

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions