Skip to content

--list-tests and --list-tests-xml CLI options do not report error when data provider method throws exception #5908

@harentius

Description

@harentius
Q A
PHPUnit version 11.2.8 10.5.28 9.6.20
PHP version 8.2.7
Installation Method Composer

Summary

list-tests (list-tests-xml) command doesn't report any error if dataProvider throws an exception and just return smaller list of tests
As this command(s) often used for CI environment and parallel runs, skipping this exception leads to false positive CI result.

Current behavior

How to reproduce

  1. The test (based on the documentation example):
<?php declare(strict_types=1);
use PHPUnit\Framework\Attributes\DataProvider;
use PHPUnit\Framework\TestCase;

final class NumericDataSetsTest extends TestCase
{
    public static function additionProvider(): array
    {
       throw new \Exception();
       return [
            [0, 0, 0],
            [0, 1, 1],
        ];
    }

    #[DataProvider('additionProvider')]
    public function testAdd(int $a, int $b, int $expected): void
    {
        $this->assertSame($expected, $a + $b);
    }

    public function testAnother(): void
    {
        $this->assertSame(0, 0);
    }
}
  1. Run
    vendor/bin/phpunit tests --list-tests

Expected behavior

I would expect displaying an error and do not return 0 code.

Metadata

Metadata

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions