Skip to content

PHPUnit 9.3 compatibility issue #1283

@sanmai

Description

@sanmai
Question Answer
Infection version 0.16.x, master
Test Framework version PHPUnit 9.3
PHP version n/a
Platform n/a

PHPUnit 9.3 comes with a change of terminology and structure, mainly going away from blacklist/whitelist, but not only.

Infection has to change provided phpunit.xml to run each and every mutation. So far I can tell these changes are in the clear.
The problem is in the changes Infection adding to collect coverage during the initial test. Specifically it will add <filter> tag like below, even if there is PHPUnit 9.3 compatible <coverage> directive:

  <coverage>
    <include>
      <directory>path/to/src</directory>
    </include>
  </coverage>
+  <filter>
+    <whitelist>
+      <directory>path/to/src</directory>
+    </whitelist>
+  </filter>

Signs of legacy configuration:

 <phpunit disableCodeCoverageIgnore="...">
 <phpunit ignoreDeprecatedCodeUnitsFromCodeCoverage="...">
 <phpunit cacheTokens="...">
 <filter><whitelist>...
 <logging><log type="...">

Signs of updated configuration:

<coverage>
<logging><*> <!-- where * isn't 'log' -->

Possible solution

There are several consideration:

  • One way to tackle this issue is to check for <coverage> element before adding <filter> element. If it so happens that this phpunit.xml is being used with earlier version of PHPUnit that isn't aware of this new element, then this will be caught during validation against xsd.
  • Additionally we have to deal with the case where there isn't <coverage> or <filter> elements. In this case we can check if vendor/phpunit/phpunit/phpunit.xsd mentions <coverage> element, and then decide to add either, but to be honest I'd rather see Infection fail with an explanation.
  • Since Infection does not depend on PHPUnit, we are not guaranteed to find phpunit.xsd. This does not say that we cannot have a negative dependency, that is a future version of Infection can be in conflict with earlier versions of PHPUnit by standard means offered by composer.json.

Alternative solution

As for failing with a message, Infection could check if there are <coverage> or <filter> elements, and if not, fail with an explanation instead of adding such elements.

Workaround

Current workaround is to run PHPUnit to collect coverage before running Infection like so:

vendor/bin/phpunit --coverage-xml=build/logs/coverage-xml --log-junit=build/logs/junit.xml
vendor/bin/infection --skip-initial-tests --threads=$(nproc) --coverage=build/logs --show-mutations --no-interaction 

I'd like to think this is the right way to run Infection during CI, but there might be other opinions.

Another workaround

If you do not plan to use PHPUnit older than 9.3, there's another workaround. One problem here is that Infection tries to validate updated phpunit.xml, but it uses newest schema where PHPUnit will be happy to use older one. When we instruct Infection to use older schema, it just works. Just like so:

-    xsi:noNamespaceSchemaLocation="vendor/phpunit/phpunit/phpunit.xsd"
+    xsi:noNamespaceSchemaLocation="vendor/phpunit/phpunit/schema/9.2.xsd"

This workaround requires the least amount of effort if you don't mind being nagged by PHPUnit about "--migrate-configuration" now and then.

If you do plan to use PHPUnit 9.2 or earlier together with 9.3, then you would probably need to inject something like this somewhere in your CI pipeline:

test -f vendor/phpunit/phpunit/schema/9.2.xsd || 
    mkdir -p vendor/phpunit/phpunit/schema && 
    cp vendor/phpunit/phpunit/phpunit.xsd vendor/phpunit/phpunit/schema/9.2.xsd

Thanks to @terabytesoftw for pointing at these changes.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions