| Question |
Answer |
| Infection version |
0.16.x, master |
| Test Framework version |
PHPUnit 9.3 |
| PHP version |
n/a |
| Platform |
n/a |
PHPUnit 9.3 comes with a change of terminology and structure, mainly going away from blacklist/whitelist, but not only.
Infection has to change provided phpunit.xml to run each and every mutation. So far I can tell these changes are in the clear.
The problem is in the changes Infection adding to collect coverage during the initial test. Specifically it will add <filter> tag like below, even if there is PHPUnit 9.3 compatible <coverage> directive:
<coverage>
<include>
<directory>path/to/src</directory>
</include>
</coverage>
+ <filter>
+ <whitelist>
+ <directory>path/to/src</directory>
+ </whitelist>
+ </filter>
Signs of legacy configuration:
<phpunit disableCodeCoverageIgnore="...">
<phpunit ignoreDeprecatedCodeUnitsFromCodeCoverage="...">
<phpunit cacheTokens="...">
<filter><whitelist>...
<logging><log type="...">
Signs of updated configuration:
<coverage>
<logging><*> <!-- where * isn't 'log' -->
Possible solution
There are several consideration:
- One way to tackle this issue is to check for
<coverage> element before adding <filter> element. If it so happens that this phpunit.xml is being used with earlier version of PHPUnit that isn't aware of this new element, then this will be caught during validation against xsd.
- Additionally we have to deal with the case where there isn't
<coverage> or <filter> elements. In this case we can check if vendor/phpunit/phpunit/phpunit.xsd mentions <coverage> element, and then decide to add either, but to be honest I'd rather see Infection fail with an explanation.
- Since Infection does not depend on PHPUnit, we are not guaranteed to find
phpunit.xsd. This does not say that we cannot have a negative dependency, that is a future version of Infection can be in conflict with earlier versions of PHPUnit by standard means offered by composer.json.
Alternative solution
As for failing with a message, Infection could check if there are <coverage> or <filter> elements, and if not, fail with an explanation instead of adding such elements.
Workaround
Current workaround is to run PHPUnit to collect coverage before running Infection like so:
vendor/bin/phpunit --coverage-xml=build/logs/coverage-xml --log-junit=build/logs/junit.xml
vendor/bin/infection --skip-initial-tests --threads=$(nproc) --coverage=build/logs --show-mutations --no-interaction
I'd like to think this is the right way to run Infection during CI, but there might be other opinions.
Another workaround
If you do not plan to use PHPUnit older than 9.3, there's another workaround. One problem here is that Infection tries to validate updated phpunit.xml, but it uses newest schema where PHPUnit will be happy to use older one. When we instruct Infection to use older schema, it just works. Just like so:
- xsi:noNamespaceSchemaLocation="vendor/phpunit/phpunit/phpunit.xsd"
+ xsi:noNamespaceSchemaLocation="vendor/phpunit/phpunit/schema/9.2.xsd"
This workaround requires the least amount of effort if you don't mind being nagged by PHPUnit about "--migrate-configuration" now and then.
If you do plan to use PHPUnit 9.2 or earlier together with 9.3, then you would probably need to inject something like this somewhere in your CI pipeline:
test -f vendor/phpunit/phpunit/schema/9.2.xsd ||
mkdir -p vendor/phpunit/phpunit/schema &&
cp vendor/phpunit/phpunit/phpunit.xsd vendor/phpunit/phpunit/schema/9.2.xsd
Thanks to @terabytesoftw for pointing at these changes.
PHPUnit 9.3 comes with a change of terminology and structure, mainly going away from blacklist/whitelist, but not only.
Infection has to change provided
phpunit.xmlto run each and every mutation. So far I can tell these changes are in the clear.The problem is in the changes Infection adding to collect coverage during the initial test. Specifically it will add
<filter>tag like below, even if there is PHPUnit 9.3 compatible<coverage>directive:<coverage> <include> <directory>path/to/src</directory> </include> </coverage> + <filter> + <whitelist> + <directory>path/to/src</directory> + </whitelist> + </filter>Signs of legacy configuration:
Signs of updated configuration:
Possible solution
There are several consideration:
<coverage>element before adding<filter>element. If it so happens that thisphpunit.xmlis being used with earlier version of PHPUnit that isn't aware of this new element, then this will be caught during validation against xsd.<coverage>or<filter>elements. In this case we can check ifvendor/phpunit/phpunit/phpunit.xsdmentions<coverage>element, and then decide to add either, but to be honest I'd rather see Infection fail with an explanation.phpunit.xsd. This does not say that we cannot have a negative dependency, that is a future version of Infection can be in conflict with earlier versions of PHPUnit by standard means offered bycomposer.json.Alternative solution
As for failing with a message, Infection could check if there are
<coverage>or<filter>elements, and if not, fail with an explanation instead of adding such elements.Workaround
Current workaround is to run PHPUnit to collect coverage before running Infection like so:
vendor/bin/phpunit --coverage-xml=build/logs/coverage-xml --log-junit=build/logs/junit.xml vendor/bin/infection --skip-initial-tests --threads=$(nproc) --coverage=build/logs --show-mutations --no-interactionI'd like to think this is the right way to run Infection during CI, but there might be other opinions.
Another workaround
If you do not plan to use PHPUnit older than 9.3, there's another workaround. One problem here is that Infection tries to validate updated
phpunit.xml, but it uses newest schema where PHPUnit will be happy to use older one. When we instruct Infection to use older schema, it just works. Just like so:This workaround requires the least amount of effort if you don't mind being nagged by PHPUnit about "--migrate-configuration" now and then.
If you do plan to use PHPUnit 9.2 or earlier together with 9.3, then you would probably need to inject something like this somewhere in your CI pipeline:
Thanks to @terabytesoftw for pointing at these changes.