Skip to content

Cache PHPStan result cache for re-use while mutation testing#721

Merged
romm merged 22 commits intoCuyZ:masterfrom
staabm:cache
Oct 10, 2025
Merged

Cache PHPStan result cache for re-use while mutation testing#721
romm merged 22 commits intoCuyZ:masterfrom
staabm:cache

Conversation

@staabm
Copy link
Copy Markdown
Contributor

@staabm staabm commented Oct 8, 2025

Reduces PHPStan overhead while mutation testing

@staabm staabm marked this pull request as ready for review October 8, 2025 14:29
@staabm
Copy link
Copy Markdown
Contributor Author

staabm commented Oct 8, 2025

I have a feeling the build currently cannot download the cache artifact, because the workflow was never run on the default branch.

I guess it will work after the PR beeing merged and a subsequent PR opened (will test after merge)

- name: Running PHPStan (global)
run: php vendor/bin/phpstan

- name: Cache PHPStan result cache for re-use while mutation testing
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The var/cache folder seems to be cached already, see line 26, so do we really need that, or moving the MT to this workflow would be enough?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hmm good catch.

I guess what you say is what we need from this PR is the phpstan.neon.dist change only and everything else can be reverted again..?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry for the late answer!

Yeah I think we can do that, mutation already has its own cache so the change in phpstan.neon.dist is probably enough. 😊

Do you know if we have a simple way to check it that works?

Copy link
Copy Markdown
Contributor Author

@staabm staabm Oct 9, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you can invoke infection with --log-verbosity=all --debug and look into the infection.log so you see the underlying phpstan output and command

like in https://github.com/phpstan/phpstan-doctrine/pull/686/files#diff-5c3fa597431eda03ac3339ae6bf7f05e1a50d6fc7333679ec38e21b337cb6721R189

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got the following from the artifact:

1) /home/runner/work/Valinor/Valinor/src/MapperBuilder.php:29    [M] IncrementInteger [ID] 14540bd378da9d0b3e054b47f7cb77b1

@@ @@
     public function __construct()
     {
         $this->settings = new Settings();
-        $x = 1;
+        $x = 2;
         // test change
     }
     /**

$ '/home/runner/work/Valinor/Valinor/vendor/bin/phpstan' '--tmp-file=/home/runner/work/Valinor/Valinor/var/cache/infection/infection/mutant.14540bd378da9d0b3e054b47f7cb77b1.infection.php' '--instead-of=/home/runner/work/Valinor/Valinor/src/MapperBuilder.php' '--configuration=/home/runner/work/Valinor/Valinor/var/cache/infection/infection/phpstan.14540bd378da9d0b3e054b47f7cb77b1.infection.neon' '--error-format=json' '--no-progress' '-vv' '--fail-without-result-cache'
  {"totals":{"errors":0,"file_errors":0},"files":{},"errors":[]}
  
  Result cache restored. 1 file will be reanalysed.
  Result cache was not saved because of --tmp-file and --instead-of CLI options passed (editor mode).
  Elapsed time: 3 seconds
  Used memory: 102.84 MB

Looks good to you?

--min-msi=100 \
--show-mutations \
--verbose \
--logger-text=php://stdout \
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@romm I've noticed you removed --log-verbosity=all - is it intentional? By default (without this option), killed mutants are not printed

https://infection.github.io/guide/how-to.html#How-to-debug-Infection

The verbosity of the log file, all - this mode will add “Killed mutants” into log file and add additional information

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indeed. Is there some way we don't show everything (for instance "error" mutants)? Because the list can get huge quite fast.

Copy link
Copy Markdown

@maks-rafalko maks-rafalko Oct 12, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

unfortunately, no. But I feel like we can introduce some update to Infection to support it, like instead of --log-verbosity=all we can use +x notation:

  • --log-verbosity=+killed,+not-covered

meaning that only killed and not covered mutants will be added to the log outputs. (It's just an idea, option and implementation can be different)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Filed infection/infection#2445. If you have other ideas, please share

@romm
Copy link
Copy Markdown
Member

romm commented Oct 10, 2025

Alright I think we're good to go, so to summarize:

  • Use cache directory
  • Update Infection's version
  • Use --logger-text=php://stdout and --debug options to have more information in the output

@romm romm merged commit aa6feb7 into CuyZ:master Oct 10, 2025
16 checks passed
@romm
Copy link
Copy Markdown
Member

romm commented Oct 10, 2025

Thank you @staabm!

@maks-rafalko
Copy link
Copy Markdown

@romm you probably missed my question #721 (comment) :) could you please take a look?

@staabm staabm deleted the cache branch October 10, 2025 12:08
if: github.event_name == 'pull_request' && always()
with:
name: "infection-log"
path: "var/infection/infection.log"
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hm.. but it's not generated on CI because it's overridden by --logger-text=php://stdout

or did I miss something?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤦

I'm actually at a conference center and it's probably not the best idea to merge a PR there while talking with people haha

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants