https://github.com/checkstyle/checkstyle/blob/master/pom.xml#L215-L216
Whenever pitest creates an infinite loop mutation, it takes a whole 50 seconds before it recognizes that the code will never finish. This slows down the pitest run considerably. Pitest can't recognize all instances of an infinite loop, and if the mutated code isn't an infinite loop, but takes a long time to finish, it can cause issues with mutation scoring.
I ran into this problem within my own project. An uncertain mutation is flagged and counted as a killed. However, if it is the edge of finishing before the timeout and does so in some circumstances, it could be flagged as survived and lower the score creating different results in different runs.
https://app.shippable.com/github/rnveach/checkstyle/runs/3/summary/console
Lists all the jobs who have uncertainties as failed.
It is possible to remove the uncertain instances by rewriting the code some.
Here is 1 example where I changed it from uncertain to survived: rnveach@b82ef2d
https://github.com/checkstyle/checkstyle/blob/master/pom.xml#L215-L216
Whenever pitest creates an infinite loop mutation, it takes a whole 50 seconds before it recognizes that the code will never finish. This slows down the pitest run considerably. Pitest can't recognize all instances of an infinite loop, and if the mutated code isn't an infinite loop, but takes a long time to finish, it can cause issues with mutation scoring.
I ran into this problem within my own project. An
uncertainmutation is flagged and counted as a killed. However, if it is the edge of finishing before the timeout and does so in some circumstances, it could be flagged assurvivedand lower the score creating different results in different runs.https://app.shippable.com/github/rnveach/checkstyle/runs/3/summary/console
Lists all the jobs who have uncertainties as
failed.It is possible to remove the uncertain instances by rewriting the code some.
Here is 1 example where I changed it from uncertain to survived: rnveach@b82ef2d