This issue will be the main tracker issue for resolving Pitest suppressions.

Summary

Surviving pit mutations are a form of technical debt, and we would like to kill all surviving mutations. It is important to first understand what pitest is and how it works. Please visit https://pitest.org/ to learn about this tool. Simply put, pitest changes our code using mutation operators (this changed code is now called a mutation), then runs our tests against this changed code. If some test fails due to the changed code, then the mutation is considered "killed". If not, then the mutation has survived, and we need to (ideally) find a test input to kill the mutation.

The point of this issue is to find a test case that kills a surviving mutation.

How does Pitest work?

Let's take a look at a suppressed surviving mutation:

The above suppression is for a mutation operator that took this original line of code:

And mutated it into this code (now called a mutant or mutation):

lcurly = lcurly;

The mutation is considered to have "survived", which means that no test failed after the code was changed. We want to find a test case that will fail if this mutation is in the code.

How do I select a mutation to work on?

Visit https://github.com/checkstyle/checkstyle/tree/master/.ci/pitest-suppressions, take a look through any of the suppression files there, and pick one mutation to work on.

How do I resolve a pitest suppression?

1. Pick a single mutation suppression from a file in https://github.com/checkstyle/checkstyle/tree/master/.ci/pitest-suppressions
2. Create a new issue, with a permalink to the suppression.
3. Send PR with single hardcoded mutation only (ex: change lcurly = child.getFirstChild(); to lcurly =lcurly;)
4. If any tests or continuous integration testing (ex: no-exception testing) are failing, you might have found a test case to kill the mutation.
5. If a test case is not found in step (4), then generate a check regression report. See https://github.com/checkstyle/contribution/tree/master/checkstyle-tester#report-generation for details. Any differences or exceptions in the report are probably a new test case.
6. If a test case still cannot be found, provide an analysis of why we cannot find a test case to kill this mutation, and share it in the issue you've created for discussion with maintainers.

Once a possible test case is found, create a new test and new test input for it, generate the pitest report locally to verify. See https://github.com/checkstyle/checkstyle/wiki/How-to-Generate-and-Analyze-Pitest-Reports#how-to-generate-pit-report for details on this.

The procedure for resolution of pitest suppressions can be found in the below diagram:

Required reading:
https://github.com/checkstyle/checkstyle/wiki/How-to-Generate-and-Analyze-Pitest-Reports

Example pull requests:

https://github.com/checkstyle/checkstyle/pulls?q=is%3Apr+is%3Amerged+%22resolve+pitest+suppression%22+

https://github.com/checkstyle/checkstyle/pulls?q=is%3Apr+is%3Amerged+%22kill+surviving+mutation%22