Issue #13095: kill mutation for VariableDeclarationUsageDistanceCheck 5#13121
Issue #13095: kill mutation for VariableDeclarationUsageDistanceCheck 5#13121romani merged 1 commit intocheckstyle:masterfrom
Conversation
|
GitHub, generate report |
|
@nrmancuso @romani @Vyom-Yadav @rdiachenko ready for review |
|
Regression after code changes :- https://kevin222004.github.io/reports/M6/2023-06-01-T-14-00-21/test-report/index.html |
...ava/com/puppycrawl/tools/checkstyle/checks/coding/VariableDeclarationUsageDistanceCheck.java
Outdated
Show resolved
Hide resolved
bf55f6f to
0b67c82
Compare
| currentScopeAst = exprWithVariableUsage; | ||
| variableUsageAst = | ||
| Objects.requireNonNullElse(exprWithVariableUsage, blockWithVariableUsage); | ||
| if (exprWithVariableUsage == null) { | ||
| variableUsageAst = blockWithVariableUsage; | ||
| } | ||
| else { | ||
| variableUsageAst = exprWithVariableUsage; | ||
| currentScopeAst = variableUsageAst; | ||
| } |
There was a problem hiding this comment.
We are not actually killing the mutation. We are just playing smart with pitest so it cannot create a mutation that passes all test cases.
The real problem is assigning variableUsageAst to blockWithVariableUsage in every circumstance i.e. variableUsageAst = blockWithVariableUsage;.
The condition:
variableUsageAst = Objects.requireNonNullElse(exprWithVariableUsage, blockWithVariableUsage);translates to:
if (exprWithVariableUsage == null) {
variableUsageAst = blockWithVariableUsage;
} else {
variableUsageAst = exprWithVariableUsage;
} Now why this won't pass pitest is due to the mutation exprWithVariableUsage == null -> true which again results in the initial mutation condition i.e. variableUsageAst = blockWithVariableUsage;
Now we take advantage of another fact that currentScopeAst == null in every iteration and it is assigned value only at currentScopeAst = exprWithVariableUsage; Now there is no need to assign a value to currentScopeAst when exprWithVariableUsage is null because we already set currentScopeAst == null.
So using this information to our advantage we put currentScopeAst = exprWithVariableUsage; in the second branch of if-else so that the mutation exprWithVariableUsage == null -> true fails a test case as in that condition currentScopeAst would not be assigned to exprWithVariableUsage.
It is basically:
currentScopeAst = exprWithVariableUsage;
if (exprWithVariableUsage == null) {
variableUsageAst = blockWithVariableUsage;
} else {
variableUsageAst = exprWithVariableUsage;
} versus
if (exprWithVariableUsage == null) {
variableUsageAst = blockWithVariableUsage;
} else {
// try replacing exprWithVariableUsage with blockWithVariableUsage,
// still all test cases would pass
variableUsageAst = exprWithVariableUsage;
currentScopeAst = exprWithVariableUsage;
} BUT this doesn't change the fact that variableUsageAst = blockWithVariableUsage; can be done in any condition. Hence, the mutation is not actually killed but we just used certain facts to our advantage to kill the mutation.
This would reduce the count but we won't get any actual benefit in code quality by using this method.
@nrmancuso @romani @rdiachenko Thoughts on this way? I would say we should tackle the original mutation variableUsageAst = blockWithVariableUsage; instead of using certain facts to our advantage.
There was a problem hiding this comment.
we do not try to kill survival. Our goal to avoid survival. Highly preferable by reasonable code.
It happens all the time with all quality tools.
Last 10% of items from quality tools are not about quality in most cases but about fact to make code clean from violation and prevent new violations to appear.
We had this problem in jacoco 100% coverage, last 10% of coverage was extremely complicated to cover, but final result is enormous, as we do not even discuss reduction of coverage. Nobody can point to bad code and say "look there are examples of bad testing, I am doing the same".
@Kevin222004 , why we can not simply apply pitest suggestion ?
There was a problem hiding this comment.
@romani we can apply Pitest's suggestion I have debugged the whole code with that as well at the end output of the method will not affect it.
I don't want to make any new changes here and this was the possible way to avoid the mutation. I have really not think as #13121 (comment)
There was a problem hiding this comment.
Instead of your changes, let's do what pitest suggests.
There was a problem hiding this comment.
we can apply Pitest's suggestion I have debugged the whole code with that as well at the end output of the method will not affect it.
@Kevin222004 Another important aspect of this project is explaining why you think it will work. Sometimes there might be gaps in your calculation or sometimes we are thinking of something else and your calculation can correct us.
So please share why you think this won't be a problem in the PR description. This way even if it comes up as a regression later, we can find faults in our thinking if we have something (like a slight explanation as to why it is correct) present.
There was a problem hiding this comment.
@Vyom-Yadav I have tried to explain such whole thing if you have any question in that let me know
Vyom-Yadav
left a comment
There was a problem hiding this comment.
Explanation is pending:
#13121 (comment)
sorry for delay will do it by today |
1 Issue #13095: kill mutation for VariableDeclarationUsageDistanceCheck 5
Check:-
https://checkstyle.org/config_coding.html#VariableDeclarationUsageDistance
mutation covered
checkstyle/config/pitest-suppressions/pitest-coding-1-suppressions.xml
Lines 111 to 118 in 6f451fb
Explaination
ok so the whole loop will start if
checkstyle/src/main/java/com/puppycrawl/tools/checkstyle/checks/coding/VariableDeclarationUsageDistanceCheck.java
Line 613 in c5566a6
currentScopeAst != null;
then
checkstyle/src/main/java/com/puppycrawl/tools/checkstyle/checks/coding/VariableDeclarationUsageDistanceCheck.java
Lines 614 to 619 in c5566a6
here
searchResultwill find the first usage of variableknow lets take a example
ast for if block in method
lets suppose we have variable
resultknow what doessearchResultgive the result asLITERAL_IF -> if [19:8]where it is used.and variable usage expression will also get same values.
in this example size of list of variableUsageExpressions would be one. so if statment will execute. and over their
blockWithVariableUsage = variableUsageExpressions.get(o)means in our example again it would be assigned asLITERAL_IF -> if [19:8]in switch case
checkstyle/src/main/java/com/puppycrawl/tools/checkstyle/checks/coding/VariableDeclarationUsageDistanceCheck.java
Lines 639 to 641 in c5566a6
getFirstNodeInsideIfBlockwill execute this method basically providecheckstyle/src/main/java/com/puppycrawl/tools/checkstyle/checks/coding/VariableDeclarationUsageDistanceCheck.java
Lines 743 to 745 in c5566a6
SLIST -> { [19:16]in all the case if the variable is used in initalization then it will returnLITERAL_IF -> if [19:8]this is same for all other case except default one.and then here lets suppose if variable is used in
initalization then
checkstyle/src/main/java/com/puppycrawl/tools/checkstyle/checks/coding/VariableDeclarationUsageDistanceCheck.java
Lines 655 to 657 in c5566a6
currentScopeAst will become null and variable usage ast become blockWithVariableUsage hence in this case their is no issue.
but lets say if it is not null in the above example. then currentScope would become exprWithVariableUsage
and variableUsageAst become exprWithVariableUsage.
and the loop start again with the value of exprWithVariableUsage
and again
searchResultwill find the usage but know the scope will start from exprWithVariableUsage and againblockWithVariableUsagewill get it value and herevariableUsageAstis assgined asblockWithVariableUsage.know loop will run till currentScopeASt would not become null and if it is null then value of variableUsageAst will become
blockWithVariableUsageso at the end the value will become only
blockWithVariableUsageso istead of assigning it tocurrentScopeAstit will not make any issue if we assign it as blockWithVariableUsageRegression :-
https://kevin222004.github.io/reports/M6/2023-06-01-T-14-00-21/test-report/index.html