Skip to content

Issue #13095: kill mutation for VariableDeclarationUsageDistanceCheck 5#13121

Merged
romani merged 1 commit intocheckstyle:masterfrom
Kevin222004:M6
Jun 12, 2023
Merged

Issue #13095: kill mutation for VariableDeclarationUsageDistanceCheck 5#13121
romani merged 1 commit intocheckstyle:masterfrom
Kevin222004:M6

Conversation

@Kevin222004
Copy link
Copy Markdown
Contributor

@Kevin222004 Kevin222004 commented Jun 1, 2023

1 Issue #13095: kill mutation for VariableDeclarationUsageDistanceCheck 5


Check:-

https://checkstyle.org/config_coding.html#VariableDeclarationUsageDistance


mutation covered

<mutation unstable="false">
<sourceFile>VariableDeclarationUsageDistanceCheck.java</sourceFile>
<mutatedClass>com.puppycrawl.tools.checkstyle.checks.coding.VariableDeclarationUsageDistanceCheck</mutatedClass>
<mutatedMethod>calculateDistanceBetweenScopes</mutatedMethod>
<mutator>org.pitest.mutationtest.engine.gregor.mutators.experimental.ArgumentPropagationMutator</mutator>
<description>replaced call to java/util/Objects::requireNonNullElse with argument</description>
<lineContent>Objects.requireNonNullElse(exprWithVariableUsage, blockWithVariableUsage);</lineContent>
</mutation>


Explaination

ok so the whole loop will start if


currentScopeAst != null;

then

final Entry<List<DetailAST>, Integer> searchResult =
searchVariableUsageExpressions(variable, currentScopeAst);
currentScopeAst = null;
final List<DetailAST> variableUsageExpressions = searchResult.getKey();

here searchResult will find the first usage of variable
know lets take a example

    public void testMethod9() {
        boolean result = false;
        boolean b1 = true;
        if (b1) {
            result = true;
        }
    }

ast for if block in method

    |       |--LITERAL_IF -> if [19:8]
    |       |   |--LPAREN -> ( [19:11]
    |       |   |--EXPR -> EXPR [19:12]
    |       |   |   `--IDENT -> b1 [19:12]
    |       |   |--RPAREN -> ) [19:14]
    |       |   `--SLIST -> { [19:16]
    |       |       |--EXPR -> EXPR [20:19]
    |       |       |   `--ASSIGN -> = [20:19]
    |       |       |       |--IDENT -> result [20:12]
    |       |       |       `--LITERAL_TRUE -> true [20:21]
    |       |       |--SEMI -> ; [20:25]
    |       |       `--RCURLY -> } [21:8]
    |       `--RCURLY -> } [22:4]

lets suppose we have variable result know what does searchResult give the result as LITERAL_IF -> if [19:8] where it is used.

and variable usage expression will also get same values.

in this example size of list of variableUsageExpressions would be one. so if statment will execute. and over their blockWithVariableUsage = variableUsageExpressions.get(o) means in our example again it would be assigned as LITERAL_IF -> if [19:8]

in switch case

case TokenTypes.LITERAL_IF:
exprWithVariableUsage = getFirstNodeInsideIfBlock(
blockWithVariableUsage, variable);

getFirstNodeInsideIfBlock will execute this method basically provide

/**
* Gets first Ast node inside IF block if variable usage is met
* only inside the block (not in its declaration!).

SLIST -> { [19:16] in all the case if the variable is used in initalization then it will return LITERAL_IF -> if [19:8] this is same for all other case except default one.

and then here lets suppose if variable is used in
initalization then

currentScopeAst = exprWithVariableUsage;
variableUsageAst =
Objects.requireNonNullElse(exprWithVariableUsage, blockWithVariableUsage);

currentScopeAst will become null and variable usage ast become blockWithVariableUsage hence in this case their is no issue.

but lets say if it is not null in the above example. then currentScope would become exprWithVariableUsage
and variableUsageAst become exprWithVariableUsage.

and the loop start again with the value of exprWithVariableUsage

and again searchResult will find the usage but know the scope will start from exprWithVariableUsage and again blockWithVariableUsage will get it value and here variableUsageAst is assgined as blockWithVariableUsage.
know loop will run till currentScopeASt would not become null and if it is null then value of variableUsageAst will become blockWithVariableUsage
so at the end the value will become only blockWithVariableUsage so istead of assigning it to
currentScopeAst it will not make any issue if we assign it as blockWithVariableUsage


Regression :-

https://kevin222004.github.io/reports/M6/2023-06-01-T-14-00-21/test-report/index.html

@Kevin222004
Copy link
Copy Markdown
Contributor Author

GitHub, generate report

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Jun 1, 2023

@Kevin222004
Copy link
Copy Markdown
Contributor Author

@nrmancuso @romani @Vyom-Yadav @rdiachenko ready for review
generating new report started

@Kevin222004
Copy link
Copy Markdown
Contributor Author

@Kevin222004 Kevin222004 force-pushed the M6 branch 2 times, most recently from bf55f6f to 0b67c82 Compare June 1, 2023 20:16
Copy link
Copy Markdown
Member

@rdiachenko rdiachenko left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

Comment on lines +655 to +660
currentScopeAst = exprWithVariableUsage;
variableUsageAst =
Objects.requireNonNullElse(exprWithVariableUsage, blockWithVariableUsage);
if (exprWithVariableUsage == null) {
variableUsageAst = blockWithVariableUsage;
}
else {
variableUsageAst = exprWithVariableUsage;
currentScopeAst = variableUsageAst;
}
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We are not actually killing the mutation. We are just playing smart with pitest so it cannot create a mutation that passes all test cases.

The real problem is assigning variableUsageAst to blockWithVariableUsage in every circumstance i.e. variableUsageAst = blockWithVariableUsage;.


The condition:

 variableUsageAst = Objects.requireNonNullElse(exprWithVariableUsage, blockWithVariableUsage);

translates to:

if (exprWithVariableUsage == null) {          
    variableUsageAst = blockWithVariableUsage;
} else {                                      
    variableUsageAst = exprWithVariableUsage; 
}                                             

Now why this won't pass pitest is due to the mutation exprWithVariableUsage == null -> true which again results in the initial mutation condition i.e. variableUsageAst = blockWithVariableUsage;


Now we take advantage of another fact that currentScopeAst == null in every iteration and it is assigned value only at currentScopeAst = exprWithVariableUsage; Now there is no need to assign a value to currentScopeAst when exprWithVariableUsage is null because we already set currentScopeAst == null.

So using this information to our advantage we put currentScopeAst = exprWithVariableUsage; in the second branch of if-else so that the mutation exprWithVariableUsage == null -> true fails a test case as in that condition currentScopeAst would not be assigned to exprWithVariableUsage.

It is basically:

currentScopeAst = exprWithVariableUsage;        
if (exprWithVariableUsage == null) {            
    variableUsageAst = blockWithVariableUsage;  
} else {                                        
    variableUsageAst = exprWithVariableUsage;   
}                                               

versus

if (exprWithVariableUsage == null) {          
    variableUsageAst = blockWithVariableUsage;
} else {                                      
    // try replacing exprWithVariableUsage with blockWithVariableUsage,
    // still all test cases would pass 
    variableUsageAst = exprWithVariableUsage;
    currentScopeAst = exprWithVariableUsage;  
}                                             

BUT this doesn't change the fact that variableUsageAst = blockWithVariableUsage; can be done in any condition. Hence, the mutation is not actually killed but we just used certain facts to our advantage to kill the mutation.

This would reduce the count but we won't get any actual benefit in code quality by using this method.

@nrmancuso @romani @rdiachenko Thoughts on this way? I would say we should tackle the original mutation variableUsageAst = blockWithVariableUsage; instead of using certain facts to our advantage.

Copy link
Copy Markdown
Member

@romani romani Jun 2, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we do not try to kill survival. Our goal to avoid survival. Highly preferable by reasonable code.
It happens all the time with all quality tools.
Last 10% of items from quality tools are not about quality in most cases but about fact to make code clean from violation and prevent new violations to appear.
We had this problem in jacoco 100% coverage, last 10% of coverage was extremely complicated to cover, but final result is enormous, as we do not even discuss reduction of coverage. Nobody can point to bad code and say "look there are examples of bad testing, I am doing the same".

@Kevin222004 , why we can not simply apply pitest suggestion ?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@romani we can apply Pitest's suggestion I have debugged the whole code with that as well at the end output of the method will not affect it.

I don't want to make any new changes here and this was the possible way to avoid the mutation. I have really not think as #13121 (comment)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of your changes, let's do what pitest suggests.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we can apply Pitest's suggestion I have debugged the whole code with that as well at the end output of the method will not affect it.

@Kevin222004 Another important aspect of this project is explaining why you think it will work. Sometimes there might be gaps in your calculation or sometimes we are thinking of something else and your calculation can correct us.

So please share why you think this won't be a problem in the PR description. This way even if it comes up as a regression later, we can find faults in our thinking if we have something (like a slight explanation as to why it is correct) present.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Vyom-Yadav I have tried to explain such whole thing if you have any question in that let me know

Copy link
Copy Markdown
Member

@romani romani left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok to merge

Copy link
Copy Markdown
Member

@Vyom-Yadav Vyom-Yadav left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Explanation is pending:
#13121 (comment)

@Kevin222004
Copy link
Copy Markdown
Contributor Author

Explanation is pending: [#13121 (comment)](#13121 (comment)

sorry for delay will do it by today

Copy link
Copy Markdown
Member

@Vyom-Yadav Vyom-Yadav left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@Vyom-Yadav Vyom-Yadav assigned romani and unassigned Vyom-Yadav Jun 12, 2023
@romani romani merged commit 85890a5 into checkstyle:master Jun 12, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants