Issue #13321: Kill mutation for SummaryJavaDocCheck-1 by Kevin222004 · Pull Request #13311 · checkstyle/checkstyle

Kevin222004 · 2023-06-26T17:51:39Z

Issue #13321: Kill mutation for SummaryJavaDocCheck-1

Check :-

https://checkstyle.org/checks/javadoc/summaryjavadoc.html#SummaryJavadoc

Mutation

checkstyle/config/pitest-suppressions/pitest-javadoc-suppressions.xml

Lines 858 to 865 in 3d8ed42

    
           <mutation unstable="false"> 
        
             <sourceFile>SummaryJavadocCheck.java</sourceFile> 
        
             <mutatedClass>com.puppycrawl.tools.checkstyle.checks.javadoc.SummaryJavadocCheck</mutatedClass> 
        
             <mutatedMethod>containsForbiddenFragment</mutatedMethod> 
        
             <mutator>org.pitest.mutationtest.engine.gregor.mutators.experimental.NakedReceiverMutator</mutator> 
        
             <description>replaced call to java/lang/String::trim with receiver</description> 
        
             <lineContent>.matcher(firstSentence).replaceAll(&quot; &quot;).trim();</lineContent> 
        
           </mutation>

Explaination

The removal of trim will effect the programme it will increase the number of times in loop.

There are total 3 usage of the containsForbiddenFragment

In validateUntaggedSummary method at

checkstyle/src/main/java/com/puppycrawl/tools/checkstyle/checks/javadoc/SummaryJavadocCheck.java

Lines 343 to 345 in 6073cd4

    
           if (endOfSentence != -1 
        
                   && containsForbiddenFragment(firstSentence.substring(0, endOfSentence))) { 
        
               log(ast.getLineNumber(), MSG_SUMMARY_JAVADOC);

Note:- If this condition execute then violation thrown and violation message doesn't contains any relation with the text.

Know when containsForbiddenFragment will be execute then

checkstyle/src/main/java/com/puppycrawl/tools/checkstyle/checks/javadoc/SummaryJavadocCheck.java

Lines 603 to 606 in 6073cd4

    
           private boolean containsForbiddenFragment(String firstSentence) { 
        
               final String javadocText = JAVADOC_MULTILINE_TO_SINGLELINE_PATTERN 
        
                       .matcher(firstSentence).replaceAll(" ").trim(); 
        
               return forbiddenSummaryFragments.matcher(trimExcessWhitespaces(javadocText)).find();

javadocText will not be trimmed lets suppose if we have sentence This is summary javadoc .
then javaDoctext would become This is summary javadoc

and as a return statment we call trimExcessWhitespaces to match forbiddenSummaryFragment and in trimExcessWhitespaces for loop will run as more time as we have extra spaces which is not been removed as we remove trim.
and the string will return with the extra space but it will not affect the output.

And for the second and third usage the explaintaion is same as above

checkstyle/src/main/java/com/puppycrawl/tools/checkstyle/checks/javadoc/SummaryJavadocCheck.java

Lines 528 to 530 in 6073cd4

    
           else if (containsForbiddenFragment(inlineReturn)) { 
        
               log(inlineReturnTag.getLineNumber(), MSG_SUMMARY_JAVADOC); 
        
           }

and

checkstyle/src/main/java/com/puppycrawl/tools/checkstyle/checks/javadoc/SummaryJavadocCheck.java

Lines 511 to 513 in 6073cd4

    
           else if (containsForbiddenFragment(inlineSummary)) { 
        
               log(inlineSummaryTag.getLineNumber(), MSG_SUMMARY_JAVADOC); 
        
           }

Regression :-

Report-1 :- https://checkstyle-diff-reports.s3.us-east-2.amazonaws.com/5d08f12_2023202818/reports/diff/index.html

Report-2:- https://checkstyle-diff-reports.s3.us-east-2.amazonaws.com/5d08f12_2023084141/reports/diff/index.html

Diff Regression config: https://gist.githubusercontent.com/Kevin222004/61eaa69fe98585b6ee1882ca082362bc/raw/c74195e574d09886a3406be17a9d46787da9545e/Summary.xml
Diff Regression projects: https://gist.githubusercontent.com/Kevin222004/9600f179b602d4c971bdb0a050099005/raw/360a95ed7bb60d7a0956e531199d484c4d6f6617/test-projects.properties
Report label: Regression-2

Kevin222004 · 2023-06-26T17:51:56Z

Github, generate report

github-actions · 2023-06-26T20:29:36Z

Regression-1:
https://checkstyle-diff-reports.s3.us-east-2.amazonaws.com/5d08f12_2023202818/reports/diff/index.html

Kevin222004 · 2023-06-27T07:18:34Z

Github, generate report

github-actions · 2023-06-27T08:42:31Z

Regression-2:
https://checkstyle-diff-reports.s3.us-east-2.amazonaws.com/5d08f12_2023084141/reports/diff/index.html

romani

Ok, looks like extra trim

rdiachenko

lgtm

Vyom-Yadav · 2023-07-02T07:34:41Z

@Kevin222004 I have found a regression test case with the current changes:

With the original code:

@Test                                                                                         
public void testAsd() throws Exception {                                                      
    final String[] expected = {                                                               
            "16: " + getCheckMessage(MSG_SUMMARY_JAVADOC),                                    
    };                                                                                        
                                                                                              
    verifyWithInlineConfigParser(                                                             
            getPath("InputSummaryJavadocTestForbiddenFragmentsFoo.java"), expected);          
}

Input file:

/*
SummaryJavadoc
violateExecutionOnNonTightHtml = true
forbiddenSummaryFragments = ^Returns the customer ID. This method returns.$
period = .

*/

package com.puppycrawl.tools.checkstyle.checks.javadoc.summaryjavadoc;

// ok
public class InputSummaryJavadocTestForbiddenFragmentsFoo {

    // violation 2 lines below
    /**
     * {@summary Returns the customer ID.
     *  This method returns. }
     */
    public void foo() {
    }
}

With your changes:

Violation lines for /home/vyom/IdeaProjects/checkstyle/src/test/resources/com/puppycrawl/tools/checkstyle/checks/javadoc/summaryjavadoc/InputSummaryJavadocTestForbiddenFragmentsFoo.java differ.
missing (1): 16
---
expected   : [16]
but was    : []

This is both a good and bad regression.
The above-mentioned is the bad part, here is the good part: (^Returns the customer ID. This method returns. $, notice the whitespace after . and before $)

/*
SummaryJavadoc
violateExecutionOnNonTightHtml = true
forbiddenSummaryFragments = ^Returns the customer ID. This method returns. $
period = .

*/

package com.puppycrawl.tools.checkstyle.checks.javadoc.summaryjavadoc;

// ok
public class InputSummaryJavadocTestForbiddenFragmentsFoo {

    // ok, no violation here with the original code, IMO there should be a violation.
    /**
     * {@summary Returns the customer ID.
     *  This method returns. }
     */
    public void foo() {
    }
}

with your changes:

/*
SummaryJavadoc
violateExecutionOnNonTightHtml = true
forbiddenSummaryFragments = ^Returns the customer ID. This method returns. $
period = .

*/

package com.puppycrawl.tools.checkstyle.checks.javadoc.summaryjavadoc;

// ok
public class InputSummaryJavadocTestForbiddenFragmentsFoo {

    // violation 2 lines below
    /**
     * {@summary Returns the customer ID.
     *  This method returns. }
     */
    public void foo() {
    }
}

@romani @rdiachenko Please share your opinion about the above mentioned test case (the one with extra space.)

Vyom-Yadav

Items:
Main item: #13311 (comment)

config/pitest-suppressions/pitest-javadoc-suppressions.xml

romani · 2023-07-02T21:01:00Z

@Kevin222004 , please add Inputs from @Vyom-Yadav to our code base.

Please share your opinion about the above mentioned test case (the one with extra space.)

It should be a violation. As with inline tags we consider all until }. Period property is not in use.

Kevin222004 · 2023-07-02T21:30:40Z

@romani done test added

romani

Items:

...rces/com/puppycrawl/tools/checkstyle/checks/javadoc/summaryjavadoc/InputSummaryJavadoc1.java

romani

ok to merge

Vyom-Yadav

LGTM! @Kevin222004 Please create an issue for the other case I mentioned in the PR. @romani CircleCI failed due to a connection reset. Please re-run.

romani · 2023-07-04T06:48:54Z

@Kevin222004 , please review what is not done

romani · 2023-07-04T12:46:57Z

@Vyom-Yadav , I think you mentioned only one case and we opened issue for it #13347

Kevin222004 · 2023-07-04T12:49:56Z

Yes I messaged him for same but not get replied yet

Vyom-Yadav · 2023-07-04T12:54:16Z

#13347 (comment)

Kevin222004 changed the title ~~Issue #13109: Kill mutation for SummaryJavaDocCheck-1~~ Issue #13321: Kill mutation for SummaryJavaDocCheck-1 Jun 27, 2023

Kevin222004 force-pushed the s1 branch from 5d08f12 to e3f326f Compare June 27, 2023 07:36

Kevin222004 force-pushed the s1 branch from 46cc4bd to 425a936 Compare June 27, 2023 13:07

Kevin222004 marked this pull request as ready for review June 27, 2023 13:07

romani mentioned this pull request Jun 27, 2023

Resolve pitest survivals in modules - 2 #13321

Closed

41 tasks

romani approved these changes Jun 27, 2023

View reviewed changes

romani requested review from Vyom-Yadav and rdiachenko June 27, 2023 17:35

romani assigned rdiachenko Jun 27, 2023

rdiachenko approved these changes Jun 28, 2023

View reviewed changes

rdiachenko assigned Vyom-Yadav and unassigned rdiachenko Jun 28, 2023

Vyom-Yadav reviewed Jul 2, 2023

View reviewed changes

config/pitest-suppressions/pitest-javadoc-suppressions.xml Show resolved Hide resolved

Kevin222004 force-pushed the s1 branch from 425a936 to 8602026 Compare July 2, 2023 21:30

romani requested changes Jul 2, 2023

View reviewed changes

...rces/com/puppycrawl/tools/checkstyle/checks/javadoc/summaryjavadoc/InputSummaryJavadoc1.java Outdated Show resolved Hide resolved

Issue checkstyle#13321: Kill mutation for SummaryJavaDocCheck-1

744abe6

Kevin222004 force-pushed the s1 branch from 8602026 to 744abe6 Compare July 3, 2023 15:10

Kevin222004 mentioned this pull request Jul 3, 2023

False Positive and Negative in SummaryJavaDocCheck #13347

Closed

romani approved these changes Jul 3, 2023

View reviewed changes

Vyom-Yadav approved these changes Jul 4, 2023

View reviewed changes

Vyom-Yadav assigned romani and unassigned Vyom-Yadav Jul 4, 2023

romani merged commit fa39b3e into checkstyle:master Jul 4, 2023

Kevin222004 mentioned this pull request Aug 2, 2023

storage Kevin222004/GitHub-action#3

Closed

	<mutation unstable="false">
	<sourceFile>SummaryJavadocCheck.java</sourceFile>
	<mutatedClass>com.puppycrawl.tools.checkstyle.checks.javadoc.SummaryJavadocCheck</mutatedClass>
	<mutatedMethod>containsForbiddenFragment</mutatedMethod>
	<mutator>org.pitest.mutationtest.engine.gregor.mutators.experimental.NakedReceiverMutator</mutator>
	<description>replaced call to java/lang/String::trim with receiver</description>
	<lineContent>.matcher(firstSentence).replaceAll(" ").trim();</lineContent>
	</mutation>

	if (endOfSentence != -1
	&& containsForbiddenFragment(firstSentence.substring(0, endOfSentence))) {
	log(ast.getLineNumber(), MSG_SUMMARY_JAVADOC);

	private boolean containsForbiddenFragment(String firstSentence) {
	final String javadocText = JAVADOC_MULTILINE_TO_SINGLELINE_PATTERN
	.matcher(firstSentence).replaceAll(" ").trim();
	return forbiddenSummaryFragments.matcher(trimExcessWhitespaces(javadocText)).find();

	else if (containsForbiddenFragment(inlineReturn)) {
	log(inlineReturnTag.getLineNumber(), MSG_SUMMARY_JAVADOC);
	}

	else if (containsForbiddenFragment(inlineSummary)) {
	log(inlineSummaryTag.getLineNumber(), MSG_SUMMARY_JAVADOC);
	}

Uh oh!

Conversation

Kevin222004 commented Jun 26, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Check :-

Mutation

Explaination

Regression :-

Uh oh!

Kevin222004 commented Jun 26, 2023

Uh oh!

github-actions bot commented Jun 26, 2023

Uh oh!

Kevin222004 commented Jun 27, 2023

Uh oh!

github-actions bot commented Jun 27, 2023

Uh oh!

romani left a comment

Choose a reason for hiding this comment

Uh oh!

rdiachenko left a comment

Choose a reason for hiding this comment

Uh oh!

Vyom-Yadav commented Jul 2, 2023

Uh oh!

Vyom-Yadav left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

romani commented Jul 2, 2023

Uh oh!

Kevin222004 commented Jul 2, 2023

Uh oh!

romani left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

romani left a comment

Choose a reason for hiding this comment

Uh oh!

Vyom-Yadav left a comment

Choose a reason for hiding this comment

Uh oh!

romani commented Jul 4, 2023

Uh oh!

romani commented Jul 4, 2023

Uh oh!

Kevin222004 commented Jul 4, 2023

Uh oh!

Vyom-Yadav commented Jul 4, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Kevin222004 commented Jun 26, 2023 •

edited

Loading