Skip to content

Issue #13672: Kill mutation for SuppressionXpathSingleFilter#13794

Merged
rnveach merged 1 commit into
checkstyle:masterfrom
Kevin222004:XpathSingleFilter
Nov 16, 2023
Merged

Issue #13672: Kill mutation for SuppressionXpathSingleFilter#13794
rnveach merged 1 commit into
checkstyle:masterfrom
Kevin222004:XpathSingleFilter

Conversation

@Kevin222004

@Kevin222004 Kevin222004 commented Sep 28, 2023

Copy link
Copy Markdown
Contributor

Issue #13672: Kill mutation for SuppressionXpathSingleFilter

Comment thread config/pitest-suppressions/pitest-filters-suppressions.xml
@romani

romani commented Oct 22, 2023

Copy link
Copy Markdown
Member

@Kevin222004 , I think you can make this PR as ready for review as ask mentors to help.

@romani romani self-assigned this Oct 31, 2023
@romani romani force-pushed the XpathSingleFilter branch 2 times, most recently from 2e8a8af to 7bb481f Compare November 10, 2023 01:46
@romani romani marked this pull request as ready for review November 10, 2023 01:47

@romani romani left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok to merge if CI pass.

And now I am rethinking.... Is it task to recheck that module can reset all fields to default. Don't we have some util method to validate this ?

@romani romani requested a review from rnveach November 10, 2023 01:52
@romani romani assigned rnveach and unassigned romani Nov 10, 2023
@romani

romani commented Nov 10, 2023

Copy link
Copy Markdown
Member

Windows failure:

[ERROR]   SuppressionXpathSingleFilterTest.testUpdateFilterSettingsInRunTime:437 » PatternSyntax Unknown character property name {In/Isr} near index 4
C:\projects\checkstyle\src\test\resources\com\puppycrawl\tools\checkstyle\filters\suppressionxpathsinglefilter\InputSuppressionXpathSingleFilterComplexQuery.java.never.match
    ^
[INFO] 

Stack

[ERROR] com.puppycrawl.tools.checkstyle.filters.SuppressionXpathSingleFilterTest.testUpdateFilterSettingsInRunTime -- Time elapsed: 0.015 s <<< ERROR!
java.util.regex.PatternSyntaxException: 
Unknown character property name {In/Isr} near index 4
C:\projects\checkstyle\src\test\resources\com\puppycrawl\tools\checkstyle\filters\suppressionxpathsinglefilter\InputSuppressionXpathSingleFilterComplexQuery.java.never.match
    ^
	at java.base/java.util.regex.Pattern.error(Pattern.java:2015)
	at java.base/java.util.regex.Pattern.family(Pattern.java:2904)
	at java.base/java.util.regex.Pattern.sequence(Pattern.java:2143)
	at java.base/java.util.regex.Pattern.expr(Pattern.java:2056)
	at java.base/java.util.regex.Pattern.compile(Pattern.java:1778)
	at java.base/java.util.regex.Pattern.<init>(Pattern.java:1427)
	at java.base/java.util.regex.Pattern.compile(Pattern.java:1068)
	at com.puppycrawl.tools.checkstyle.filters.SuppressionXpathSingleFilter.setFiles(SuppressionXpathSingleFilter.java:128)
	at com.puppycrawl.tools.checkstyle.filters.SuppressionXpathSingleFilterTest.testUpdateFilterSettingsInRunTime(SuppressionXpathSingleFilterTest.java:437)
	

@rnveach

rnveach commented Nov 10, 2023

Copy link
Copy Markdown
Contributor

You didn't sanitize the path before passing it on. Files is a regexp field.

@romani romani changed the title Issue #13672: Kill mutation for Filter Issue #13672: Kill mutation for SuppressionXpathSingleFilter Nov 10, 2023
@romani

romani commented Nov 10, 2023

Copy link
Copy Markdown
Member

fixed.


// reset files to default value of filter
filter.setFiles(null);
filter.finishLocalSetup();

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am pretty sure this goes against the normal lifecycle. We finish the setup 3 separate times. What is the mutation we are trying to kill?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mutation is in diff as removal, in suppression files.

@rnveach rnveach Nov 10, 2023

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I assume this is it.

https://pitest.org/quickstart/mutators/#EXPERIMENTAL_MEMBER_VARIABLE

Please Note: This mutator is likely to create equivalent mutations if a member variable is explicitly initialized with the Java default value for the specific type of the member variable as in

This seems like another example we should report to pitest to change.

Pitest should have the below change behavior for Objects:

variable = assignment;

into

if (assignment == null) new RuntimeException();
variable = null;

Similar to what it does for other mutators like RETURN_VALS.
https://pitest.org/quickstart/mutators/#return-values-mutator-return_vals

@romani romani Nov 11, 2023

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I interpreted pitest as:

public void setFiles(String files) {
        if (files == null) {
            
        }
        else {
            this.files = Pattern.compile(files);
        }
    }

and covered ^^^ mutation.
pitest was right, removal of such line was not covered.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why can't we change this code to:

 if (files != null) {
     this.files = Pattern.compile(files);
 }

?

@romani romani Nov 12, 2023

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We try the best to allow user to be able to mutate modules to default state of properties.
In other words, user created module, changed values of properties, changed his mind and want to change property value back to default.

it is not possible in XML in most cases where null is default value. But xml config is not whole world for us, I hope one day there will be more formats we support. If our java beans modules continue to allow setting default value, we will always be good.

So this is extreme edge case for us, I doubt much users care about this, but as a library we should not have opinion on our modules usage, and just allow to set default value back by setter.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do users really consider check module properties mutable? I understand that due to the way we instantiate/configure checks that it is not reasonable to assign all properties via constructor as final fields, but usage of check modules as mutable objects seems like a bad idea to me.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Modules were mutable for 20 years.
We do not know how our users use us.
But ideally module should be initialized by explicit default values and work same as without being explicit.

@rnveach rnveach requested a review from nrmancuso November 11, 2023 21:50
@rnveach rnveach assigned nrmancuso and unassigned rnveach Nov 11, 2023

@nrmancuso nrmancuso left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMO, the pitest suppression is better to keep than this crazy test. But if we want to translate tech debt from suppression files to tests, let's do it.

@nrmancuso nrmancuso removed their assignment Nov 15, 2023
@romani

romani commented Nov 15, 2023

Copy link
Copy Markdown
Member

Keep him mind that survival is here because we use null for default value of property. All modules which doing this are prone to it. By default value is null and there is special behavior for such null. But user can not set null by xml config, if user want something to be unset, he should not have property tag in config.

But this creates inconsistency. All other fields can be defined in xml with explicit value that is equal to default and it works as default. But nullable properties are different.

That is why I demanding in new design not null property, and be like empty string or anything that can be set in xml as value.

Ideally we should refactor module and do not allow java nulls , but this is breaking compatibility and a way more complicated, out of scope of this project.

If we have time, we need to update all modules to have ability to set default value explicitly iby supported configs.

This problem very visible in plugins who do some UI for user to create config, dealing with such nulls is a pain and special hacks to handle differently than other properties.

@rnveach

rnveach commented Nov 15, 2023

Copy link
Copy Markdown
Contributor

Ideally we should refactor module and do not allow java nulls
but this is breaking compatibility
a way more complicated

I agree on refactoring to remove nulls. I disagree that this is complicated. Why can't we just remove the null and be that? What more needs to be involved that makes this complicated.

Instead of doing all at once, just do one by one when we run into it with pitest.

very visible in plugins who do some UI for user to create config, dealing with such nulls is a pain and special hacks to handle differently than other properties

What plugins do this hack with modules and nulls? The deepest eclipse-cs goes that I can recall is creating the checker themselves, not the nested modules.

@romani

romani commented Nov 15, 2023

Copy link
Copy Markdown
Member

I have no time for this update now, I still have macros project as not finished.

All is doable just takes a bit of planning, investigation, scope definition, ..... .

We can have this battle later on, we need to finish projects and clean up our PRs list, and start new project without backlog, too much is assigned to be to unblock.

@rnveach rnveach assigned romani and unassigned rnveach Nov 15, 2023
@romani romani assigned rnveach and unassigned romani Nov 15, 2023
@rnveach rnveach assigned romani and unassigned rnveach Nov 15, 2023
@romani

romani commented Nov 16, 2023

Copy link
Copy Markdown
Member

Complications of to think to make breaking changes or not.
https://checkstyle.org/filters/suppressionxpathsinglefilter.html#Properties
All nulls.
We usually break something when we have a reason, now reason is weak.
This fix just bubble up problem that run into before and decided to postpone and even now I do not have energy to think on this and define what it should be.

Values in xdoc are values that user can set to java beans, so our API is java beans. Xml config is just convenience format, the only firmat, and unfortunately it does not have null. We already have somewhere idea to support json like config, and it will have null definition :) so we will be able to cover such mutation by json config implementation.

I am in position do not break, do not be involved in new project, and simple put coverage for pitest. Imagine we need this test for jacoco, it is same.

I can put extra comment on this test that it is required until we support config with null as first class element (json).

Each module should get explicitly provided default values, and it should work same as no values provided.

What plugins do this hack with modules and nulls?

I do not have exact links, Imagine you render some form for user, there are some fields, you already put default values in them to show user what is default, it is convenient for module creation you just take data entry fields values and set in bean.
If you need to deal with nulls , it is unfortunate.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants