Skip to content

[core] Violation suppressors#2055

Merged
oowekyala merged 29 commits into
pmd:pmd/7.0.xfrom
oowekyala:violation-suppressors
Dec 11, 2019
Merged

[core] Violation suppressors#2055
oowekyala merged 29 commits into
pmd:pmd/7.0.xfrom
oowekyala:violation-suppressors

Conversation

@oowekyala

@oowekyala oowekyala commented Oct 5, 2019

Copy link
Copy Markdown
Member
  • Add the interface ViolationSuppressor. Implementations filter violations before they reach the report.
    • Reimplement the logic of the violationSuppress_ properties, the NOPMD comments and annotation suppression with a ViolationSuppressor each
    • So they're all treated all uniformly instead of part in the Report, part in RuleViolation#isSuppressed (with overlap)
  • Remove RuleViolation#isSuppressed. Suppressed rule violations now are always represented by a SuppressedViolation.
  • The SuppressedViolation stores the ViolationSuppressor that suppressed the violation, so that renderers can report it correctly. This was previously hardcoded in renderers as just being comment suppression vs. annotation suppression, though other types of suppression exist.
  • AbstractRuleViolationFactory is not abstract anymore as it's a completely functional implementation. I renamed it to DefaultRuleViolationFactory and removed the implementing classes that added no logic at all.
  • Similarly, some implementations of RuleViolation add no logic and are unnecessary (ApexRuleViolation).
  • Remove Parser#getSuppressMap from the Parser interface. This was just weird and made the parser stateful. It's on RootNode temporarily, though I don't like that either and think we could improve it later. I marked it InternalApi.

See also #1962 (comment)

TODO on master

  • Deprecate Parser#getSuppressMap (internal api) [core] Deprecate Parser#getSuppressMap #2084
  • Mark the implementations of RuleViolationFactory and RuleViolation as internal API. The only published API there should be the interface.

Future

I'm still not satisfied with the current amount of boilerplate. Violation suppressors are orthogonal from the violation factory (the thing creating the violation), it shouldn't be necessary for a language implementation to extend the class just to provide new suppressors. Similarly, you shouldn't have to extend RuleViolation to add new info to the violation (class name, method name, etc). Implementations don't provide any behavior, they just populate fields in the constructor. An interface ViolationDecorator + a map of properties could be used instead. All that is todo for the future and is not the topic of this PR. I'll describe that on the wiki instead.

@oowekyala oowekyala added this to the 7.0.0 milestone Oct 5, 2019

@adangel adangel left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the work on this part of the API. I have a few questions:

  • Is this PR for java-grammar or for pmd/7.0.x? I think, this is a language agnostic API change... so, you need to switch the branches for this PR.
  • I was first thinking: Is Violation suppressors really a API, do we need it? I've seen now, that it is needed for language specific suppressions à la @SuppressWarnings. So, I agree, it is public API. Then next question: Which part/division of the PMD API does this belong to? Not the one, the rule implementors use, but the one, that language implementors use. Currently the main interface ViolationSuppressor is in the main pmd package. I think, it should be in some other, more specific sub-package. I don't know, whether we talked about this yet. But currently, too many things are in the main package and I'd like to clean this up eventually. But for this, I still need to build a global understanding of all PMD APIs (at least for me), to see, how everything would fit together. That's what I'm missing unfortunately atm (the "big picture"...).
  • For most languages, the DefaultRuleViolationFactory is good enough. That made me thinking: Do we really need to support customizing this core part of PMD for languages? I have seen now two reasons for language specific customization: (1) suppressors (2) language specific violation classes. The custom suppressors I could also see as part of the language module/language registration interface: We could have similar to LanguageVersionHandler#getXPathHandler a method "getAdditionalSuppressors". For (2), I don't know whether this really makes sense. It seems to be used by Java only with JavaRuleViolations. But pmd-core doesn't know about JavaRuleViolations, it knows mostly about ParametrizedRuleViolations and therefore only the informations from that class can be used in renderers. However, as this seems to have caused a problem, JavaRuleViolation doesn't add new fields, all Java-specific fields are actually in ParametrizedRuleViolations (classname, methodname, variablename, ...) which is a bigger problem... So, what I wanted to say: We should only open up stuff for customization if it is really required. Closing down e.g. the RuleViolationFactory would reduce the API surface that we need to maintain later on.
  • Adding the "noPmdComments" to the RootNodes is not ideal, but works for the moment. The information will only be available after the parsing is done. Maybe we change the parsing at some point to include all comments as individual nodes in the AST. Not sure.
  • There is one feature, we should at least think about, how we could support it: We currently report about suppressed violations, but we don't report about "unused suppression markers" - that is a "//NOPMD" comment, that wasn't used to suppress a rule or a "@SuppressWarnings" that didn't suppress any violation. This seems to require an additional first run through the source to collect the markers and a consolidation phase at the end after the last rule has been executed.

btw.: the build is failing with some compile error:

[ERROR] /home/travis/build/pmd/pmd/pmd-core/src/main/java/net/sourceforge/pmd/lang/ast/impl/antlr4/AntlrRuleViolationFactory.java:[11,37] cannot find symbol

  symbol:   class AbstractRuleViolationFactory

  location: package net.sourceforge.pmd.lang.rule
[ERROR] /home/travis/build/pmd/pmd/pmd-core/src/main/java/net/sourceforge/pmd/lang/ast/impl/antlr4/AntlrRuleViolationFactory.java:[15,54] cannot find symbol

  symbol: class AbstractRuleViolationFactory
[ERROR] /home/travis/build/pmd/pmd/pmd-core/src/main/java/net/sourceforge/pmd/lang/ast/impl/antlr4/AntlrRuleViolationFactory.java:[16,57] incompatible types: net.sourceforge.pmd.lang.ast.impl.antlr4.AntlrRuleViolationFactory cannot be converted to net.sourceforge.pmd.lang.rule.RuleViolationFactory
[ERROR] /home/travis/build/pmd/pmd/pmd-core/src/main/java/net/sourceforge/pmd/lang/ast/impl/antlr4/AntlrRuleViolationFactory.java:[21,5] method does not override or implement a method from a supertype
[ERROR] /home/travis/build/pmd/pmd/pmd-core/src/main/java/net/sourceforge/pmd/lang/ast/impl/antlr4/AntlrRuleViolationFactory.java:[27,5] method does not override or implement a method from a supertype

Comment thread pmd-java/src/main/ant/alljavacc.xml Outdated
Comment thread pmd-jsp/src/main/java/net/sourceforge/pmd/lang/jsp/JspHandler.java Outdated
Comment thread pmd-swift/src/main/java/net/sourceforge/pmd/lang/swift/SwiftHandler.java Outdated
Comment thread pmd-visualforce/src/main/java/net/sourceforge/pmd/lang/vf/VfHandler.java Outdated
Comment thread pmd-vm/src/main/java/net/sourceforge/pmd/lang/vm/VmHandler.java Outdated
Comment thread pmd-xml/src/main/java/net/sourceforge/pmd/lang/xml/XmlHandler.java Outdated
@oowekyala

Copy link
Copy Markdown
Member Author

@adangel

Is this PR for java-grammar or for pmd/7.0.x? I think, this is a language agnostic API change... so, you need to switch the branches for this PR.

You're right it is.. I need to rework the branch a bit because I built it up on top of #1927, though as seen in the Apex module it isn't really necessary

Also I messed up the branch and this contains changes making parser support classes package-private (the branch started about that, and I forgot to branch away when shifting focus). So maybe:

  1. One PR for java-grammar about parser class visibility
  2. One PR for pmd/7.0.x with the violation suppressors
  3. Fix merge conflicts later when merging back 7.0.x into java-grammar (conflicts caused by [java] Refactor annotation suppression #1927)

Then next question: Which part/division of the PMD API does this belong to? Not the one, the rule implementors use, but the one, that language implementors use.

I think that's true. In fact, I think we could package those special APIs into impl subpackages, for code organisation.

So, for a package p for a feature (eg rule violations):

  • published API are in p -> eg the interface RuleViolation, ViolationSuppressor
    • This should be enough to understand what the feature does while browsing javadocs
  • implementation classes that we publish and support for language implementations to use are in p.impl -> eg the DefaultRuleViolationFactory, the RuleViolationFactory interface (if we need it anyway)
  • internal API, that p.impl is built upon, or that we don't want to publish yet are in p.internal

I think this packaging scheme would help keep root packages clean of implementation details and help navigate the API. This is what I've started doing when cleaning up the packaging of the antlr adapters in b51c02f (part of #1796), eg

https://github.com/pmd/pmd/blob/55fea6a18fb7bdaa4e1d250506975f6ce06f75db/pmd-core/src/main/java/net/sourceforge/pmd/lang/ast/impl/antlr4/AntlrBaseNode.java#L5

Wdyt?

We could have similar to LanguageVersionHandler#getXPathHandler a method "getAdditionalSuppressors"
Closing down e.g. the RuleViolationFactory would reduce the API surface that we need to maintain later on.

I think that would be better yes. There is no point in having language-specific violation classes either. This is what I hinted at in the "future" section of the PR description. I think the data holder described on the wiki could be used to make rule violation extensible by any language module, and allow us to remove those language specific methods on the RuleViolation interface.

Excerpt from the wiki because it's not in a section so I can't link:

Consider adding something like T getData(DataKey<T> key) to the node interface, where DataKey just ensures the visible API is strongly typed

  • In fact, the designer already has something like that: DataHolder. Nodes should most probably contain a DataHolder of the sort.
  • The current getUserData would map naturally to this new system.
  • This makes the AST "extensible" externally by our internal subsystems, and by external tools (eg designer)

There's much more to be said about how to improve our extension model but it's not the main topic of the PR

We currently report about suppressed violations, but we don't report about "unused suppression markers"

Good idea! This looks like some kind of "meta rule" actually.
More generally, it would be nice to have a rule to detect unused @SuppressWarnings. Eg a @SuppressWarnings("unchecked") when a compiler wouldn't report any unchecked warning anyway.

@oowekyala oowekyala added the is:WIP For PRs that are not fully ready, or issues that are actively being tackled label Oct 6, 2019
@oowekyala oowekyala force-pushed the violation-suppressors branch from 55fea6a to cd79d64 Compare October 6, 2019 22:13
@oowekyala oowekyala changed the base branch from java-grammar to pmd/7.0.x October 6, 2019 22:14
@adangel

adangel commented Oct 8, 2019

Copy link
Copy Markdown
Member
  1. One PR for java-grammar about parser class visibility
  2. One PR for pmd/7.0.x with the violation suppressors
  3. Fix merge conflicts later when merging back 7.0.x into java-grammar (conflicts caused by [java] Refactor annotation suppression #1927)

Sounds good.

I think this packaging scheme would help keep root packages clean of implementation details and help navigate the API. This is what I've started doing when cleaning up the packaging of the antlr adapters in b51c02f (part of #1796), eg

https://github.com/pmd/pmd/blob/55fea6a18fb7bdaa4e1d250506975f6ce06f75db/pmd-core/src/main/java/net/sourceforge/pmd/lang/ast/impl/antlr4/AntlrBaseNode.java#L5

Wdyt?

Yes, this package scheme looks good. We need to define however, what p will be and how many we will have. Some API classes will belong into one package (like RuleViolations), but certainly used by other packages as well (like used by Renderers).
Oh, and we don't need to do this right now with this PR. I've just noticed here a new class in the main package....

There is no point in having language-specific violation classes either. This is what I hinted at in the "future" section of the PR description. I think the data holder described on the wiki could be used to make rule violation extensible by any language module, and allow us to remove those language specific methods on the RuleViolation interface.

Yes, that could be possible. Although we create a "Meta-API" then between a language and a renderer...e.g. the keys need to be known by the renderer so that it can display the method name and so on. Since renderers are language independent, the DataKeys need to be as well. So we might end up "optional" common keys, and renderers can display sometimes more information from a rule violation and sometimes not (if the language doesn't provide the info).
But that's for another PR.

There's much more to be said about how to improve our extension model but it's not the main topic of the PR

exactly

We currently report about suppressed violations, but we don't report about "unused suppression markers"

Good idea! This looks like some kind of "meta rule" actually.
More generally, it would be nice to have a rule to detect unused @SuppressWarnings. Eg a @SuppressWarnings("unchecked") when a compiler wouldn't report any unchecked warning anyway.

I was thinking, whether this could be part of this ViolationSuppressor interface? Something like getAllSuppressionMarkers() for a file. So that this functionality needs to be provided as part of the language implementation. We would then somehow figure out afterwards, which of the markers have been used...

@ghost

ghost commented Oct 9, 2019

Copy link
Copy Markdown
1 Message
📖 No java rules are changed!

Generated by 🚫 Danger

@oowekyala oowekyala force-pushed the violation-suppressors branch from dd143c2 to 3d6f660 Compare October 9, 2019 17:51
@oowekyala oowekyala removed the is:WIP For PRs that are not fully ready, or issues that are actively being tackled label Oct 16, 2019
@oowekyala

Copy link
Copy Markdown
Member Author

@adangel I moved parts of this discussion into #2074 and to the wiki. I think the PR can be merged now, which I'll do when I have time

@adangel

adangel commented Oct 16, 2019

Copy link
Copy Markdown
Member

Maybe we should do the two deprecations tasks on master before merging this PR, otherwise we might forget them... otherwise I'm fine with merging it.

@oowekyala oowekyala force-pushed the violation-suppressors branch from 30752a4 to ae8c70b Compare December 10, 2019 12:03
@oowekyala oowekyala merged commit ae8c70b into pmd:pmd/7.0.x Dec 11, 2019
@oowekyala oowekyala deleted the violation-suppressors branch December 11, 2019 19:28
@adangel adangel mentioned this pull request Jan 23, 2023
55 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants