[java] Added 4 performance rules originating from PMD-jPinpoint-rules#1932
[java] Added 4 performance rules originating from PMD-jPinpoint-rules#1932jborgers wants to merge 3 commits into
Conversation
…oidCalendarDateCreation, AvoidConcatInAppend and AvoidConcatInLoop. Including unit tests and examples. Originating from github.com/jborgers/PMD-jPinpoint-rules. Links to more doc later.
Generated by 🚫 Danger |
oowekyala
left a comment
There was a problem hiding this comment.
Thanks for the PR! I think some things can be simplified/ are debatable
Also, please note for the future:
- the documentation is generated by the CI build - you don't need to generate it manually. Though now it's there, it can stay
- it would be nice to make a separate PR for each rule next time
| </example> | ||
| </rule> | ||
|
|
||
| <rule name="AvoidConcatInAppend" |
There was a problem hiding this comment.
Isn't that exactly what InefficientStringBuffering is meant to be?
There was a problem hiding this comment.
Possibly. We don't have that rule in the subset we use, probably because we didn't update that selection for some time. There is certainly overlap, not sure if they are exactly the same.
There was a problem hiding this comment.
On a separate PR for each rule: Juan asked me to create a PR per batch, a couple of rules together. Not sure of best choice here.
There was a problem hiding this comment.
They appear to have the same intention, but there are some differences in their implementation:
For example
- your rule flags
wrappedLine.append(str + "c", a, c);correctly whereas the other rule gives up - your rule flag
sb.append(1 + 2), which is not a concatenation, so that's a false positive - your rule considers concatenation with a non-static final field as a constant expression, which it isn't
- the other rule doesn't flag any cases of
new StringBuilder().append(...), ie use of append directly on the constructor invocation, whereas yours does it correctly
I think we should fix the false negatives of the existing rule but not merge this new rule.
There was a problem hiding this comment.
Actually this rule has many more problems... It uses too many // tests, so that you end up testing unrelated nodes. Consider this violation where the rule sees an append, a string concatenation somewhere in another statement (the throw statement) and reports on a common ancestor (the for statement), even though the concatenation and the append are unrelated. All the violations I've seen on the spring framework in the report are false positives either of this kind, or where there's no concatenation but just numeric addition.
There was a problem hiding this comment.
I agree, we should enhance the existing rule InefficientStringBuffering instead of adding this new rule.
FWIW: I've noticed, that the found violation don't highlight the append - at least for the 3 violations found in checkstyle: I wouldn't know, what to fix, if I get these violations. We should aim to report exactly the node which contains the concatenation. Maybe the reason for this is the //, that Clément mentioned?
ss suggested by oowekyala.
…oop and String field.
oowekyala
left a comment
There was a problem hiding this comment.
Thanks again for your work on this. I've taken a look at the CI report though, and there are many false positives. Please take a look at the following comments
About using one PR per rule, I think this would have made it easier to review rules separately, and to merge incrementally. While the rule about FileItems and AvoidConcatInLoops are not far from ready, AvoidConcatInAppend should IMO not be merged, and changes to CalendarDateCreation may delay the other rules. So I think separate PRs would have been a better choice
| class="net.sourceforge.pmd.lang.rule.XPathRule" | ||
| typeResolution="true" | ||
| externalInfoUrl="${pmd.website.baseurl}/pmd_rules_java_performance.html#avoidcalendardatecreation"> | ||
| <description>Problem: A Calendar is a heavyweight object and expensive to create. |
There was a problem hiding this comment.
The assumption of the rule doesn't hold if the calendar is not created just to call this method, or if it is eg passed as a parameter and not created locally. Consider for example this violation in the Spring Framework, which looks like a false positive.
| </example> | ||
| </rule> | ||
|
|
||
| <rule name="AvoidConcatInAppend" |
There was a problem hiding this comment.
Actually this rule has many more problems... It uses too many // tests, so that you end up testing unrelated nodes. Consider this violation where the rule sees an append, a string concatenation somewhere in another statement (the throw statement) and reports on a common ancestor (the for statement), even though the concatenation and the append are unrelated. All the violations I've seen on the spring framework in the report are false positives either of this kind, or where there's no concatenation but just numeric addition.
| <property name="version" value="2.0"/> | ||
| <property name="xpath"> | ||
| <value><![CDATA[ | ||
| (//ForStatement | //WhileStatement | //DoStatement)//AssignmentOperator[ |
There was a problem hiding this comment.
The rule should be checking that the variable being concatenated is defined outside the loop.
Consider this violation:
for (String fileExtension : fileExtensions) {
if (fileExtension.charAt(0) != '.') {
fileExtension = "." + fileExtension; // here
}
this.fileExtensions.add(fileExtension);
}The concatenation is local to each iteration and replacing it with a StringBuilder would only be noise.
| </example> | ||
| </rule> | ||
|
|
||
| <rule name="AvoidApacheCommonsFileItemNonStreaming" |
There was a problem hiding this comment.
Maybe a better name would be UseIOStreamsWithApacheCommonsFileItem? IMHO "streaming" immediately hints at the Stream API and not IO streams, so I think it would be clearer
| externalInfoUrl="${pmd.website.baseurl}/pmd_rules_java_performance.html#avoidapachecommonsfileitemnonstreaming"> | ||
| <description> | ||
| Problem: Use of FileItem.get and FileItem.getString could exhaust memory since they load the entire file into memory | ||
| Solution: Use streaming methods and buffering. |
There was a problem hiding this comment.
Could you mention getInputStream() explicitly in the description? Like for the name of the rule, the "streaming methods" here sounds unclear to me
| class="net.sourceforge.pmd.lang.rule.XPathRule" | ||
| typeResolution="true" | ||
| externalInfoUrl="${pmd.website.baseurl}/pmd_rules_java_performance.html#avoidconcatinappend"> | ||
| <description>Concatenation of Strings is used inside an StringBuilder.append argument. Problem: Each statement with one or more +-operators creates a hidden temporary StringBuilder, a char[] and a new String object, which all have to be garbage collected. |
There was a problem hiding this comment.
FYI: Just use markdown syntax for the description. No need to use for a new line, just add a empty line. We'll parse the text here (including the line breaks) and use it in our markdown-based rule documentation.
In order to easily read this on github, also consider breaking the lines after 120 characters.
| xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" | ||
| xsi:schemaLocation="http://pmd.sourceforge.net/rule-tests http://pmd.sourceforge.net/rule-tests_1_0_0.xsd"> | ||
| <test-code> | ||
| <description>Violation: Avoid contact in append method invocations</description> |
|
|
||
| private String good(String arg) { | ||
| StringBuilder sb = new StringBuilder(); | ||
| sb.append("arg='").append(arg).append("'"); |
There was a problem hiding this comment.
The better one:
StringBuilder sb = new StringBuilder("arg='").append(arg).append("'");Perhaps, it's better to use best practices or at least mention that this example isn't the best.
There was a problem hiding this comment.
Agreed, we should change the sample code. Note: performance.md is generated, the actual change has to happen in performance.xml.
There was a problem hiding this comment.
Note: performance.md is generated, the actual change has to happen in performance.xml.
Is it something that can be automated with Danger? :)
| String log = ""; | ||
| List<String> values = Arrays.asList("tic ", "tac ", "toe "); | ||
| for (String val : values) { | ||
| log += val; |
There was a problem hiding this comment.
Is string concatenation is still an issue nowadays (2019) ? I've heard that there were optimizations for that in JDK9, for example.
There was a problem hiding this comment.
There is indeed some optimization done with Java 9:
- http://openjdk.java.net/jeps/280
- https://stackoverflow.com/questions/46512888/how-is-string-concatenation-implemented-in-java-9
Before Java9, inside the loop, the code generated was basically:
StringBuilder temp = new StringBuilder();
log = temp.append(log).append(val).toString();
Since Java9, it looks more like (roughly):
log = StringConcatFactory.makeConcatWithConstants(..., log, val);
So, the rule is definitely less relevant for java9+.
There was a problem hiding this comment.
it's worth noting, that this specific example would be unaffected by Java 9+ optimizations, as the concat is done within a loop
Originally from pmd#1932 Author: jborgers <jborgers@jpinpoint.com>
And also fix false positive when concatenation is local to each for-loop iteration. Test cases originally from pmd#1932 Author: jborgers <jborgers@jpinpoint.com>
Happy to add the following Java performance rules:
Including unit tests and examples. Originating from github.com/jborgers/PMD-jPinpoint-rules.
PR Description:
This is the first set of rules from PMD-jPinpoint-rules, sponsored by Rabobank. I discussed the rules and PR with Juan Martín Sotuyo Dodero.
Regards, Jeroen