/cc @jsotuyod re #817 (comment)
I've tried really hard (6 hours) to understand the issue and produce a minimal example, can't do better than this. Sorry for the long-ass description it may be TMI in some places.
Affects PMD Version: 6.0.0
Rule: UnusedPrivateMethod
Description: The attached project gives two false positive violations for the two used methods. After hours of investigation (went pretty deep into the rabbit hole sun.misc.URLClassPath) I figured out that the .class files were added to the classpath in the wrong way... I wasn't expecting the classpath to influence the results, because I thought PMD was a source analyzer, but I guess since the type resolution was introduced, the actual classes are required.
Anyway, net.sourceforge.pmd.lang.java.symboltable.ClassScope#findVariableHere@hasAuxclasspath is too much an important variable. It decides whether the method resolution is by name+argcount or by full type. I think there's a better way: as far as I see now the boolean hasAuxclasspath argument to the matchMethodDeclaration can be removed and replaced with a calculated value. parameterTypes and argumentTypes both contain SimpleTypedNameDeclaration objects, which, in case of missing .classes have null values in type field. My proposal is to define
hasAuxclasspath = all index in params/args (when count is the same) {
(args[i].type != null && params[i].type != null) // best scenario, full type information available
|| ((args[i].type != null || params[i].type != null) && args[i].image == params[i].image) // partial info
// if both null, or the image is not exactly the same,
// don't try to guess, just accept the failure and match by name and arg number
}
Oh, and back to how this fixes UnusedPrivateMethod: it simply will fill in the scope based on the best available information, and hence the missing .class files will trigger a more lenient resolution thus preventing false positives.
Code Sample demonstrating the issue:
(see attached .zip file)
void load(EngineKey key) {
logWithTimeAndKey("", key);
loadFromActiveResources(key, true);
}
private static void logWithTimeAndKey(String log, Key key) {
}
private void loadFromActiveResources(Key key, boolean isMemoryCacheable) {
}
Running PMD through: [CLI | Ant | Maven | Gradle | Designer | Other] Gradle
Debugging
Here's how I debugged the issue:
-
Extract attached project
-
Import Gradle project into AS/IntellJ IDEA
If Sync fails with
Error:The modules ['buildSrc', 'pmd-classpath'] point to the same directory in the file system. Each module must have a unique path.
you can fix it by going to View > Tool Windows > Gradle and using the refresh button there.
It doesn't matter once it's synced, even if it says needs to resync.
-
Go to Run/Debug Configurations, and + a Remote run config
-
Set [x] Socket [x] Attach Port: 5005, module: root
-
From Terminal (in IDEA or external), run
gradlew --stop && gradlew --no-daemon --debug -Dorg.gradle.debug=true cleanPmd pmd`
-
notice it stops at "> Starting Daemon"
Observe: "daemon out: Listening for transport dt_socket at address: 5005"
-
Jump to class net.sourceforge.pmd.lang.java.symboltable.ClassScope
-
Put a breakpoint at line 244 (for > if) in matchMethodDeclaration
-
Debug the just-created run configuration,
(wait a few seconds while Gradle build progresses to :pmd task)
UnusedPrivateMethodRule.visit: occs.isEmpty() == true -> violation
Why is it empty?
- The
ClassScope's getMethodDeclarations has the method, but there are no NameOccurrences for it
AbstractScope.nameDeclarations doesn't have values for MethodNameDeclaration
ClassScope.addNameOccurrence is not executed for Engine.loadFromCache
- That method is called from
Search which is used by OccurrenceFinder
Why isn't Search traversing the load() method?
OccurrenceFinder stops on "loadFromCache".equals(occ.getImage()), and that is in the "load" method: occ.location.getFirstParentOfType(ASTMethodDeclaration.class).findChildrenOfType(ASTMethodDeclarator.aclass).get(0)
- Search traverses the scopes via
searchUpward, I expect it to stop and match on ClassScope.
ClassScope.contains -> findVariableHere(image="loadFromCache", isMethodOrConstructorInvocation=true)
getMethodDeclarations() contains the image
ClassScope.matchMethodDeclaration(): there must be a mismatch here
based on hacking around empirically EngineKey implement Key will mismatch somehow.
- name matches, count matches,
parameterTypes.equals(argumentTypes) false because one is EngineKey and the other is com.bumptech.glide.load.Key; neither SimpleTypedNameDeclaration has .type nor .next.
- getEnclosingScope(SourceFileScope.class).resolveType -> TypeSet.findClass doesn't find them.
pmdClassLoader.loadClass("com.bumptech.glide.load.Key") can't find it even though the ClassLoader.path has the file:/.../Key.class inside
- URLClassPath.path has all the class files, but URLClassPath.loaders only has 16 loaders, all them are JarLoader.
- sun.misc.URLClassPath#getLoader(java.net.URL) lead me to solution:
task pmd {
- classpath += fileTree("${project.buildDir}/intermediates/classes/debug/")
+ classpath += files("${project.buildDir}/intermediates/classes/debug/")
}
/cc @jsotuyod re #817 (comment)
I've tried really hard (6 hours) to understand the issue and produce a minimal example, can't do better than this. Sorry for the long-ass description it may be TMI in some places.
Affects PMD Version: 6.0.0
Rule:
UnusedPrivateMethodDescription: The attached project gives two false positive violations for the two used methods. After hours of investigation (went pretty deep into the rabbit hole
sun.misc.URLClassPath) I figured out that the .class files were added to theclasspathin the wrong way... I wasn't expecting the classpath to influence the results, because I thought PMD was a source analyzer, but I guess since the type resolution was introduced, the actual classes are required.Anyway,
net.sourceforge.pmd.lang.java.symboltable.ClassScope#findVariableHere@hasAuxclasspathis too much an important variable. It decides whether the method resolution is by name+argcount or by full type. I think there's a better way: as far as I see now the booleanhasAuxclasspathargument to thematchMethodDeclarationcan be removed and replaced with a calculated value.parameterTypesandargumentTypesboth containSimpleTypedNameDeclarationobjects, which, in case of missing.classes havenullvalues intypefield. My proposal is to defineOh, and back to how this fixes
UnusedPrivateMethod: it simply will fill in the scope based on the best available information, and hence the missing .class files will trigger a more lenient resolution thus preventing false positives.Code Sample demonstrating the issue:
(see attached .zip file)
Running PMD through: [CLI | Ant | Maven | Gradle | Designer | Other] Gradle
Debugging
Here's how I debugged the issue:
Extract attached project
Import Gradle project into AS/IntellJ IDEA
If Sync fails with
you can fix it by going to View > Tool Windows > Gradle and using the refresh button there.
It doesn't matter once it's synced, even if it says needs to resync.
Go to Run/Debug Configurations, and + a Remote run config
Set [x] Socket [x] Attach Port: 5005, module: root
From Terminal (in IDEA or external), run
notice it stops at "> Starting Daemon"
Observe: "daemon out: Listening for transport dt_socket at address: 5005"
Jump to class
net.sourceforge.pmd.lang.java.symboltable.ClassScopePut a breakpoint at line 244 (
for>if) inmatchMethodDeclarationDebug the just-created run configuration,
(wait a few seconds while Gradle build progresses to
:pmdtask)UnusedPrivateMethodRule.visit:occs.isEmpty() == true-> violationWhy is it empty?
ClassScope'sgetMethodDeclarationshas the method, but there are noNameOccurrencesfor itAbstractScope.nameDeclarationsdoesn't have values forMethodNameDeclarationClassScope.addNameOccurrenceis not executed forEngine.loadFromCacheSearchwhich is used byOccurrenceFinderWhy isn't Search traversing the
load()method?OccurrenceFinder stops on
"loadFromCache".equals(occ.getImage()), and that is in the "load" method:occ.location.getFirstParentOfType(ASTMethodDeclaration.class).findChildrenOfType(ASTMethodDeclarator.aclass).get(0)searchUpward, I expect it to stop and match on ClassScope.ClassScope.contains->findVariableHere(image="loadFromCache", isMethodOrConstructorInvocation=true)getMethodDeclarations()contains theimageClassScope.matchMethodDeclaration(): there must be a mismatch herebased on hacking around empirically
EngineKey implement Keywill mismatch somehow.parameterTypes.equals(argumentTypes)false because one isEngineKeyand the other iscom.bumptech.glide.load.Key; neither SimpleTypedNameDeclaration has.typenor.next.pmdClassLoader.loadClass("com.bumptech.glide.load.Key")can't find it even though theClassLoader.pathhas thefile:/.../Key.classinsidetask pmd { - classpath += fileTree("${project.buildDir}/intermediates/classes/debug/") + classpath += files("${project.buildDir}/intermediates/classes/debug/") }