Skip to content

Private subclasses of public classes should not be reported as linkage errors #1608

@suztomo

Description

@suztomo

Followup of #1599 (comment). It was a false positive.

com.sun.tools.internal.ws.wscompile.WsgenOptions (source) references com.sun.xml.internal.ws.api.BindingID$SOAPHTTPImpl (target). The latter is a private class. Linkage Checker detected it as a linkage error. However, the reference turned out to be not harmful. This is because the source class is not instantiating the private class or declaring a variable with the type.

One of the referencing method is the code below:

// in WsgenOptions.java
  BindingID getBindingID(String protocol) {
    if (protocol.equals("soap1.1")) {
      return BindingID.SOAP11_HTTP;
    } else if (protocol.equals("Xsoap1.2")) {
      return BindingID.SOAP12_HTTP;
    } else {
      String lexical = (String)this.nonstdProtocols.get(protocol);
      return lexical != null ? BindingID.parse(lexical) : null;
    }
  }

Here, BindingID.SOAP11_HTTP has type com.sun.xml.internal.ws.api.BindingID$SOAPHTTPImpl (private class), which is a subclass of BindingID class. As WsgenOptions is not touching the private class, calling the getBindingID method does not cause a LinkageError at runtime.

Simplified Example

Suppose we have the following class A, B, and C:

package foo;

public class A {
  public static final B f = new B();

  private static class B extends A {
  }
}
package foo;

public class C {
  public static void main(String[] arguments) {
    A a = A.f;
    System.out.println(a);
  }

  void f(A.B b) { // This should fail. How?
    System.out.println(b);
  }
}

When compiled, C's class file contains a reference to (private) A$B class. Linkage Checker would report this reference as a linkage error, simply because A$B is not accessible from C. However, this is a false positive as the code compiles and run without a problem.

  Compiled from "C.java"
Constant pool:
...
   #3 = Fieldref           #23.#24        // foo/A.f:Lfoo/A$B;
...
  #23 = Class              #32            // foo/A
  #24 = NameAndType        #33:#37        // f:Lfoo/A$B;
...
  #32 = Utf8               foo/A
  #33 = Utf8               f
  #34 = Class              #41            // foo/A$B
  #37 = Utf8               Lfoo/A$B;
...
  #41 = Utf8               foo/A$B

Interestingly the field #34 is not referenced by other constant pool entries or byte code. How about the case of com.sun.tools.internal.ws.wscompile.WsgenOptions?

com.sun.tools.internal.ws.wscompile.WsgenOptions has the class reference (javap output) used in InnerClasses attribute:

#392 = Class              #408          // com/sun/xml/internal/ws/api/BindingID$SOAPHTTPImpl
...
InnerClasses:
     private static final #393= #392 of #300; //SOAPHTTPImpl=class com/sun/xml/internal/ws/api/BindingID$SOAPHTTPImpl of class com/sun/xml/internal/ws/api/BindingID

https://docs.oracle.com/javase/specs/jvms/se8/html/jvms-4.html#jvms-4.7.6

If the constant pool of a class or interface C contains at least one CONSTANT_Class_info entry (§4.4.1) which represents a class or interface that is not a member of a package, then there must be exactly one InnerClasses attribute in the attributes table of the ClassFile structure for C.

How about "If a class reference is only used by the InnerClasses annotation, we skip it"?

Java Language Specification: 12.3.3. Resolution of Symbolic References

https://docs.oracle.com/javase/specs/jls/se7/html/jls-12.html#jls-12.3.3

For constructor, we can checknew. But it cannot detect wrong access of A.B

How about "If a class reference is only used by NameAndType, we ignore it"?

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions