I was working on potential changes to the API and got dragged down a rabbit hole when japicmp was inappropriately flagging a change as breaking binary compatibility. I.e. japicmp had a false positive.

In my case, I was toying with lifting a method from Span to a new superinterface. Japicmp flagged the method as deleted even tho it still exists on the new superinterface. After too many hours debugging and inspecting the japicmp source code, I figured out that its because we improperly configure japicmp. Japicmp has config properties that allow you to set the old and new archives to compare, and the old and new classpaths for those archives. We are:

1. Setting classpath instead of setting archive.
2. Only setting the classpath to the single archive we're comparing.

The second point causes japicmp to blow up when it tries to resolve superinterfaces / superclasses which come from different artifacts. For example, Span extends ImplicitKeyedContext - japicmp is unable to resolve ImplicitKeyedContext because opentelemetry-context is not on the classpath.

My solution: for each artifact we're comparing, set the classpath to the set of all artifacts published by opentelemetry-java. If the artifact isn't needed, it doesn't hurt for it to be present. This means we're still exposed to these types of false positives if published classes / interfaces which extended types from external projects (i.e. if we had something in the opentelemetry-exporter-zipkin that extended something from zipkin-sender-okhttp3), but this should be rare or non existent.

I spent too much time on this for the impact, but I'm pretty sure I've come across these types of false positives in the past. At some point, you just have to bite the bullet and take the time to understand the tools you depend on.