I've been trying to implement a class loader handler for jars produced by uno-jar (https://github.com/nsoft/uno-jar) which I intend to contribute here and use in JesterJ (https://github.com/nsoft/jesterj). I'm moving away from reflections, but I'm running into the following problem. Below is an excerpt from a 119M log output... (customer name has been redacted here, and redacting customer info from a 119M log file is intractable)
2020-04-29T21:13:24.795+0000 ClassGraph Opening classpath elements
2020-04-29T21:13:24.806+0000 ClassGraph -- Opening classpath element /mnt/data1/jesterj/customer-demo-dep-0.1-SNAPSHOT.jar!/lib/commons-codec-1.11.jar!/org/apache/commons/codec/language/bm/Rule$1.class
2020-04-29T21:13:24.806+0000 ClassGraph ---- Opening jar: /mnt/data1/jesterj/customer-demo-dep-0.1-SNAPSHOT.jar!/lib/commons-codec-1.11.jar!/org/apache/commons/codec/language/bm/Rule$1.class
2020-04-29T21:13:24.815+0000 ClassGraph ------ Deflating nested zip entry: jar:file:/mnt/data1/jesterj/customer-demo-dep-0.1-SNAPSHOT.jar!/lib/commons-codec-1.11.jar ; uncompressed size: 335042
2020-04-29T21:13:24.818+0000 ClassGraph ------ Getting zipfile slice [/mnt/data1/jesterj/customer-demo-dep-0.1-SNAPSHOT.jar!/lib/commons-codec-1.11.jar -> null ; byte range: 0..335042 / 335042] for nested jar lib/commons-codec-1.11.jar
2020-04-29T21:13:24.823+0000 ClassGraph -------- Found Automatic-Module-Name entry in manifest file: org.apache.commons.codec
2020-04-29T21:13:24.823+0000 ClassGraph ------ Deflating nested zip entry: jar:file:/mnt/data1/jesterj/customer-demo-dep-0.1-SNAPSHOT.jar!/lib/commons-codec-1.11.jar!/org/apache/commons/codec/language/bm/Rule$1.class ; uncompressed size: 641
2020-04-29T21:13:24.823+0000 ClassGraph ------ Getting zipfile slice [/mnt/data1/jesterj/customer-demo-dep-0.1-SNAPSHOT.jar!/lib/commons-codec-1.11.jar!/org/apache/commons/codec/language/bm/Rule$1.class -> null ; byte range: 0..641 / 641] for nested jar org/apache/commons/codec/language/bm/Rule$1.class
2020-04-29T21:13:24.824+0000 ClassGraph ------ Could not open jarfile /mnt/data1/jesterj/customer-demo-dep-0.1-SNAPSHOT.jar!/lib/commons-codec-1.11.jar!/org/apache/commons/codec/language/bm/Rule$1.class : java.io.IOException: Jarfile central directory signature not found: /mnt/data1/jesterj/customer-demo-dep-0.1-SNAPSHOT.jar!/lib/commons-codec-1.11.jar!/org/apache/commons/codec/language/bm/Rule$1.class
2020-04-29T21:13:24.806+0000 ClassGraph -- Opening classpath element /mnt/data1/jesterj/customer-demo-dep-0.1-SNAPSHOT.jar!/lib/lucene-core-7.5.0.jar!/org/apache/lucene/index/LiveIndexWriterConfig.class
2020-04-29T21:13:24.809+0000 ClassGraph ---- Opening jar: /mnt/data1/jesterj/customer-demo-dep-0.1-SNAPSHOT.jar!/lib/lucene-core-7.5.0.jar!/org/apache/lucene/index/LiveIndexWriterConfig.class
2020-04-29T21:13:24.830+0000 ClassGraph ------ Deflating nested zip entry: jar:file:/mnt/data1/jesterj/customer-demo-dep-0.1-SNAPSHOT.jar!/lib/lucene-core-7.5.0.jar ; uncompressed size: 3056210
2020-04-29T21:13:24.960+0000 ClassGraph ------ Getting zipfile slice [/mnt/data1/jesterj/customer-demo-dep-0.1-SNAPSHOT.jar!/lib/lucene-core-7.5.0.jar -> null ; byte range: 0..3056210 / 3056210] for nested jar lib/lucene-core-7.5.0.jar
2020-04-29T21:13:24.986+0000 ClassGraph -------- This is a multi-release jar, with versions: 9
2020-04-29T21:13:24.987+0000 ClassGraph -------- META-INF/versions/9/org/apache/lucene/analysis/tokenattributes/CharTermAttributeImpl.class masks org/apache/lucene/analysis/tokenattributes/CharTermAttributeImpl.class
2020-04-29T21:13:24.987+0000 ClassGraph -------- META-INF/versions/9/org/apache/lucene/codecs/compressing/LZ4.class masks org/apache/lucene/codecs/compressing/LZ4.class
2020-04-29T21:13:24.987+0000 ClassGraph -------- META-INF/versions/9/org/apache/lucene/document/BinaryPoint$2.class masks org/apache/lucene/document/BinaryPoint$2.class
2020-04-29T21:13:24.987+0000 ClassGraph -------- META-INF/versions/9/org/apache/lucene/document/DoubleRange.class masks org/apache/lucene/document/DoubleRange.class
2020-04-29T21:13:24.987+0000 ClassGraph -------- META-INF/versions/9/org/apache/lucene/document/FloatRange.class masks org/apache/lucene/document/FloatRange.class
2020-04-29T21:13:24.987+0000 ClassGraph -------- META-INF/versions/9/org/apache/lucene/document/IntRange.class masks org/apache/lucene/document/IntRange.class
2020-04-29T21:13:24.987+0000 ClassGraph -------- META-INF/versions/9/org/apache/lucene/document/LatLonPointDistanceQuery$1$2.class masks org/apache/lucene/document/LatLonPointDistanceQuery$1$2.class
2020-04-29T21:13:24.987+0000 ClassGraph -------- META-INF/versions/9/org/apache/lucene/document/LatLonPointDistanceQuery$1$3.class masks org/apache/lucene/document/LatLonPointDistanceQuery$1$3.class
2020-04-29T21:13:24.987+0000 ClassGraph -------- META-INF/versions/9/org/apache/lucene/document/LatLonPointInPolygonQuery$1$1.class masks org/apache/lucene/document/LatLonPointInPolygonQuery$1$1.class
2020-04-29T21:13:24.987+0000 ClassGraph -------- META-INF/versions/9/org/apache/lucene/document/LongRange.class masks org/apache/lucene/document/LongRange.class
2020-04-29T21:13:24.987+0000 ClassGraph -------- META-INF/versions/9/org/apache/lucene/document/RangeFieldQuery$QueryType$1.class masks org/apache/lucene/document/RangeFieldQuery$QueryType$1.class
2020-04-29T21:13:24.987+0000 ClassGraph -------- META-INF/versions/9/org/apache/lucene/document/RangeFieldQuery$QueryType$2.class masks org/apache/lucene/document/RangeFieldQuery$QueryType$2.class
2020-04-29T21:13:24.987+0000 ClassGraph -------- META-INF/versions/9/org/apache/lucene/document/RangeFieldQuery$QueryType$3.class masks org/apache/lucene/document/RangeFieldQuery$QueryType$3.class
2020-04-29T21:13:24.987+0000 ClassGraph -------- META-INF/versions/9/org/apache/lucene/index/BitsSlice.class masks org/apache/lucene/index/BitsSlice.class
2020-04-29T21:13:24.987+0000 ClassGraph -------- META-INF/versions/9/org/apache/lucene/index/CheckIndex$VerifyPointsVisitor.class masks org/apache/lucene/index/CheckIndex$VerifyPointsVisitor.class
2020-04-29T21:13:24.987+0000 ClassGraph -------- META-INF/versions/9/org/apache/lucene/index/CodecReader.class masks org/apache/lucene/index/CodecReader.class
2020-04-29T21:13:24.988+0000 ClassGraph -------- META-INF/versions/9/org/apache/lucene/index/MergeReaderWrapper.class masks org/apache/lucene/index/MergeReaderWrapper.class
2020-04-29T21:13:24.988+0000 ClassGraph -------- META-INF/versions/9/org/apache/lucene/index/PointValues.class masks org/apache/lucene/index/PointValues.class
2020-04-29T21:13:24.988+0000 ClassGraph -------- META-INF/versions/9/org/apache/lucene/search/BooleanScorer$TailPriorityQueue.class masks org/apache/lucene/search/BooleanScorer$TailPriorityQueue.class
2020-04-29T21:13:24.988+0000 ClassGraph -------- META-INF/versions/9/org/apache/lucene/search/PointInSetQuery$SinglePointVisitor.class masks org/apache/lucene/search/PointInSetQuery$SinglePointVisitor.class
2020-04-29T21:13:24.988+0000 ClassGraph -------- META-INF/versions/9/org/apache/lucene/search/PointRangeQuery$1$1.class masks org/apache/lucene/search/PointRangeQuery$1$1.class
2020-04-29T21:13:24.988+0000 ClassGraph -------- META-INF/versions/9/org/apache/lucene/search/PointRangeQuery$1$2.class masks org/apache/lucene/search/PointRangeQuery$1$2.class
2020-04-29T21:13:24.988+0000 ClassGraph -------- META-INF/versions/9/org/apache/lucene/search/PointRangeQuery$1.class masks org/apache/lucene/search/PointRangeQuery$1.class
2020-04-29T21:13:24.988+0000 ClassGraph -------- META-INF/versions/9/org/apache/lucene/util/BytesRef.class masks org/apache/lucene/util/BytesRef.class
2020-04-29T21:13:24.988+0000 ClassGraph -------- META-INF/versions/9/org/apache/lucene/util/BytesRefArray.class masks org/apache/lucene/util/BytesRefArray.class
2020-04-29T21:13:24.988+0000 ClassGraph -------- META-INF/versions/9/org/apache/lucene/util/CharsRef$UTF16SortedAsUTF8Comparator.class masks org/apache/lucene/util/CharsRef$UTF16SortedAsUTF8Comparator.class
2020-04-29T21:13:24.988+0000 ClassGraph -------- META-INF/versions/9/org/apache/lucene/util/CharsRef.class masks org/apache/lucene/util/CharsRef.class
2020-04-29T21:13:24.988+0000 ClassGraph -------- META-INF/versions/9/org/apache/lucene/util/IntsRef.class masks org/apache/lucene/util/IntsRef.class
2020-04-29T21:13:24.988+0000 ClassGraph -------- META-INF/versions/9/org/apache/lucene/util/LongsRef.class masks org/apache/lucene/util/LongsRef.class
2020-04-29T21:13:24.988+0000 ClassGraph -------- META-INF/versions/9/org/apache/lucene/util/StringHelper.class masks org/apache/lucene/util/StringHelper.class
2020-04-29T21:13:24.988+0000 ClassGraph -------- META-INF/versions/9/org/apache/lucene/util/automaton/Automaton$Builder.class masks org/apache/lucene/util/automaton/Automaton$Builder.class
2020-04-29T21:13:24.988+0000 ClassGraph -------- META-INF/versions/9/org/apache/lucene/util/automaton/Automaton.class masks org/apache/lucene/util/automaton/Automaton.class
2020-04-29T21:13:24.988+0000 ClassGraph -------- META-INF/versions/9/org/apache/lucene/util/bkd/BKDReader.class masks org/apache/lucene/util/bkd/BKDReader.class
2020-04-29T21:13:24.988+0000 ClassGraph -------- META-INF/versions/9/org/apache/lucene/util/bkd/BKDWriter$BKDMergeQueue.class masks org/apache/lucene/util/bkd/BKDWriter$BKDMergeQueue.class
2020-04-29T21:13:24.988+0000 ClassGraph -------- META-INF/versions/9/org/apache/lucene/util/bkd/BKDWriter.class masks org/apache/lucene/util/bkd/BKDWriter.class
2020-04-29T21:13:24.988+0000 ClassGraph -------- META-INF/versions/9/org/apache/lucene/util/bkd/MutablePointsReaderUtils$1$1.class masks org/apache/lucene/util/bkd/MutablePointsReaderUtils$1$1.class
2020-04-29T21:13:24.988+0000 ClassGraph -------- META-INF/versions/9/org/apache/lucene/util/bkd/MutablePointsReaderUtils$2.class masks org/apache/lucene/util/bkd/MutablePointsReaderUtils$2.class
2020-04-29T21:13:24.988+0000 ClassGraph -------- META-INF/versions/9/org/apache/lucene/util/bkd/MutablePointsReaderUtils$3$1.class masks org/apache/lucene/util/bkd/MutablePointsReaderUtils$3$1.class
2020-04-29T21:13:24.988+0000 ClassGraph ------ Deflating nested zip entry: jar:file:/mnt/data1/jesterj/customer-demo-dep-0.1-SNAPSHOT.jar!/lib/lucene-core-7.5.0.jar!/org/apache/lucene/index/LiveIndexWriterConfig.class ; uncompressed size: 9225
2020-04-29T21:13:24.989+0000 ClassGraph ------ Getting zipfile slice [/mnt/data1/jesterj/customer-demo-dep-0.1-SNAPSHOT.jar!/lib/lucene-core-7.5.0.jar!/org/apache/lucene/index/LiveIndexWriterConfig.class -> null ; byte range: 0..9225 / 9225] for nested jar org/apache/lucene/index/LiveIndexWriterConfig.class
2020-04-29T21:13:24.990+0000 ClassGraph ------ Could not open jarfile /mnt/data1/jesterj/customer-demo-dep-0.1-SNAPSHOT.jar!/lib/lucene-core-7.5.0.jar!/org/apache/lucene/index/LiveIndexWriterConfig.class : java.io.IOException: Jarfile central directory signature not found: /mnt/data1/jesterj/customer-demo-dep-0.1-SNAPSHOT.jar!/lib/lucene-core-7.5.0.jar!/org/apache/lucene/index/LiveIndexWriterConfig.class
2020-04-29T21:13:24.806+0000 ClassGraph -- Opening classpath element /mnt/data1/jesterj/customer-demo-dep-0.1-SNAPSHOT.jar!/lib/tika-core-1.16.jar!/org/apache/tika/sax/xpath/SubtreeMatcher.class
Prior to that there's a lot of messages about ignoring duplicate stuff, but that doesn't seem to be an error of any sort. This is the first suspicious point... after hundreds (thousands?) of sections like the two above, complaining about central directory signature not found, I eventually get:
2020-04-30T13:33:42.350+0000 ClassGraph Uncaught exception during scan
2020-04-30T13:33:42.350+0000 ClassGraph -- java.lang.ArrayIndexOutOfBoundsException: Index -1 out of bounds for length 34
2020-04-30T13:33:42.350+0000 ClassGraph -- at nonapi.io.github.classgraph.fileslice.reader.RandomAccessArrayReader.readInt(RandomAccessArrayReader.java:142)
2020-04-30T13:33:42.350+0000 ClassGraph -- at nonapi.io.github.classgraph.fastzipfilereader.LogicalZipFile.readCentralDirectory(LogicalZipFile.java:451)
2020-04-30T13:33:42.350+0000 ClassGraph -- at nonapi.io.github.classgraph.fastzipfilereader.LogicalZipFile.<init>(LogicalZipFile.java:155)
2020-04-30T13:33:42.350+0000 ClassGraph -- at nonapi.io.github.classgraph.fastzipfilereader.NestedJarHandler$3.newInstance(NestedJarHandler.java:144)
2020-04-30T13:33:42.350+0000 ClassGraph -- at nonapi.io.github.classgraph.fastzipfilereader.NestedJarHandler$3.newInstance(NestedJarHandler.java:139)
2020-04-30T13:33:42.350+0000 ClassGraph -- at nonapi.io.github.classgraph.concurrency.SingletonMap.get(SingletonMap.java:189)
2020-04-30T13:33:42.350+0000 ClassGraph -- at nonapi.io.github.classgraph.fastzipfilereader.NestedJarHandler$4.newInstance(NestedJarHandler.java:323)
2020-04-30T13:33:42.350+0000 ClassGraph -- at nonapi.io.github.classgraph.fastzipfilereader.NestedJarHandler$4.newInstance(NestedJarHandler.java:154)
2020-04-30T13:33:42.350+0000 ClassGraph -- at nonapi.io.github.classgraph.concurrency.SingletonMap.get(SingletonMap.java:189)
2020-04-30T13:33:42.350+0000 ClassGraph -- at io.github.classgraph.ClasspathElementZip.open(ClasspathElementZip.java:141)
2020-04-30T13:33:42.350+0000 ClassGraph -- at io.github.classgraph.Scanner$3.processWorkUnit(Scanner.java:581)
2020-04-30T13:33:42.350+0000 ClassGraph -- at io.github.classgraph.Scanner$3.processWorkUnit(Scanner.java:553)
2020-04-30T13:33:42.350+0000 ClassGraph -- at nonapi.io.github.classgraph.concurrency.WorkQueue.runWorkLoop(WorkQueue.java:246)
2020-04-30T13:33:42.350+0000 ClassGraph -- at nonapi.io.github.classgraph.concurrency.WorkQueue.access$000(WorkQueue.java:50)
2020-04-30T13:33:42.350+0000 ClassGraph -- at nonapi.io.github.classgraph.concurrency.WorkQueue$1.call(WorkQueue.java:201)
2020-04-30T13:33:42.350+0000 ClassGraph -- at nonapi.io.github.classgraph.concurrency.WorkQueue$1.call(WorkQueue.java:198)
2020-04-30T13:33:42.350+0000 ClassGraph -- at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
2020-04-30T13:33:42.350+0000 ClassGraph -- at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
2020-04-30T13:33:42.350+0000 ClassGraph -- at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
2020-04-30T13:33:42.350+0000 ClassGraph -- at java.base/java.lang.Thread.run(Thread.java:834)
This was running with the patch for #421 and the handler seen here: nsoft@14719d0 which relies on the following enhancement to UnoJar nsoft/uno-jar@9c59d84
Given that there is so much going on here I can't be entirely certain that this is your problem of if it's a problem I've caused somehow, but it seems like it could be an issue with your zip reading routines...
The unojar files are readable by java, and emacs and I have no problems browsing down into files within jars within the unojar in emacs, so it seems like the file and jar files within the file are valid.
I've been trying to implement a class loader handler for jars produced by uno-jar (https://github.com/nsoft/uno-jar) which I intend to contribute here and use in JesterJ (https://github.com/nsoft/jesterj). I'm moving away from reflections, but I'm running into the following problem. Below is an excerpt from a 119M log output... (customer name has been redacted here, and redacting customer info from a 119M log file is intractable)
Prior to that there's a lot of messages about ignoring duplicate stuff, but that doesn't seem to be an error of any sort. This is the first suspicious point... after hundreds (thousands?) of sections like the two above, complaining about central directory signature not found, I eventually get:
This was running with the patch for #421 and the handler seen here: nsoft@14719d0 which relies on the following enhancement to UnoJar nsoft/uno-jar@9c59d84
Given that there is so much going on here I can't be entirely certain that this is your problem of if it's a problem I've caused somehow, but it seems like it could be an issue with your zip reading routines...
The unojar files are readable by java, and emacs and I have no problems browsing down into files within jars within the unojar in emacs, so it seems like the file and jar files within the file are valid.