-
Notifications
You must be signed in to change notification settings - Fork 22
Closed
Description
Hello,
We are maintaining the open source project https://github.com/twitter/algebird, and try to upgrade algebird-spark module to new spark version 2.4.0 and scala version 2.12.7 (https://github.com/twitter/algebird/tree/apanasenko/spark_2.4.0). But we are hit a problem (works fine on 2.11.12):
https://travis-ci.org/twitter/algebird/jobs/451666066#L2031
java.lang.ArrayIndexOutOfBoundsException: 15859
at scala.tools.asm.ClassReader.readUTF(ClassReader.java:2624)
at scala.tools.asm.ClassReader.readUTF8(ClassReader.java:2596)
at scala.tools.nsc.backend.jvm.opt.InlineInfoAttribute.nextUTF8$1(InlineInfoAttribute.scala:95)
at scala.tools.nsc.backend.jvm.opt.InlineInfoAttribute.$anonfun$read$1(InlineInfoAttribute.scala:116)
at scala.tools.nsc.backend.jvm.opt.InlineInfoAttribute.$anonfun$read$1$adapted(InlineInfoAttribute.scala:114)
at scala.collection.TraversableLike.$anonfun$map$1(TraversableLike.scala:234)
at scala.collection.immutable.Range.foreach(Range.scala:156)
at scala.collection.TraversableLike.map(TraversableLike.scala:234)
at scala.collection.TraversableLike.map$(TraversableLike.scala:227)
at scala.collection.AbstractTraversable.map(Traversable.scala:104)
at scala.tools.nsc.backend.jvm.opt.InlineInfoAttribute.read(InlineInfoAttribute.scala:114)
at scala.tools.nsc.backend.jvm.opt.InlineInfoAttribute.read(InlineInfoAttribute.scala:33)
at scala.tools.asm.ClassReader.readAttribute(ClassReader.java:2458)
at scala.tools.asm.ClassReader.accept(ClassReader.java:610)
at scala.tools.nsc.backend.jvm.opt.ByteCodeRepository.$anonfun$parseClass$1(ByteCodeRepository.scala:259)
at scala.tools.nsc.backend.jvm.opt.ByteCodeRepository.parseClass(ByteCodeRepository.scala:250)
at scala.tools.nsc.backend.jvm.opt.ByteCodeRepository.$anonfun$parsedClassNode$1(ByteCodeRepository.scala:65)
at scala.collection.mutable.MapLike.getOrElseUpdate(MapLike.scala:206)
at scala.collection.mutable.MapLike.getOrElseUpdate$(MapLike.scala:203)
at scala.collection.mutable.AbstractMap.getOrElseUpdate(Map.scala:80)
at scala.tools.nsc.backend.jvm.opt.ByteCodeRepository.parsedClassNode(ByteCodeRepository.scala:65)
and
java.lang.AssertionError: assertion failed: ClassBType.info not yet assigned: Lorg/apache/spark/rdd/RDD;
at scala.tools.nsc.backend.jvm.BTypes$ClassBType.info(BTypes.scala:629)
at scala.tools.nsc.backend.jvm.BTypes$ClassBType.isNestedClass(BTypes.scala:681)
at scala.tools.nsc.backend.jvm.analysis.BackendUtils$Collector.getClassIfNested(BackendUtils.scala:332)
at scala.tools.nsc.backend.jvm.analysis.BackendUtils$NestedClassesCollector.visitInternalName(BackendUtils.scala:657)
at scala.tools.nsc.backend.jvm.analysis.BackendUtils$NestedClassesCollector.visitDescriptor(BackendUtils.scala:686)
at scala.tools.nsc.backend.jvm.analysis.BackendUtils$NestedClassesCollector.$anonfun$visit$2(BackendUtils.scala:618)
at scala.tools.nsc.backend.jvm.analysis.BackendUtils$NestedClassesCollector.$anonfun$visit$2$adapted(BackendUtils.scala:617)
at scala.collection.Iterator.foreach(Iterator.scala:944)
at scala.collection.Iterator.foreach$(Iterator.scala:944)
at scala.collection.AbstractIterator.foreach(Iterator.scala:1432)
at scala.collection.IterableLike.foreach(IterableLike.scala:71)
To reproduce results, you need to do:
$ git clone git@github.com:twitter/algebird.git
$ cd algebird/
$ git checkout apanasenko/spark_2.4.0
$ ./sbt -212 ";project algebird-spark; compile"
a problem reprdocible even if I delete all code in algebird-spark module and keep only one file with:
class AlgebirdRDD[T](val rdd: RDD[T])
Reactions are currently unavailable