[nomerge] optimise the addition of immutable HashMap#8466
[nomerge] optimise the addition of immutable HashMap#8466retronym merged 4 commits intoscala:2.12.xfrom
Conversation
a7fc744 to
7a6ee73
Compare
|
Minimization of the bootstrap failure: |
7a6ee73 to
cc549b7
Compare
854edb4 to
133bc9b
Compare
133bc9b to
4babb8a
Compare
|
rebased to resolve expected conflict with #8467 |
5e1f061 to
e0386ec
Compare
| val unwrapped = w.unwrap | ||
| (unwrapped eq HashMap.canBuildFrom) || (unwrapped eq Map.canBuildFrom) |
There was a problem hiding this comment.
I'm not sure if you can ever get multiple levels of wrapping but isCompatibleCBF(w.unwrap) would also be more concise and avoid repeating the test
|
|
||
| override def ++[C >: (A, B), That](that: GenTraversableOnce[C])(implicit bf: CanBuildFrom[HashMap[A, B], C, That]): That = | ||
| addImpl(that, bf) | ||
| private def addImpl[C >: (A, B), That](that: GenTraversableOnce[C], bf: CanBuildFrom[HashMap[A, B], C, That]): That = { |
There was a problem hiding this comment.
Is the separate method necessary? Couldn't you put the implementation directly into the second ++ method?
There was a problem hiding this comment.
if I chnage to
override def ++[B1 >: B](xs: GenTraversableOnce[(A, B1)]): Map[A, B1] = ++(xs)(HashMap.canBuildFrom[A, B1])
I get a compile error
Error:(166, 102) type mismatch;
found : scala.collection.generic.CanBuildFrom[scala.collection.immutable.HashMap.Coll,(A, B1),scala.collection.immutable.HashMap[A,B1]]
(which expands to) scala.collection.generic.CanBuildFrom[scala.collection.immutable.HashMap[_, _],(A, B1),scala.collection.immutable.HashMap[A,B1]]
required: A
override def ++[B1 >: B](xs: GenTraversableOnce[(A, B1)]): Map[A, B1] = ++(xs)(HashMap.canBuildFrom[A, B1])
There was a problem hiding this comment.
It works if you specify the type parameters:
override def ++[B1 >: B](xs: GenTraversableOnce[(A, B1)]): Map[A, B1] = ++[(A, B1), Map[A, B1]](xs)(HashMap.canBuildFrom[A, B1])
override def ++[C >: (A, B), That](that: GenTraversableOnce[C])(implicit bf: CanBuildFrom[HashMap[A, B], C, That]): That = {
The difference must be due to ++ being overloaded. It's strange that it fails with this type error rather than an ambiguous overload error.
There was a problem hiding this comment.
done - thanks for the type magic :-)
| //default Merge prefers to keep than replace | ||
| //so we merge from thatHash | ||
| (thatHash.merged(this) (null) ).asInstanceOf[That] |
There was a problem hiding this comment.
Is the handling of key and value identities here and in ++: below compatible with the current default (and new fallback) implementation consisting of successive + calls?
There was a problem hiding this comment.
I believe so - that was my intention
| var result: HashMap[A, B] = this.asInstanceOf[HashMap[A, B]] | ||
| that foreach { case kv: (A, B) => | ||
| val key = kv._1 | ||
| result = result.updated0(key, computeHash(key), 0, kv._2, kv, HashMap.liftMerger[A,B](null)) |
There was a problem hiding this comment.
Is the default merger (aka liftMerger(null)) different from a null merger (as used when calling +)?
There was a problem hiding this comment.
yes - liftMerger(null) uses a merger with retainIdentical = true
private def liftMerger[A1, B1](mergef: MergeFunction[A1, B1]): Merger[A1, B1] =
if (mergef == null) defaultMerger.asInstanceOf[Merger[A1, B1]] else liftMerger0(mergef)
private val defaultMerger : Merger[Any, Any] = new Merger[Any, Any] {
override def apply(a: (Any, Any), b: (Any, Any)): (Any, Any) = a
override def retainIdentical: Boolean = true
override val invert: Merger[Any, Any] = new Merger[Any, Any] {
override def apply(a: (Any, Any), b: (Any, Any)): (Any, Any) = b
override def retainIdentical: Boolean = true
override def invert = defaultMerger
}
}
this means that the merge code can ignore structurally shared trees
other than that the behaviour is , or should be, identical
| if (isCompatibleCBF(bf)) { | ||
| //here we know that That =:= HashMap[_, _], or compatible with it | ||
| if (this eq that) that.asInstanceOf[That] | ||
| else if (that.isEmpty) this.asInstanceOf[That] | ||
| else that match { | ||
| case thatHash: HashMap[A, B] => | ||
| //default Merge prefers to keep than replace | ||
| //so we merge from this | ||
| (this.merged(thatHash)(null)).asInstanceOf[That] | ||
| case that => | ||
| var result: HashMap[A, B] = this.asInstanceOf[HashMap[A, B]] | ||
| that foreach { case kv: (A, B) => | ||
| val key = kv._1 | ||
| result = result.updated0(key, computeHash(key), 0, kv._2, kv, HashMap.liftMerger[A,B](null)) | ||
| } | ||
| result.asInstanceOf[That] | ||
| } |
There was a problem hiding this comment.
This whole branch appears to be identical to the one from the other ++: method above. It could be factored out and shared.
There was a problem hiding this comment.
done - and removed the ObjectRef
TODO - should cope with the Maps that now have forEachEntry
There was a problem hiding this comment.
did the forEachEntry change, and copied most of that to another PR to reduce the noise in the one
|
|
||
| override def newBuilder[A, B]: mutable.Builder[(A, B), HashMap[A, B]] = new HashMapBuilder[A,B] | ||
| private class HashMapBuilder[A, B] extends mutable.MapBuilder[A, B, HashMap[A, B]](HashMap.empty) { | ||
| //not sure if this should be part of MapBuilder |
There was a problem hiding this comment.
Probably, but the override wouldn't be binary compatible and there's a risk of breaking existing implementations that make ++ in Map call MapBuilder.
There was a problem hiding this comment.
Maybe we could do something in 2.15, when that branch starts
| // condition below is due to 2 things: | ||
| // 1) no unsigned int compare on JVM | ||
| // 2) 0 (no lsb) should always be greater in comparison | ||
| if (unsignedCompare(thislsb - 1, thatlsb - 1)) { |
There was a problem hiding this comment.
Off-topic: I wonder if unsignedCompare (for Int) was ever benchmarked against a more straight-forward version that performs unsigned conversions to Long. It's also confusingly named (compare? for what?)
There was a problem hiding this comment.
I think (based in the HashSet changes, this can probably go from this implementation later
There was a problem hiding this comment.
removed reference to unsignedCompare from HashMap. There are still some from HashSet until #8589 merges
| def + [V1](kv: (Any, V1)): Map[Any, V1] = updated(kv._1, kv._2) | ||
| override def ++[V1 >: Nothing](xs: GenTraversableOnce[(Any, V1)]): Map[Any, V1] = addImpl(xs) | ||
| override def ++[B >: (Any, Nothing), That](that: GenTraversableOnce[B])(implicit bf: CanBuildFrom[Map[Any, Nothing], B, That]): That = super.++(that) | ||
| private def addImpl[B >: (Any, Nothing), That](that: GenTraversableOnce[B])(implicit bf: CanBuildFrom[Map[Any, Nothing], B, That]): That = { |
There was a problem hiding this comment.
It looks like this is only called from the first ++ above where the cbf instance is statically known.
There was a problem hiding this comment.
doh - changed to be similar to HashMap ++ and addImpl
| def apply(from: From) = wrapped.apply() | ||
|
|
||
| def apply() = wrapped.apply() | ||
| final def unwrap: CanBuildFrom[_, T, To] = wrapped |
There was a problem hiding this comment.
Why do we need this method that drops the From type instead of making wrapped a val and keeping Nothing?
There was a problem hiding this comment.
no idea. Changed it to be a val
e0386ec to
1408bce
Compare
|
addressed some of @szeiger comments |
a0d9859 to
f3bd263
Compare
|
rebased and squashed |
d765a5a to
30db477
Compare
8c45087 to
c042057
Compare
|
Pushed a commit to rebase atop the latest 2.12.x (which has |
|
/synch |
|
/nothingtoseehere (Clearing spurious "cla Expected — Waiting for status to be reported ") |
|
LGTM but it's hard to judge the correctness of all the new code so I'd prefer to get a 2nd approval from @retronym before merging. |
This avoids intermediate Tuple2 allocation. Selective forward port of scala#8466 with a few additional opportunities seized.
This avoids intermediate Tuple2 allocation. Selective forward port of scala#8466 with a few additional opportunities seized.
|
after this PR, scala-ide/scalariform has test failures in the community build: scala/scala-dev#672 . let's discuss further there |
|
Fixed in #8764 |
Regressed in scala#8466
This avoids intermediate Tuple2 allocation. Selective forward port of scala#8466 with a few additional opportunities seized.
…HashMap [nomerge] optimise the addition of immutable HashMap
Regressed in scala/scala#8466
This avoids intermediate Tuple2 allocation. Selective forward port of scala/scala#8466 with a few additional opportunities seized.
…HashMap [nomerge] optimise the addition of immutable HashMap
Regressed in scala/scala#8466
This avoids intermediate Tuple2 allocation. Selective forward port of scala/scala#8466 with a few additional opportunities seized.
Note - for 2.12 only
We can duplicate the functionality to 2.13, but I think it is with a seperate PR as 2.13 collections are differently structured
Prefer to use
mergewhen we canoptimised the builder to allow
++directlysmall collection of other optimisations
need some additional tests for the edge cases introduced