Improvements to the type variable solver.#6127
Improvements to the type variable solver.#6127milessabin wants to merge 2 commits intoscala:2.13.xfrom
Conversation
|
Changes to type inference and other core parts of the compiler should go to 2.13 at this point. |
|
@adriaanm gotcha ... I'll rebase. |
|
Green community build here. |
24442fe to
7bd9851
Compare
|
Rebased to 2.13.x ... |
This fixes typelevel#172 reported against the literal types implementation, but is a symptom of more general issues in the solver that I've been exploring in scala#6127. Ideally this would be an independent PR, but it's very difficult to reproduce without ValueOf which would have to be replicated as a macro, so on balance I think it's better to in include it here.
This fixes typelevel#172 reported against the literal types implementation, but is a symptom of more general issues in the solver that I've been exploring in scala#6127. Ideally this would be an independent PR, but it's very difficult to reproduce without ValueOf which would have to be replicated as a macro, so on balance I think it's better to in include it here.
This fixes typelevel#172 reported against the literal types implementation, but is a symptom of more general issues in the solver that I've been exploring in scala#6127. Ideally this would be an independent PR, but it's very difficult to reproduce without ValueOf which would have to be replicated as a macro, so on balance I think it's better to in include it here.
This fixes typelevel#172 reported against the literal types implementation, but is a symptom of more general issues in the solver that I've been exploring in scala#6127. Ideally this would be an independent PR, but it's very difficult to reproduce without ValueOf which would have to be replicated as a macro, so on balance I think it's better to in include it here.
This fixes typelevel#172 reported against the literal types implementation, but is a symptom of more general issues in the solver that I've been exploring in scala#6127. Ideally this would be an independent PR, but it's very difficult to reproduce without ValueOf which would have to be replicated as a macro, so on balance I think it's better to in include it here.
This fixes typelevel#172 reported against the literal types implementation, but is a symptom of more general issues in the solver that I've been exploring in scala#6127. Ideally this would be an independent PR, but it's very difficult to reproduce without ValueOf which would have to be replicated as a macro, so on balance I think it's better to in include it here.
This fixes typelevel#172 reported against the literal types implementation, but is a symptom of more general issues in the solver that I've been exploring in scala#6127. Ideally this would be an independent PR, but it's very difficult to reproduce without ValueOf which would have to be replicated as a macro, so on balance I think it's better to in include it here.
This fixes typelevel#172 reported against the literal types implementation, but is a symptom of more general issues in the solver that I've been exploring in scala#6127. Ideally this would be an independent PR, but it's very difficult to reproduce without ValueOf which would have to be replicated as a macro, so on balance I think it's better to in include it here.
| def propagate(bound0: Type, instantiate: => Type): Unit = bound0 match { | ||
| case TypeRef(_, `tparam`, _) if !tvar.isHigherKinded && !bound0.isHigherKinded => | ||
| addBound(instantiate) | ||
| case TypeRef(_, `tparam`, _) if tvar.isHigherKinded && bound0.isHigherKinded && tparam.typeParams.size == bound0.typeParams.size => |
There was a problem hiding this comment.
I'm confused. When can this happen? If bound0.isHigherKinded shouldn't it be a PolyType and not a TypeRef? Also checking the size of type parameters might not be enough. Basically we care only if the bound is F[x, y, z[_]] <: G[x, y, z], but it could be F[x, y, z[_]] <: G[z, x, y] or F[x, y, z[_]] <: G[x, y, List].
There was a problem hiding this comment.
I can't remember the scenario now. It's possible that it's something that came up with incorrect code.
| // mentioned in the active bound are solved first. | ||
| // 1. Is this reordering desirable? Should solving always be left to right instead? | ||
| // 2. Should this take account of the variance of tparam2 and flip the bounds if | ||
| // necessary? |
There was a problem hiding this comment.
When would it be necessary to flip the bounds?
There was a problem hiding this comment.
Flip the bounds depending on whether the param is co/inv vs. contra variant.
There was a problem hiding this comment.
up depends on the variance of tparam and the variance of tparam2 shouldn't matter I think.
There was a problem hiding this comment.
I'd like to be convinced of that :-)
|
@milessabin do you need help with this? What more needs to be done? |
|
I wanted to get things to a state where |
|
I think I made some progress on my fork that improves the status quo test. It also shows that the reordering of type variables is necessary to make that happen (otherwise
|
|
I added a few comments on the Can you elaborate on why it's safe/needed to check kind conformance from |
|
That said I can be convinced to merge this, of course. Will also need a bit more comments, with motivation / brain dump, while it's fresh. Green community build. |
|
@adriaanm I don't feel strongly about checking kind conformance. I was mostly worried about the first two points from #6127 (comment), but like I said I couldn't find any counterexamples. I'm inclined to agree that it's not worth the cost, hence I will remove the check.
@milessabin this is still mostly your work, so I can PR my additions to your branch. |
b6a2076 to
018b64e
Compare
|
Rebased and merged @joroKr21's updates. |
|
@joroKr21 good work on that :-) This must fix a bunch of type inference- and hkt-related tickets ... do we know which? |
| // Propagate type constructor: F[A] <: B => F <: B.typeConstructor | ||
| // Propagate type arguments: F[+A, -B] <: C => A <: C.typeArgs(0) && B >: C.typeArgs(1) | ||
| case TypeRef(_, tsym, targs) => | ||
| val inst = toInst(instantiate) |
There was a problem hiding this comment.
Just added a check for NoType returned from toInst.
| case TypeRef(_, tsym, targs) => | ||
| val inst = toInst(instantiate) | ||
| if (tsym == tparam) addBound(variance, inst.typeConstructor) | ||
| foreach3(tsym.typeParams, targs, inst.typeArgs) { |
There was a problem hiding this comment.
I'm not convinced of this part. Subtyping only moves to arguments once the type constructors are equal, not just in some subtype relation. You'd have to do something like relating F's type args with C.baseType(F).typeArgs, but we don't support doing that stuff on type vars. Can you show with an example that this code path is crucial, and how it works?
There was a problem hiding this comment.
I feel like we're talking about the same thing over and over. This was my concern to begin with. For example:
Nil.type <:< List[x]but they don't have the same type constructorMap[k, v] <:< Traversable[(k, v)]but they don't have the same type arguments
I tried to be on the safer side by doing a kind conformance check, but as you noted it's not a 100% guarantee and it's costly.
Propagating the type arguments is necessary for t7289_status_quo. Otherwise implicitly[Ext[_ <: List[List[Int]]]] and implicitly[ExtCov[List[List[Int]]]] are ambiguous.
There was a problem hiding this comment.
In any case we have type variables on both sides so in the worst case we will infer something that doesn't respect the bounds and it will fail to compile when we check them. I couldn't find an example that compiled before and would fail with the more aggressive propagation, but if you can find one, that would be awesome.
There was a problem hiding this comment.
Thanks for bearing with me, this is tricky stuff and it's core to the type checker, so we need to take our time to understand the change and make sure it's right.
I don't see how this step propagates the constraints to the right type args, since addBound will still be constraining the type var that was originally bounded by the applied type constructor, and now we are recursing on its args without changing the type var that addBound works on.
Even if we did propagate those args to the right type var somehow, I'm not sure it would help. When evaluating the progress made in the test, it's important to keep in mind the actual types inferred using -Xprint:typer. In the covariant case (and equivalents) we just end up with Nothing. In the example, this seems to be an accident, because the type args in bounds of CC end up constraining CC itself. (Turn the debugLog in addBound into a println to see the trace.)
There was a problem hiding this comment.
@adriaanm you're right, we are propagating constraints to the type var currently under consideration. We're getting those constraints from bounds on other type variables that it appears in.
For instance, consider def foo[A <: List[B], B](x: A): Option[B] = None where A is solved as List[Int] (see, test/files/pos/bounds-propagation.scala). Now when we come to solve for B we inspect the bounds that B appears in, here the bound on A. From that we can infer an Int bound on B and so solve B as Int. With the status quo we don't consider B's occurrence in other bounds so the best we get is Nothing.
Apropos the use of baseType in this context, at least one of the participants will be an unsolved type var, so we'll not get anything useful out of it. As @joroKr21 says we'll be checking for within bounds later anyway, so I don't think there's a problem here.
|
|
||
| val variance = if (up) Variance.Covariant else Variance.Contravariant | ||
| val bound = if (up) tparam2.info.bounds.lo else tparam2.info.bounds.hi | ||
| propagate(variance, bound.dealias, tparam2.tpeHK.instantiateTypeParams(tparams, tvars)) |
There was a problem hiding this comment.
Why not do a foreach2(tparams, tvars)(propagateBounds) below, so that tparam2.tpeHK.instantiateTypeParams(tparams, tvars) can simply become tvar2? The tpeHK of a type parameter tparam2 is TypeRef(NoPrefix, tparam2, Nil), upon which we subst that to the corresponding tvar...
There was a problem hiding this comment.
I realize you didn't write this code, but now that we're all on this crazy ride down the rabbit hole...
There was a problem hiding this comment.
@adriaanm I'm not quite following what you're suggesting ... propagateBounds takes a single argument, so it doesn't fit as an argument to foreach2. We do already have tparams.foreach(propagateBounds) and propagateBounds closes over tvars.
There was a problem hiding this comment.
I agree, also in that case instantiate doesn't need to be a by-name parameter.
There was a problem hiding this comment.
I pushed a cleanup of the existing code that I see as uncontroversial: https://github.com/adriaanm/scala/tree/review/6127. I would be grateful if you could rebase your PR on top of this. I know this has already been a lot of work, but I'd be more comfortable if we move slowly here.
There was a problem hiding this comment.
Ahh ... OK, got it. Trying that now.
There was a problem hiding this comment.
My commit has that refactoring already in place. The remaining bit would be to dive into higher-order args, and peel off eta-expansion.
There was a problem hiding this comment.
Flippin' 'eck ... the solver has barely been touched in 13 years!?!
There was a problem hiding this comment.
Yup... All the more reason to be weary of users having come to depend on its precise (mal)functioning.... Experience over the years has shown that touching old code is bound to upset users. As I said before, 2.13 is focused on library changes, so we must be especially careful as we're counting on spending our "breakage budget" on the collections rework.
Based on work by joroKr21 and milessabin in scala#6127
|
Looks like Jenkins fell over ... |
|
/rebuild |
|
/synch |
|
Artifactory has been crashing recently. Not sure why. Jenkins itself has been pretty stable. |
|
|
|
@adriaanm your |
|
I've pushed my work to https://github.com/adriaanm/scala/tree/review/6127, planning to leave it at adriaanm@dc39256. Would appreciate if you could rebase on top of that as our baseline, so that the review can focus on the core of the contribution. |
Based on work by joroKr21 and milessabin in scala#6127
1b81615 to
0a78e27
Compare
|
Squashed and rebased onto @adriaanm's cleanup. |
|
To avoid causing more back-and-forth, I'll adopt the part of this PR I agree with. I don't think we should go look in the type arguments. Closing in favor of #6492 |
|
That's disappointing. We'll continue in Typelevel Scala. |
This is an attempt to fix some issues and make some small improvements in the type variable solver. It's not complete, but I'm pushing to get feedback and so that I can run a community build against it.
The main change is that I've added a more sophisticated test for the currently-being-solved
TypeVarbeing used in the bounds of another type variable, allowing more information to be propagated. This also fixes a bug where the testcase TypeRef(_, `tparam`, _)allows a fully applied type to be added to the bounds of a higher kinded type resulting in a blow up in glb/lub.I've left various open issues in comments ... these should be pursued if this is followed up on.
Nb. the test cases compile in Dotty, so this is a move in the right direction.