SIP-23 WIP by adriaanm · Pull Request #21 · adriaanm/scala

adriaanm · 2015-03-20T23:32:06Z

WIP PR to centralize discussion, accept improvements/tests/polish/docs/...

Everyone is welcome to look for the checkboxes/TODO items here and in the discussion below and take them on! If you're not sure where/how to start, just ask!

Testing

syntax (infix notation, backquotes, quasiquotes, string interpolation?,...)
subtyping
implicits
dependent method types
type inference: not inferred for signatures, inferred when <: Singleton

Spec/docs

include negated literal types in InfixType BNF (like prefixExpr)
update syntax overview in spec
spec the subtyping relationship between ConstantType / LiteralType and scala.Singleton.
include SIP doc under spec/

Type inference / type checking

When inferring signatures, treat literal types like singleton types (i.e., don't infer them -- widen instead)
treat singleton type bound on type param same as Singleton bound (stopWidening when bounds exists _.isStable)
include fix for https://issues.scala-lang.org/browse/SI-8564 (idm.deconst)
lub/glb
instanceOf checks: x.isInstanceOf[3] should expand to x == 3, and x.asInstanceOf[3] should be if (x != 3) throw new ClassCastException(..)

Corner cases

val t: 1 = t (NOTE: t has value 0...)

Design space

Support negative literals
Support Symbol literals --> challenge: desugars into Symbol.apply("symbol")
Support class literals? (probably not)
Model () as LiteralType? (I say 'no': () is not a syntactic literal, it's also too deeply ingrained to incorporate here)

Feature interaction

Predicate on language flag (can't do this in parser, though -- use -Xsip:23 instead?)
Update quasiquotes
Inconsistent representation of parse trees in different contexts
scalap

`LiteralType` is a `SingleType` with a literal for its path. With an example: `1` is to `p.type`, as `1` is to `p`. `lit` is the most precise type for an expression known to evaluate to `lit`. Note that we cannot constant fold the expression to `lit`, since it may have a side effect. `ConstantType(c)` is the type of an expression that can be constant folded to `c`. When an expressions should not be constant folded, the corresponding `LiteralType` is created: `ConstantType(c).deconst == LiteralType(c)` (this is where all `LiteralType`s originate, except for unpickling). `typedLiteral` always infers a `ConstantType`, which is subsequently `deconst`ed or `widen`ed where necessary (see `widenIfNecessary`). In short, only `final val`s may have `ConstantType`s, and `widen`ing is required to ensure that a path is stable and accessible (roughly speaking). When checking subtyping, this invariant is useful: `LiteralType(c).widen == c.tpe`. TODO: - test! (subtyping, type inference, type tests, patmat, dependent method types) - fully predicate implementation on -Xsip:23 - is it safe to reduce LITERAL in pickler? - symbols as literals? Notes: - `()` is not a literal Equality test for singleton type <:< AnyVal test case: ``` val y: 5 = 5 def g(x: Int) = x match { case _: y.type => 0 } ``` Accept negated literal in infixTypeRest e.g., val x : -1 => -1 = ??? TODO positions in parser

Before: sandbox/test.scala:2: error: type mismatch; found : Array[1] required: Array[Int]

Deriving this from type param upper bound in addition (as before) to constraints that we encounter scala> def stable[T <: 1](x: T): T = x stable: [T <: 1](x: T)T scala> stable(1) res0: 1 = 1 scala> stable(2) <console>:9: error: inferred type arguments [2] do not conform to method stable's type parameter bounds [T <: 1] stable(2) ^ <console>:9: error: type mismatch; found : Int(2) required: T stable(2) ^ scala> def stable[T <: Singleton](x: T): T = x stable: [T <: Singleton](x: T)T scala> val x: Int = 2 x: Int = 2 scala> val y: x.type = x y: x.type = 2 scala> stable(y) res3: x.type = 2

adriaanm · 2015-03-20T23:33:30Z

I rebased and cleaned up the PR to revive the effort. The last couple of commits (asInstanceOf for singleton types, widenIfNecessary, deconstWiden) are experiments, but the rest is looking pretty good to me.

adriaanm · 2015-03-20T23:33:40Z

/cc @milessabin

``` scala> val x = "a" x: "a" = a scala> x.asInstanceOf["a"] res1: "a" = a scala> "1".asInstanceOf["2"] x asInstanceOf Throwable java.lang.ClassCastException ... 33 elided scala> "1".asInstanceOf["1"] res3: "1" = 1 scala> 1.asInstanceOf[1] res4: 1 = 1 scala> 1.asInstanceOf[2] x asInstanceOf Throwable java.lang.ClassCastException ... 33 elided ```

Rewrite `sym.isVariable || sym.isMethod && !sym.hasAccessorFlag` to `!sym.isStable`, and drop `!tpe.typeSymbolDirect.isModuleClass`, as we already infer Foo.type instead of "object Foo" upstream (there was no regression test for it, but verified this manually).

adriaanm · 2015-03-24T09:23:31Z

See test failures at https://scala-ci.typesafe.com/job/scala-2.11.x-validate-test/457/console. Probably need to replace more deconst's with widens...

milessabin · 2015-08-06T07:32:53Z

Sorry this didn't get the attention that it deserved ... pretty much everyone who would have been involved has been wrapped up in Cats for the duration. We'll pick this up in Typelevel Scala /cc @folone.

This corrects an error in the change to the trait encoding in scala#5003: getters in traits should have empty bodies and be emitted as abstract. ``` % ~/scala/2.12.0-M4/bin/scalac sandbox/test.scala && javap -c T Compiled from "test.scala" public interface T { public abstract void T$_setter_$x_$eq(int); public int x(); Code: 0: aload_0 1: invokeinterface #15, 1 // InterfaceMethod x:()I 6: ireturn public int y(); Code: 0: aload_0 1: invokeinterface #20, 1 // InterfaceMethod y:()I 6: ireturn public void y_$eq(int); Code: 0: aload_0 1: iload_1 2: invokeinterface #24, 2 // InterfaceMethod y_$eq:(I)V 7: return public void $init$(); Code: 0: aload_0 1: bipush 42 3: invokeinterface scala#29, 2 // InterfaceMethod T$_setter_$x_$eq:(I)V 8: aload_0 9: bipush 24 11: invokeinterface #24, 2 // InterfaceMethod y_$eq:(I)V 16: return } % qscalac sandbox/test.scala && javap -c T Compiled from "test.scala" public interface T { public abstract void T$_setter_$x_$eq(int); public abstract int x(); public abstract int y(); public abstract void y_$eq(int); public static void $init$(T); Code: 0: aload_0 1: bipush 42 3: invokeinterface #21, 2 // InterfaceMethod T$_setter_$x_$eq:(I)V 8: aload_0 9: bipush 24 11: invokeinterface #23, 2 // InterfaceMethod y_$eq:(I)V 16: return public void $init$(); Code: 0: aload_0 1: invokestatic scala#27 // Method $init$:(LT;)V 4: return } ```

milessabin · 2016-06-12T19:39:19Z

FYI, I've picked this up now and I'm working on it here: https://github.com/milessabin/scala/commits/topic/sip-23-redux.

This corrects an error in the change to the trait encoding in scala#5003: getters in traits should have empty bodies and be emitted as abstract. ``` % ~/scala/2.12.0-M4/bin/scalac sandbox/test.scala && javap -c T Compiled from "test.scala" public interface T { public abstract void T$_setter_$x_$eq(int); public int x(); Code: 0: aload_0 1: invokeinterface #15, 1 // InterfaceMethod x:()I 6: ireturn public int y(); Code: 0: aload_0 1: invokeinterface #20, 1 // InterfaceMethod y:()I 6: ireturn public void y_$eq(int); Code: 0: aload_0 1: iload_1 2: invokeinterface #24, 2 // InterfaceMethod y_$eq:(I)V 7: return public void $init$(); Code: 0: aload_0 1: bipush 42 3: invokeinterface scala#29, 2 // InterfaceMethod T$_setter_$x_$eq:(I)V 8: aload_0 9: bipush 24 11: invokeinterface #24, 2 // InterfaceMethod y_$eq:(I)V 16: return } % qscalac sandbox/test.scala && javap -c T Compiled from "test.scala" public interface T { public abstract void T$_setter_$x_$eq(int); public abstract int x(); public abstract int y(); public abstract void y_$eq(int); public static void $init$(T); Code: 0: aload_0 1: bipush 42 3: invokeinterface #21, 2 // InterfaceMethod T$_setter_$x_$eq:(I)V 8: aload_0 9: bipush 24 11: invokeinterface #23, 2 // InterfaceMethod y_$eq:(I)V 16: return public void $init$(); Code: 0: aload_0 1: invokestatic scala#27 // Method $init$:(LT;)V 4: return } ```

Top level modules in Scala currently desugar as: ``` class C; object O extends C { toString } ``` ``` public final class O$ extends C { public static final O$ MODULE$; public static {}; Code: 0: new #2 // class O$ 3: invokespecial #12 // Method "<init>":()V 6: return private O$(); Code: 0: aload_0 1: invokespecial #13 // Method C."<init>":()V 4: aload_0 5: putstatic #15 // Field MODULE$:LO$; 8: aload_0 9: invokevirtual #21 // Method java/lang/Object.toString:()Ljava/lang/String; 12: pop 13: return } ``` The static initalizer `<clinit>` calls the constructor `<init>`, which invokes superclass constructor, assigns `MODULE$= this`, and then runs the remainder of the object's constructor (`toString` in the example above.) It turns out that this relies on a bug in the JVM's verifier: assignment to a static final must occur lexically within the <clinit>, not from within `<init>` (even if the latter is happens to be called by the former). I'd like to move the assignment to <clinit> but that would change behaviour of "benign" cyclic references between modules. Example: ``` package p1; class CC { def foo = O.bar}; object O {new CC().foo; def bar = println(1)}; // Exiting paste mode, now interpreting. scala> p1.O 1 ``` This relies on the way that we assign MODULE$ field after the super class constructors are finished, but before the rest of the module constructor is called. Instead, this commit removes the ACC_FINAL bit from the field. It actually wasn't behaving as final at all, precisely the issue that the stricter verifier now alerts us to. ``` scala> :paste -raw // Entering paste mode (ctrl-D to finish) package p1; object O // Exiting paste mode, now interpreting. scala> val O1 = p1.O O1: p1.O.type = p1.O$@ee7d9f1 scala> scala.reflect.ensureAccessible(p1.O.getClass.getDeclaredConstructor()).newInstance() res0: p1.O.type = p1.O$@64cee07 scala> O1 eq p1.O res1: Boolean = false ``` We will still achieve safe publication of the assignment to other threads by virtue of the fact that `<clinit>` is executed within the scope of an initlization lock, as specified by: https://docs.oracle.com/javase/specs/jvms/se8/html/jvms-5.html#jvms-5.5 Fixes scala/scala-dev#SD-194

@inlune

Non local returns aren't eliminated after inlined in 2.11 or 2.12 ``` ⚡ scala Welcome to Scala 2.12.1 (Java HotSpot(TM) 64-Bit Server VM, Java 1.8.0_112). Type in expressions for evaluation. Or try :help. scala> @inlune def foo(a: => Any) = if ("".isEmpty) a else "" <console>:11: error: not found: type inlune @inlune def foo(a: => Any) = if ("".isEmpty) a else "" ^ scala> @inline def foo(a: => Any) = if ("".isEmpty) a else "" foo: (a: => Any)Any scala> class InlineReturn { def test: Any = foo(return "") } defined class InlineReturn scala> :javap -c InlineReturn#test public java.lang.Object test(); Code: 0: new #4 // class java/lang/Object 3: dup 4: invokespecial scala#32 // Method java/lang/Object."<init>":()V 7: astore_1 8: getstatic scala#36 // Field $line4/$read$$iw$$iw$.MODULE$:L$line4/$read$$iw$$iw$; 11: aload_1 12: invokedynamic scala#59, 0 // InvokeDynamic #0:apply:(Ljava/lang/Object;)Lscala/Function0; 17: invokevirtual scala#63 // Method $line4/$read$$iw$$iw$.foo:(Lscala/Function0;)Ljava/lang/Object; 20: goto 44 23: astore_2 24: aload_2 25: invokevirtual scala#66 // Method scala/runtime/NonLocalReturnControl.key:()Ljava/lang/Object; 28: aload_1 29: if_acmpne 39 32: aload_2 33: invokevirtual scala#69 // Method scala/runtime/NonLocalReturnControl.value:()Ljava/lang/Object; 36: goto 41 39: aload_2 40: athrow 41: goto 44 44: areturn Exception table: from to target type 8 20 23 Class scala/runtime/NonLocalReturnControl ``` ``` ⚡ ~/scala/2.11.8/bin/scala Welcome to Scala 2.11.8 (Java HotSpot(TM) 64-Bit Server VM, Java 1.8.0_112). Type in expressions for evaluation. Or try :help. scala> @inline def foo(a: => Any) = if ("".isEmpty) a else "" foo: (a: => Any)Any scala> class InlineReturn { def test: Any = foo(return "") } defined class InlineReturn scala> :javap -c InlineReturn#test public java.lang.Object test(); Code: 0: new #4 // class java/lang/Object 3: dup 4: invokespecial #13 // Method java/lang/Object."<init>":()V 7: astore_1 8: getstatic #19 // Field .MODULE$:L; 11: new #21 // class InlineReturn$$anonfun$test$1 14: dup 15: aload_0 16: aload_1 17: invokespecial #24 // Method InlineReturn$$anonfun$test$1."<init>":(LInlineReturn;Ljava/lang/Object;)V 20: invokevirtual scala#28 // Method .foo:(Lscala/Function0;)Ljava/lang/Object; 23: goto 39 26: astore_2 27: aload_2 28: invokevirtual scala#31 // Method scala/runtime/NonLocalReturnControl.key:()Ljava/lang/Object; 31: aload_1 32: if_acmpne 40 35: aload_2 36: invokevirtual scala#34 // Method scala/runtime/NonLocalReturnControl.value:()Ljava/lang/Object; 39: areturn 40: aload_2 41: athrow Exception table: from to target type 8 26 26 Class scala/runtime/NonLocalReturnControl scala> :quit ```

Before: ``` scala> trait T { val foo = 24 }; class C extends T defined trait T defined class C scala> :javap -private -c C Compiled from "<console>" public class $line3.$read$$iw$$iw$C implements $line3.$read$$iw$$iw$T { private final int foo; public int foo(); Code: 0: aload_0 1: getfield #21 // Field foo:I 4: ireturn public void $line3$$read$$iw$$iw$T$_setter_$foo_$eq(int); Code: 0: aload_0 1: iload_1 2: putfield #21 // Field foo:I 5: return public $line3.$read$$iw$$iw$C(); Code: 0: aload_0 1: invokespecial scala#30 // Method java/lang/Object."<init>":()V 4: aload_0 5: invokestatic scala#34 // InterfaceMethod $line3/$read$$iw$$iw$T.$init$:(L$line3/$read$$iw$$iw$T;)V 8: return } ``` The assignment to the final field `foo` has always contravened the JVM spec, and this rule is enforced for any classfiles of format 53 and higher. After this patch: ``` scala> trait T { val foo = 24 }; class C extends T defined trait T defined class C scala> :javap -private -c C Compiled from "<console>" public class $line3.$read$$iw$$iw$C implements $line3.$read$$iw$$iw$T { private int foo; public int foo(); Code: 0: aload_0 1: getfield #21 // Field foo:I 4: ireturn public void $line3$$read$$iw$$iw$T$_setter_$foo_$eq(int); Code: 0: aload_0 1: iload_1 2: putfield #21 // Field foo:I 5: return public $line3.$read$$iw$$iw$C(); Code: 0: aload_0 1: invokespecial scala#30 // Method java/lang/Object."<init>":()V 4: aload_0 5: invokestatic scala#34 // InterfaceMethod $line3/$read$$iw$$iw$T.$init$:(L$line3/$read$$iw$$iw$T;)V 8: getstatic scala#40 // Field scala/runtime/ScalaRunTime$.MODULE$:Lscala/runtime/ScalaRunTime$; 11: invokevirtual scala#43 // Method scala/runtime/ScalaRunTime$.releaseFence:()V 14: return } ```

adriaanm and others added 8 commits March 21, 2015 07:30

SIP-23: verify pickling assumption

816aa80

SIP-23 Account for literal types in varargs translation

2cb53d7

Before: sandbox/test.scala:2: error: type mismatch; found : Array[1] required: Array[Int]

SIP-23: numericLub

7cf148c

SI-8564 deconst result of dependent method type

129ea11

widen when checking for typeref

fb0f0d2

simplify subtyping of SingletonType?

1fca48c

adriaanm added 3 commits March 24, 2015 16:10

SIP-23 widen LiteralType in result type

f68aad8

adriaanm force-pushed the sip23 branch from 53f7603 to f68aad8 Compare March 24, 2015 09:20

SIP-23 always on

16d36eb

adriaanm closed this Aug 5, 2015

milessabin mentioned this pull request Jun 14, 2016

Topic/sip 23 redux [ci: last-only] scala/scala#5227

Closed

milessabin mentioned this pull request Jul 29, 2016

SIP-23 Implementation of literal types scala/scala#5310

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

SIP-23 WIP#21

SIP-23 WIP#21
adriaanm wants to merge 12 commits into2.11.xfrom
sip23

adriaanm commented Mar 20, 2015

Uh oh!

adriaanm commented Mar 20, 2015

Uh oh!

adriaanm commented Mar 20, 2015

Uh oh!

adriaanm commented Mar 24, 2015

Uh oh!

milessabin commented Aug 6, 2015

Uh oh!

milessabin commented Jun 12, 2016

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

adriaanm commented Mar 20, 2015

Testing

Spec/docs

Type inference / type checking

Corner cases

Design space

Feature interaction

Uh oh!

adriaanm commented Mar 20, 2015

Uh oh!

adriaanm commented Mar 20, 2015

Uh oh!

adriaanm commented Mar 24, 2015

Uh oh!

milessabin commented Aug 6, 2015

Uh oh!

milessabin commented Jun 12, 2016

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants