Don't lift by-name parameter args to defs in constructor context#25157
Don't lift by-name parameter args to defs in constructor context#25157tanishiking wants to merge 4 commits intoscala:mainfrom
Conversation
tests/run/i24201c.check
Outdated
| b evaluated | ||
| c evaluated |
There was a problem hiding this comment.
It's not related to this change, but I thought c should be evaluated first 🤔
//> using scala 3.8.1
abstract class Foo(a: => Unit, b: Int, c: Int)
object Bar extends Foo(
c = { println("c evaluated"); 2 },
a = { println("a evaluated") },
b = { println("b evaluated"); 1 }
)
@main def main(): Unit =
Bar
// b evaluated
// c evaluatedHaving by-name parameter changes the evaluation order? needs to check Scala spec.
//> using scala 3.8.1
abstract class Foo(a: Int, b: Int, c: Int)
object Bar extends Foo(
c = { println("c evaluated"); 2 },
a = { println("a evaluated"); 3 },
b = { println("b evaluated"); 1 }
)
@main def main(): Unit =
Bar
// c evaluated
// a evaluated
// b evaluatedThere was a problem hiding this comment.
I thought c should be evaluated first
Indeed, it seems something already goes wrong semantically somewhere.
| protected def liftedType(expr: Tree)(using Context): Type = | ||
| // don't instantiate here, as the type params could be further constrained, see tests/pos/pickleinf.scala | ||
| val tp = expr.tpe.widen.deskolemized | ||
| if liftedFlags.is(Method) then ExprType(tp) else tp |
There was a problem hiding this comment.
Can the protected method be final?
| * see https://github.com/scala/scala3/issues/24201 | ||
| */ | ||
| override def noLift(expr: tpd.Tree)(using Context): Boolean = | ||
| super.noLift(expr) || (liftedType(expr).isInstanceOf[ExprType] && ctx.owner.isConstructor) |
There was a problem hiding this comment.
IIRC, the check ctx.owner.isConstructor is not essential, but to limit scope of impact, right? Otherwise, the criteria is ad-hoc and difficult to justify semantically. Better document it explicitly.
I was reading the doc at the beginning of the file:
Lifting generally lifts impure expressions only, except in the case of possible
default arguments, where we lift also complex pure expressions, since in that case
arguments can be duplicated as arguments to default argument methods.
The change here seems to weaken the invariant, but semantically duplication seems to be OK for by-name arguments. WDYT? /cc: @sjrd @smarter @odersky
IMHO there is a minor engineering concern: whether the change eliminates a whole category of bugs? If not, the complication here can be controversial given that it's semantically valid for Lifter to lift by-name arguments.
There was a problem hiding this comment.
Hmm, right. Checking ctx.owner.isConstructor is ad-hoc, and this re-introduce a bug that is fixed by #3839 to constructor context:
abstract class Foo(f: => Unit)(naming: Int = f + 4)
class Bar extends Foo({
val x: Int = 5
println(x)
})()
// After Typer
class Bar extends Foo(
{ val x: Int = 5; println(x) }
)(
Foo.$lessinit$greater$default$2(
{ val x: Int = 5; println(x) } // duplicated for default args
)
)
def $lessinit$greater$default$2(f: => Unit): Int = f + 4because default values can depend on preceding parameters.
However, for by-name parameter args, duplication is fine indeed, because it doesn't break evaluation order (and how many times it's evaluated).
Currently, this leads to compiler crash because eventually it's transformed by hositSuperArgs as follows. we might want to duplicating with fresh symbol? If that's not an option, we need to lift the by-name param args in different way (like lift directly to static or local definition in constructor context?)
class Bar extends
Foo(Bar$superArg$1())(Bar$superArg$2()) {
private static def Bar$superArg$1(): () ?=>
Unit = () => { val x: Int = 5; println(x) }
private static def Bar$superArg$2(): Int =
Foo.$lessinit$greater$default$2(() =>
{ val x: Int = 5; println(x) } // same SYMBOL as above x. check failed
)
def $lessinit$greater$default$2(f: => Unit): Int = f + 4
}Fatal compiler crash when compiling: tests/pos/i25160.scala:-compiler-nonbootstrapped / Test /assertion failed: bad owner; value x has owner method $anonfun, expected was method $anonfun
owner chain = value x, method $anonfun, method Bar$superArg$2, class Bar, package <empty>, package <root>, ctxOwners = method $anonfun, method $anonfun, method $anonfun, method Bar$superArg$1, method Bar$superArg$1, method Bar$superArg$1, class Bar, class Bar, package <empty>, package <root>, package <root>, package <root>, package <root>, package <root>, package <root>, package <root>, <none>, <none>, <none>, <none>, <none>
c4050ca to
0c34b92
Compare
It turned out by-name actuals do not need a Typer-local `def` alias to preserve their semantics. `ElimByName` already lowers them to by-name closures, so the `LiftToDefs` path in `EtaExpansion` was only changing typed tree shape and constraint propagation. This removes the by-name-specific lifting path from `EtaExpansion`, drops the constructor parent-call helper plumbing in `Typer`, and falls back to the normal `superCallContext`. The `i24201` regression tests stay in place. They still pass without the Typer-side helper path, which confirms that their behavior is handled by later phases rather than by `LiftToDefs`. This also changes the outcome of `i18123`. With the old lifting scheme, Typer inserted synthetic `parse0$N` defs for by-name extension receivers, and those proxies over-constrained the second `rep` call. That made the non-transparent version fail and forced the existing `pos/i18123.scala` to use `transparent inline`. Without those synthetic defs, the non-transparent version typechecks as well, so move it to `pos/i18123b.scala` and drop the old negative tests. Also remove the temporary `byname-named-reorder` experiment, which was only used to check whether plain argument reordering depended on `LiftToDefs`. It did not.
…esh syms fix scala#24201 This change also fixes scala#18123 (not fully understand why) **Background** `LiftToDefs` lifts by-name arguments to synthetic `def`s so that a method argument would not be duplicated both as the real argument and as an argument to a default getter. see scala#2939 A transformation for by-name semantics is done later in `ElimByName`, which turns a by-name actual argument into a closure. **Problem** However, lifting by-name arguments can be problematic in a specific scenario. scala#24201 For example, in `extends Foo[Baz](Baz.E1, arg2 = 3)`: ```scala abstract class Foo[T](value: => T, arg1: Int = 1, arg2: Int = 2) enum Baz { case E1 } object Baz extends Foo[Baz](Baz.E1, arg2 = 3) ``` `LiftToDefs` lifts the by-name argument `Baz.E1` to the instance method like: ```scala object Baz extends { def defaultValue$1: Baz = Baz.E1 // instance method val arg1$1: Int = Foo.$lessinit$greater$default$2[Baz] new Foo[Baz](defaultValue$1, arg1$1, arg2 = 3) } ``` where `defaultValue$1` is a synthesized instance method owned by the constructor. This is not valid code, because the constructor super call needs to access uninitialized `this` to call `defaultValue$1`, which results in a `VerifyError` during JVM bytecode verification. **Why previous patches did not work** - scala#24983 tried to undo lifting in a later phase (`HoistSuperArgs`) - scala#25157 skips lifting when the owner is a constructor but neither worked well, because both essentially cancel lifting in the specific code shape (when inside a constructor super call). However, cancelling lifting reintroduces the problem fixed by scala#3839 for those code shapes. **Solution** This commit fixes scala#24201 without breaking scala#2939 by removing `LiftToDefs` entirely. Instead, we avoid the duplicated symbol problem by copying by-name parameter arguments with fresh symbols. This removes the constructor-local synthetic-method path entirely and keeps the code as-is. (This is not a problem regarding evaluation order, because `ElimByName` translates `Baz.E1` into the closure `() => Baz.E1`.) ```scala object Baz extends Foo[Baz](Baz.E1, arg2 = 3) ``` The default-getter handling is moved to the point where duplication actually occurs (`Applications.spliceMeth`). When `spliceMeth` rebuilds a default-getter call, it now freshens only by-name arguments. For example, the example code below: ```scala // i7477.scala def spawn(f: => Unit)(naming: Int = 4): Unit = ??? def test(): Unit = spawn { val x: Int = 5 x }() ``` previously, it is transformed into: ```scala def f$1: Unit = { val x: Int = 5 { x; () } } this.spawn(f$1)(this.spawn$default$2(f$1)) ``` now, the same tree is transformed into: ```scala this.spawn({ val x$1: Int = 5 { x$1 () }} )(this.spawn$default$2({ val x$2: Int = 5 { x$2 () } }) ) ```
…esh syms fix scala#24201 This change also fixes scala#18123 (not fully understand why) **Background** `LiftToDefs` lifts by-name arguments to synthetic `def`s so that a method argument would not be duplicated both as the real argument and as an argument to a default getter. see scala#2939 A transformation for by-name semantics is done later in `ElimByName`, which turns a by-name actual argument into a closure. **Problem** However, lifting by-name arguments can be problematic in a specific scenario. scala#24201 For example, in `extends Foo[Baz](Baz.E1, arg2 = 3)`: ```scala abstract class Foo[T](value: => T, arg1: Int = 1, arg2: Int = 2) enum Baz { case E1 } object Baz extends Foo[Baz](Baz.E1, arg2 = 3) ``` `LiftToDefs` lifts the by-name argument `Baz.E1` to the instance method like: ```scala object Baz extends { def defaultValue$1: Baz = Baz.E1 // instance method val arg1$1: Int = Foo.$lessinit$greater$default$2[Baz] new Foo[Baz](defaultValue$1, arg1$1, arg2 = 3) } ``` where `defaultValue$1` is a synthesized instance method owned by the constructor. This is not valid code, because the constructor super call needs to access uninitialized `this` to call `defaultValue$1`, which results in a `VerifyError` during JVM bytecode verification. **Why previous patches did not work** - scala#24983 tried to undo lifting in a later phase (`HoistSuperArgs`) - scala#25157 skips lifting when the owner is a constructor but neither worked well, because both essentially cancel lifting in the specific code shape (when inside a constructor super call). However, cancelling lifting reintroduces the problem fixed by scala#3839 for those code shapes. **Solution** This commit fixes scala#24201 without breaking scala#2939 by removing `LiftToDefs` entirely. Instead, we avoid the duplicated symbol problem by copying by-name parameter arguments with fresh symbols. This removes the constructor-local synthetic-method path entirely and keeps the code as-is. (This is not a problem regarding evaluation order, because `ElimByName` translates `Baz.E1` into the closure `() => Baz.E1`.) ```scala object Baz extends Foo[Baz](Baz.E1, arg2 = 3) ``` The default-getter handling is moved to the point where duplication actually occurs (`Applications.spliceMeth`). When `spliceMeth` rebuilds a default-getter call, it now freshens only by-name arguments. For example, the example code below: ```scala // i7477.scala def spawn(f: => Unit)(naming: Int = 4): Unit = ??? def test(): Unit = spawn { val x: Int = 5 x }() ``` previously, it is transformed into: ```scala def f$1: Unit = { val x: Int = 5 { x; () } } this.spawn(f$1)(this.spawn$default$2(f$1)) ``` now, the same tree is transformed into: ```scala this.spawn({ val x$1: Int = 5 { x$1 () }} )(this.spawn$default$2({ val x$2: Int = 5 { x$2 () } }) ) ```
…esh syms fix scala#24201 This change also fixes scala#18123 (not fully understand why) **Background** `LiftToDefs` lifts by-name arguments to synthetic `def`s so that a method argument would not be duplicated both as the real argument and as an argument to a default getter. see scala#2939 A transformation for by-name semantics is done later in `ElimByName`, which turns a by-name actual argument into a closure. **Problem** However, lifting by-name arguments can be problematic in a specific scenario. scala#24201 For example, in `extends Foo[Baz](Baz.E1, arg2 = 3)`: ```scala abstract class Foo[T](value: => T, arg1: Int = 1, arg2: Int = 2) enum Baz { case E1 } object Baz extends Foo[Baz](Baz.E1, arg2 = 3) ``` `LiftToDefs` lifts the by-name argument `Baz.E1` to the instance method like: ```scala object Baz extends { def defaultValue$1: Baz = Baz.E1 // instance method val arg1$1: Int = Foo.$lessinit$greater$default$2[Baz] new Foo[Baz](defaultValue$1, arg1$1, arg2 = 3) } ``` where `defaultValue$1` is a synthesized instance method owned by the constructor. This is not valid code, because the constructor super call needs to access uninitialized `this` to call `defaultValue$1`, which results in a `VerifyError` during JVM bytecode verification. **Why previous patches did not work** - scala#24983 tried to undo lifting in a later phase (`HoistSuperArgs`) - scala#25157 skips lifting when the owner is a constructor but neither worked well, because both essentially cancel lifting in the specific code shape (when inside a constructor super call). However, cancelling lifting reintroduces the problem fixed by scala#3839 for those code shapes. **Solution** This commit fixes scala#24201 without breaking scala#2939 by removing `LiftToDefs` entirely. Instead, we avoid the duplicated symbol problem by copying by-name parameter arguments with fresh symbols. This removes the constructor-local synthetic-method path entirely and keeps the code as-is. (This is not a problem regarding evaluation order, because `ElimByName` translates `Baz.E1` into the closure `() => Baz.E1`.) ```scala object Baz extends Foo[Baz](Baz.E1, arg2 = 3) ``` The default-getter handling is moved to the point where duplication actually occurs (`Applications.spliceMeth`). When `spliceMeth` rebuilds a default-getter call, it now freshens only by-name arguments. For example, the example code below: ```scala // i7477.scala def spawn(f: => Unit)(naming: Int = 4): Unit = ??? def test(): Unit = spawn { val x: Int = 5 x }() ``` previously, it is transformed into: ```scala def f$1: Unit = { val x: Int = 5 { x; () } } this.spawn(f$1)(this.spawn$default$2(f$1)) ``` now, the same tree is transformed into: ```scala this.spawn({ val x$1: Int = 5 { x$1 () }} )(this.spawn$default$2({ val x$2: Int = 5 { x$2 () } }) ) ```
…esh syms (#25502) fix #24201 This change also fixes #18123 (not fully understand why) **Background** `LiftToDefs` lifts by-name arguments to synthetic `def`s so that a method argument would not be duplicated both as the real argument and as an argument to a default getter. see #2939 A transformation for by-name semantics is done later in `ElimByName`, which turns a by-name actual argument into a closure. **Problem** However, lifting by-name arguments can be problematic in a specific scenario. #24201 For example, in `extends Foo[Baz](Baz.E1, arg2 = 3)`: ```scala abstract class Foo[T](value: => T, arg1: Int = 1, arg2: Int = 2) enum Baz { case E1 } object Baz extends Foo[Baz](Baz.E1, arg2 = 3) ``` `LiftToDefs` lifts the by-name argument `Baz.E1` to the instance method like: ```scala object Baz extends { def defaultValue$1: Baz = Baz.E1 // instance method val arg1$1: Int = Foo.$lessinit$greater$default$2[Baz] new Foo[Baz](defaultValue$1, arg1$1, arg2 = 3) } ``` where `defaultValue$1` is a synthesized instance method owned by the constructor. This is not valid code, because the constructor super call needs to access uninitialized `this` to call `defaultValue$1`, which results in a `VerifyError` during JVM bytecode verification. **Why previous patches did not work** - #24983 tried to undo lifting in a later phase (`HoistSuperArgs`) - #25157 skips lifting when the owner is a constructor but neither worked well, because both essentially cancel lifting in the specific code shape (when inside a constructor super call). However, cancelling lifting reintroduces the problem fixed by #3839 for those code shapes. **Solution** This commit fixes #24201 without breaking #2939 by removing `LiftToDefs` entirely. Instead, we avoid the duplicated symbol problem by copying by-name parameter arguments with fresh symbols. This removes the constructor-local synthetic-method path entirely and keeps the code as-is. (This is not a problem regarding evaluation order, because `ElimByName` translates `Baz.E1` into the closure `() => Baz.E1`.) ```scala object Baz extends Foo[Baz](Baz.E1, arg2 = 3) ``` The default-getter handling is moved to the point where duplication actually occurs (`Applications.spliceMeth`). When `spliceMeth` rebuilds a default-getter call, it now freshens only by-name arguments. For example, the example code below: ```scala // i7477.scala def spawn(f: => Unit)(naming: Int = 4): Unit = ??? def test(): Unit = spawn { val x: Int = 5 x }() ``` previously, it is transformed into: ```scala def f$1: Unit = { val x: Int = 5 { x; () } } this.spawn(f$1)(this.spawn$default$2(f$1)) ``` now, the same tree is transformed into: ```scala this.spawn({ val x$1: Int = 5 { x$1 () }} )(this.spawn$default$2({ val x$2: Int = 5 { x$2 () } }) ) ``` <!-- Fixes #XYZ (where XYZ is the issue number from the issue tracker) --> <!-- TODO description of the change --> <!-- Ideally should have a title like "Fix #XYZ: Short fix description" --> <!-- TODO first sign the CLA https://contribute.akka.io/cla/scala --> <!-- if the PR is still a WIP, create it as a draft PR (or convert it into one) --> ## How much have your relied on LLM-based tools in this contribution? work with GPT5.4 under supervision ## How was the solution tested? <!-- If automated tests are included, mention it. If they are not, explain why and how the solution was tested. --> ## Additional notes Maybe this could be a code size problem when the by-name parameter arg has a huge body?
This is rework of #24983 this is much simpler FYI @liufengyun @sjrd
Fix #24201
When a super constructor call uses named arguments with default parameters,
LiftToDefslifts the by-name argument to an instance method.For example,
Without this fix, at typer,
LiftToDefsliftsBaz.E1to an instance method:This
defaultValue$1instance method should normally hoisted byHoistSuperArgsto a static function. However, beforeHoistSuperArgs,ElimByNameconvert by-name parameter argument to a closure:new Foo[Baz](() ?=> defaultValue$1, arg1$1, arg2 = 3)As a result,
HoistSuperArgshoists the generated closure,Later,
HoistSuperArgshoists super constructor arguments to a static method, but leaves instance methoddefaultValue$1as it is.As a result, passing
defaultValue$1toBaz$$superArg$1requires accessingthis, before initialization, which causes VerifyError in JVM.So the problem is,
LiftToDefsandElimByNamelifts by-name param arguments twice (to isntance method, and closure). And "hide" the instance method fromHoistSuperArgsby closure.This commit fix the problem by skip lifting by-name parameter args in constructor context. The expression stays inline and
ElimByNamecreates closure withoutthisreference.