Skip to content

Fix #2939: Avoid duplicating symbols in default arguments#3839

Merged
odersky merged 4 commits intoscala:masterfrom
dotty-staging:fix-#2939
Feb 2, 2018
Merged

Fix #2939: Avoid duplicating symbols in default arguments#3839
odersky merged 4 commits intoscala:masterfrom
dotty-staging:fix-#2939

Conversation

@odersky
Copy link
Contributor

@odersky odersky commented Jan 15, 2018

Previously a method argument could be duplicated because it was passed
to the method as well as to its default argument methods. This was fatal
if the argument contained local definitions or was a closure.

We now unconditionally lift such arguments if the method has default
parameters.

@odersky
Copy link
Contributor Author

odersky commented Jan 21, 2018

Something fails with the CI.

@smarter
Copy link
Member

smarter commented Jan 21, 2018

I can reproduce the failure locally, the added run test compiles correctly but generates invalid bytecode which is detected at runtime:

> run tests/run/i2939.scala
[info] Running dotty.tools.dotc.Main -classpath /home/smarter/opt/dotty/library/target/scala-2.12/dotty-library_2.12-0.7.0-bin-SNAPSHOT-nonbootstrapped.jar tests/run/i2939.scala
[success] Total time: 4 s, completed 21 janv. 2018 18:57:56
> dotr Test
Exception in thread "main" java.lang.VerifyError: Bad local variable type
Exception Details:
  Location:
    Test$.x$1$1()I @0: aload_1
  Reason:
    Type top (current frame, locals[1]) is not assignable to reference type
  Current Frame:
    bci: @0
    flags: { }
    locals: { 'Test$' }
    stack: { }
  Bytecode:
    0x0000000: 2bbe ac                                

        at Test.main(i2939.scala)

@smarter
Copy link
Member

smarter commented Jan 21, 2018

Actually, compilation also fails with Ycheck:

> run -Ycheck:all tests/run/i2939.scala
exception while typing def x$2: Int = args.length of class class dotty.tools.dotc.ast.Trees$DefDef # 584
...
Exception in thread "main" java.lang.AssertionError: assertion failed: wrong type, expect a method type for Test$._$x$2, but found: Int
        at scala.Predef$.assert(Predef.scala:219)
        at dotty.tools.dotc.transform.TreeChecker$Checker.$anonfun$typedDefDef$2(TreeChecker.scala:405)
        at dotty.tools.dotc.transform.TreeChecker$Checker.withDefinedSyms(TreeChecker.scala:180)
        at dotty.tools.dotc.transform.TreeChecker$Checker.$anonfun$typedDefDef$1(TreeChecker.scala:393)
        at dotty.tools.dotc.transform.TreeChecker$Checker.withDefinedSyms(TreeChecker.scala:180)
        at dotty.tools.dotc.transform.TreeChecker$Checker.typedDefDef(TreeChecker.scala:392)
        at dotty.tools.dotc.typer.Typer.typedNamed$1(Typer.scala:1708)
...

@odersky odersky assigned smarter and unassigned odersky Jan 21, 2018
Previously a method argument could be duplicated because it was passed
to the method as well as to its default argument methods. This was fatal
if the argument contained local definitions or was a closure.

We now unconditionally lift such arguments if the method has default
parameters.
@@ -345,7 +346,7 @@ trait TypedTreeInfo extends TreeInfo[Type] { self: Trees.Instance[Type] =>
| Super(_, _)
| Literal(_)
| Closure(_, _, _) =>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How come a Closure is considered a PurePath? It can contain local definitions.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah no it doesn't, nevermind.

Copy link
Member

@smarter smarter left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

/** Test indicating `expr` does not need lifting */
def noLift(expr: Tree)(implicit ctx: Context): Boolean

/** The corresponding lifter for paam-by-name arguments */
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

typo: paam -> param ?

* @return pure if expression has no side effects
* idempotent if running the expression a second time has no side effects
* impure otherwise
* @return PurePath if expression has no side effects and cannot contain local definitions
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if PurePath is the best name for this since not every PurePath expression is a valid path. Maybe NoDefPure? Or SimplePure/ComplexPure considering that later on we talk about "complex expressions" ?

@smarter smarter assigned odersky and unassigned smarter Feb 2, 2018
@odersky odersky merged commit 50f9cfa into scala:master Feb 2, 2018
tanishiking added a commit to tanishiking/scala3 that referenced this pull request Mar 12, 2026
…esh syms

fix scala#24201
This change also fixes scala#18123 (not fully understand why)

**Background**
`LiftToDefs` lifts by-name arguments to synthetic `def`s so that a
method argument would not be duplicated both as the real argument and as an argument to a default getter.
see scala#2939

A transformation for by-name semantics is done later in `ElimByName`, which turns a by-name actual argument into a closure.

**Problem**

However, lifting by-name arguments can be problematic in a specific scenario. scala#24201

For example, in `extends Foo[Baz](Baz.E1, arg2 = 3)`:

```scala
abstract class Foo[T](value: => T, arg1: Int = 1, arg2: Int = 2)
enum Baz { case E1 }
object Baz extends Foo[Baz](Baz.E1, arg2 = 3)
```

`LiftToDefs` lifts the by-name argument `Baz.E1` to the instance method like:

```scala
object Baz extends {
  def defaultValue$1: Baz = Baz.E1   // instance method
  val arg1$1: Int = Foo.$lessinit$greater$default$2[Baz]
  new Foo[Baz](defaultValue$1, arg1$1, arg2 = 3)
}
```

where `defaultValue$1` is a synthesized instance method owned by the constructor. This is not valid code, because the constructor super call needs to access uninitialized `this` to call `defaultValue$1`, which results in a `VerifyError` during JVM bytecode verification.

**Why previous patches did not work**

- scala#24983 tried to undo lifting in a
  later phase (`HoistSuperArgs`)
- scala#25157 skips lifting when the
  owner is a constructor

but neither worked well, because both essentially cancel lifting in the specific code shape (when inside a constructor super call). However, cancelling lifting reintroduces the problem fixed by scala#3839 for those code shapes.

**Solution**

This commit fixes scala#24201 without breaking scala#2939 by removing `LiftToDefs` entirely. Instead, we avoid the duplicated symbol problem by copying by-name parameter arguments with fresh symbols.

This removes the constructor-local synthetic-method path entirely and keeps the code as-is. (This is not a problem regarding evaluation order,
because `ElimByName` translates `Baz.E1` into the closure `() => Baz.E1`.)

```scala
object Baz extends Foo[Baz](Baz.E1, arg2 = 3)
```

The default-getter handling is moved to the point where duplication actually occurs (`Applications.spliceMeth`). When `spliceMeth` rebuilds a default-getter call, it now freshens only by-name arguments.

For example, the example code below:

```scala
// i7477.scala
def spawn(f: => Unit)(naming: Int = 4): Unit = ???
def test(): Unit = spawn {
  val x: Int = 5
  x
}()
```

previously, it is transformed into:

```scala
def f$1: Unit = {
  val x: Int = 5
  { x; () }
}
this.spawn(f$1)(this.spawn$default$2(f$1))
```

now, the same tree is transformed into:

```scala
this.spawn({
  val x$1: Int = 5
  {
    x$1
    ()
  }}
)(this.spawn$default$2({
  val x$2: Int = 5
  {
    x$2
    ()
  }
  })
)
```
tanishiking added a commit to tanishiking/scala3 that referenced this pull request Mar 12, 2026
…esh syms

fix scala#24201
This change also fixes scala#18123 (not fully understand why)

**Background**
`LiftToDefs` lifts by-name arguments to synthetic `def`s so that a
method argument would not be duplicated both as the real argument and as an argument to a default getter.
see scala#2939

A transformation for by-name semantics is done later in `ElimByName`, which turns a by-name actual argument into a closure.

**Problem**

However, lifting by-name arguments can be problematic in a specific scenario. scala#24201

For example, in `extends Foo[Baz](Baz.E1, arg2 = 3)`:

```scala
abstract class Foo[T](value: => T, arg1: Int = 1, arg2: Int = 2)
enum Baz { case E1 }
object Baz extends Foo[Baz](Baz.E1, arg2 = 3)
```

`LiftToDefs` lifts the by-name argument `Baz.E1` to the instance method like:

```scala
object Baz extends {
  def defaultValue$1: Baz = Baz.E1   // instance method
  val arg1$1: Int = Foo.$lessinit$greater$default$2[Baz]
  new Foo[Baz](defaultValue$1, arg1$1, arg2 = 3)
}
```

where `defaultValue$1` is a synthesized instance method owned by the constructor. This is not valid code, because the constructor super call needs to access uninitialized `this` to call `defaultValue$1`, which results in a `VerifyError` during JVM bytecode verification.

**Why previous patches did not work**

- scala#24983 tried to undo lifting in a
  later phase (`HoistSuperArgs`)
- scala#25157 skips lifting when the
  owner is a constructor

but neither worked well, because both essentially cancel lifting in the specific code shape (when inside a constructor super call). However, cancelling lifting reintroduces the problem fixed by scala#3839 for those code shapes.

**Solution**

This commit fixes scala#24201 without breaking scala#2939 by removing `LiftToDefs` entirely. Instead, we avoid the duplicated symbol problem by copying by-name parameter arguments with fresh symbols.

This removes the constructor-local synthetic-method path entirely and keeps the code as-is. (This is not a problem regarding evaluation order,
because `ElimByName` translates `Baz.E1` into the closure `() => Baz.E1`.)

```scala
object Baz extends Foo[Baz](Baz.E1, arg2 = 3)
```

The default-getter handling is moved to the point where duplication actually occurs (`Applications.spliceMeth`). When `spliceMeth` rebuilds a default-getter call, it now freshens only by-name arguments.

For example, the example code below:

```scala
// i7477.scala
def spawn(f: => Unit)(naming: Int = 4): Unit = ???
def test(): Unit = spawn {
  val x: Int = 5
  x
}()
```

previously, it is transformed into:

```scala
def f$1: Unit = {
  val x: Int = 5
  { x; () }
}
this.spawn(f$1)(this.spawn$default$2(f$1))
```

now, the same tree is transformed into:

```scala
this.spawn({
  val x$1: Int = 5
  {
    x$1
    ()
  }}
)(this.spawn$default$2({
  val x$2: Int = 5
  {
    x$2
    ()
  }
  })
)
```
tanishiking added a commit to tanishiking/scala3 that referenced this pull request Mar 12, 2026
…esh syms

fix scala#24201
This change also fixes scala#18123 (not fully understand why)

**Background**
`LiftToDefs` lifts by-name arguments to synthetic `def`s so that a
method argument would not be duplicated both as the real argument and as an argument to a default getter.
see scala#2939

A transformation for by-name semantics is done later in `ElimByName`, which turns a by-name actual argument into a closure.

**Problem**

However, lifting by-name arguments can be problematic in a specific scenario. scala#24201

For example, in `extends Foo[Baz](Baz.E1, arg2 = 3)`:

```scala
abstract class Foo[T](value: => T, arg1: Int = 1, arg2: Int = 2)
enum Baz { case E1 }
object Baz extends Foo[Baz](Baz.E1, arg2 = 3)
```

`LiftToDefs` lifts the by-name argument `Baz.E1` to the instance method like:

```scala
object Baz extends {
  def defaultValue$1: Baz = Baz.E1   // instance method
  val arg1$1: Int = Foo.$lessinit$greater$default$2[Baz]
  new Foo[Baz](defaultValue$1, arg1$1, arg2 = 3)
}
```

where `defaultValue$1` is a synthesized instance method owned by the constructor. This is not valid code, because the constructor super call needs to access uninitialized `this` to call `defaultValue$1`, which results in a `VerifyError` during JVM bytecode verification.

**Why previous patches did not work**

- scala#24983 tried to undo lifting in a
  later phase (`HoistSuperArgs`)
- scala#25157 skips lifting when the
  owner is a constructor

but neither worked well, because both essentially cancel lifting in the specific code shape (when inside a constructor super call). However, cancelling lifting reintroduces the problem fixed by scala#3839 for those code shapes.

**Solution**

This commit fixes scala#24201 without breaking scala#2939 by removing `LiftToDefs` entirely. Instead, we avoid the duplicated symbol problem by copying by-name parameter arguments with fresh symbols.

This removes the constructor-local synthetic-method path entirely and keeps the code as-is. (This is not a problem regarding evaluation order,
because `ElimByName` translates `Baz.E1` into the closure `() => Baz.E1`.)

```scala
object Baz extends Foo[Baz](Baz.E1, arg2 = 3)
```

The default-getter handling is moved to the point where duplication actually occurs (`Applications.spliceMeth`). When `spliceMeth` rebuilds a default-getter call, it now freshens only by-name arguments.

For example, the example code below:

```scala
// i7477.scala
def spawn(f: => Unit)(naming: Int = 4): Unit = ???
def test(): Unit = spawn {
  val x: Int = 5
  x
}()
```

previously, it is transformed into:

```scala
def f$1: Unit = {
  val x: Int = 5
  { x; () }
}
this.spawn(f$1)(this.spawn$default$2(f$1))
```

now, the same tree is transformed into:

```scala
this.spawn({
  val x$1: Int = 5
  {
    x$1
    ()
  }}
)(this.spawn$default$2({
  val x$2: Int = 5
  {
    x$2
    ()
  }
  })
)
```
odersky added a commit that referenced this pull request Mar 17, 2026
…esh syms (#25502)

fix #24201
This change also fixes #18123 (not fully understand why)

**Background**
`LiftToDefs` lifts by-name arguments to synthetic `def`s so that a
method argument would not be duplicated both as the real argument and as
an argument to a default getter. see
#2939

A transformation for by-name semantics is done later in `ElimByName`,
which turns a by-name actual argument into a closure.

**Problem**

However, lifting by-name arguments can be problematic in a specific
scenario. #24201

For example, in `extends Foo[Baz](Baz.E1, arg2 = 3)`:

```scala
abstract class Foo[T](value: => T, arg1: Int = 1, arg2: Int = 2)
enum Baz { case E1 }
object Baz extends Foo[Baz](Baz.E1, arg2 = 3)
```

`LiftToDefs` lifts the by-name argument `Baz.E1` to the instance method
like:

```scala
object Baz extends {
  def defaultValue$1: Baz = Baz.E1   // instance method
  val arg1$1: Int = Foo.$lessinit$greater$default$2[Baz]
  new Foo[Baz](defaultValue$1, arg1$1, arg2 = 3)
}
```

where `defaultValue$1` is a synthesized instance method owned by the
constructor. This is not valid code, because the constructor super call
needs to access uninitialized `this` to call `defaultValue$1`, which
results in a `VerifyError` during JVM bytecode verification.

**Why previous patches did not work**

- #24983 tried to undo lifting in a
later phase (`HoistSuperArgs`)
- #25157 skips lifting when the
owner is a constructor

but neither worked well, because both essentially cancel lifting in the
specific code shape (when inside a constructor super call). However,
cancelling lifting reintroduces the problem fixed by
#3839 for those code shapes.

**Solution**

This commit fixes #24201 without breaking #2939 by removing `LiftToDefs`
entirely. Instead, we avoid the duplicated symbol problem by copying
by-name parameter arguments with fresh symbols.

This removes the constructor-local synthetic-method path entirely and
keeps the code as-is. (This is not a problem regarding evaluation order,
because `ElimByName` translates `Baz.E1` into the closure `() =>
Baz.E1`.)

```scala
object Baz extends Foo[Baz](Baz.E1, arg2 = 3)
```

The default-getter handling is moved to the point where duplication
actually occurs (`Applications.spliceMeth`). When `spliceMeth` rebuilds
a default-getter call, it now freshens only by-name arguments.

For example, the example code below:

```scala
// i7477.scala
def spawn(f: => Unit)(naming: Int = 4): Unit = ???
def test(): Unit = spawn {
  val x: Int = 5
  x
}()
```

previously, it is transformed into:

```scala
def f$1: Unit = {
  val x: Int = 5
  { x; () }
}
this.spawn(f$1)(this.spawn$default$2(f$1))
```

now, the same tree is transformed into:

```scala
this.spawn({
  val x$1: Int = 5
  {
    x$1
    ()
  }}
)(this.spawn$default$2({
  val x$2: Int = 5
  {
    x$2
    ()
  }
  })
)
```

<!-- Fixes #XYZ (where XYZ is the issue number from the issue tracker)
-->

<!-- TODO description of the change -->
<!-- Ideally should have a title like "Fix #XYZ: Short fix description"
-->

<!--
  TODO first sign the CLA
  https://contribute.akka.io/cla/scala
-->

<!-- if the PR is still a WIP, create it as a draft PR (or convert it
into one) -->

## How much have your relied on LLM-based tools in this contribution?

work with GPT5.4 under supervision

## How was the solution tested?

<!-- 
  If automated tests are included, mention it.
  If they are not, explain why and how the solution was tested.
-->

## Additional notes

Maybe this could be a code size problem when the by-name parameter arg
has a huge body?
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants