Skip to content

Special treatment of emptyValDef in reify#3

Merged
paulp merged 1 commit intoscala:masterfrom
odersky:topic/reify
Dec 1, 2011
Merged

Special treatment of emptyValDef in reify#3
paulp merged 1 commit intoscala:masterfrom
odersky:topic/reify

Conversation

@odersky
Copy link
Contributor

@odersky odersky commented Dec 1, 2011

emptyValDef has special meaning in the compiler, so reify needs to preserve it by identity and not just by structure. Review by @xeno-by

emptyValDef has special meaning in the compiler, so reify needs to preserve it by identity and not just by structure.
@paulp paulp merged commit b683b83 into scala:master Dec 1, 2011
gkossakowski referenced this pull request in gkossakowski/scala Jan 11, 2012
Merge-in forward jumps fixes, turned on specialization and few other things.
axel22 referenced this pull request in axel22/scala-github Feb 3, 2012
This change resolves some issues with ParCtrie splitters and their
`remaining` method, which currently evaluates the size of the Ctrie.
Since this is still not done lazily, nor in parallel, it has a certain cost,
which is unacceptable.

Change #1: The `shouldSplitFurther` method is by default implemented by
calling the `remaining` method. This method now forwards the call to the
same method in the splitter which is by default implemented in the same
way as before, but can be overridden by custom collections such as the
ParCtrie.

Change #2: ParCtrie splitter now has a `level` member which just counts
how many times the method has been split. This information is used to
override the default `shouldSplitFurther` implementation.

Change #3: The tasks and splitters rely heavily on the `remaining` method
in the splitter for most operations. There is an additional method called
`isRemainingCheap` which returns true by default, but can be overridden
by custom collections such as the `Ctrie`.
paulp added a commit that referenced this pull request Feb 14, 2012
Now it copies in the current versions of BoxesRunTime and ScalaRunTime
and applies patches to them, and the whole build is automated.

  # This is the only thing I actually typed, the rest is fancy echo.
  $ test/instrumented/mkinstrumented.sh build

  % rm -rf /scratch/trunk1/test/instrumented/classes
  % cp /scratch/trunk1/test/instrumented/../../src/library/scala/runtime/BoxesRunTime.java /scratch/trunk1/test/instrumented/../../src/library/scala/runtime/ScalaRunTime.scala /scratch/trunk1/test/instrumented/library/scala/runtime
  % patch BoxesRunTime.java /scratch/trunk1/test/instrumented/boxes.patch
  patching file BoxesRunTime.java
  % patch ScalaRunTime.scala /scratch/trunk1/test/instrumented/srt.patch
  patching file ScalaRunTime.scala
  Hunk #3 succeeded at 63 (offset 23 lines).
  Hunk #4 succeeded at 78 (offset 23 lines).
  Hunk #5 succeeded at 81 (offset 23 lines).
  Hunk #6 succeeded at 96 (offset 23 lines).
  % /scratch/trunk1/test/instrumented/../../build/pack/bin/scalac -d /scratch/trunk1/test/instrumented/classes /scratch/trunk1/test/instrumented/library/scala/runtime/BoxesRunTime.java /scratch/trunk1/test/instrumented/library/scala/runtime/ScalaRunTime.scala
  % javac -cp /scratch/trunk1/test/instrumented/../../build/pack/lib/scala-library.jar -d /scratch/trunk1/test/instrumented/classes /scratch/trunk1/test/instrumented/library/scala/runtime/BoxesRunTime.java
  % cd /scratch/trunk1/test/instrumented/classes
  % jar cf instrumented.jar .
  % mv -f instrumented.jar /scratch/trunk1/test/instrumented/../../test/files/speclib
  /scratch/trunk1/test/files/speclib/instrumented.jar has been created.
odersky added a commit to odersky/scala that referenced this pull request Jul 14, 2012
Fixed fingerPrinting scheme to work with rehashes, also added finger prints to typedIdent searches.
adriaanm referenced this pull request in adriaanm/scala Jul 14, 2012
Fixed fingerPrinting scheme to work with rehashes, also added finger prints to typedIdent searches.
xeno-by added a commit that referenced this pull request Feb 27, 2013
First of all, GIL should only apply to runtime reflection, because noone
is going to run toolboxes in multiple threads: a) that's impossible, b/c
the compiler isn't thread safe, b) ToolBox api prevents that.

Secondly, the only things in symbols which require synchronization are:
1) info/validTo (completers aren't thread-safe),
2) rawInfo and its dependencies (it shares a mutable field with info)
3) non-trivial caches like in typeAsMemberOfLock

If you think about it, other things like sourceModule or associatedFile
don't need synchronization, because they are either set up when a symbol
is created or cloned or when it's completed. The former is obviously safe,
while the latter is safe as well, because before acquiring init-dependent
state of symbols, the compiler calls `initialize`, which is synchronized.

We can say that symbols can be in four possible states: 1) being created,
2) created, but not yet initialized, 3) initializing, 4) initialized.
in runtime reflection can undergo is init. #3 is dangerous and needs protection
xeno-by added a commit to xeno-by/scala that referenced this pull request Jul 25, 2013
First of all, GIL should only apply to runtime reflection, because noone
is going to run toolboxes in multiple threads: a) that's impossible, b/c
the compiler isn't thread safe, b) ToolBox api prevents that.

Secondly, the only things in symbols which require synchronization are:
1) info/validTo (completers aren't thread-safe),
2) rawInfo and its dependencies (it shares a mutable field with info)
3) non-trivial caches like in typeAsMemberOfLock

If you think about it, other things like sourceModule or associatedFile
don't need synchronization, because they are either set up when a symbol
is created or cloned or when it's completed. The former is obviously safe,
while the latter is safe as well, because before acquiring init-dependent
state of symbols, the compiler calls `initialize`, which is synchronized.

We can say that symbols can be in four possible states: 1) being created,
2) created, but not yet initialized, 3) initializing, 4) initialized.
in runtime reflection can undergo is init. scala#3 is dangerous and needs protection
xeno-by added a commit to xeno-by/scala that referenced this pull request Aug 6, 2013
First of all, GIL should only apply to runtime reflection, because noone
is going to run toolboxes in multiple threads: a) that's impossible, b/c
the compiler isn't thread safe, b) ToolBox api prevents that.

Secondly, the only things in symbols which require synchronization are:
1) info/validTo (completers aren't thread-safe),
2) rawInfo and its dependencies (it shares a mutable field with info)
3) non-trivial caches like in typeAsMemberOfLock

If you think about it, other things like sourceModule or associatedFile
don't need synchronization, because they are either set up when a symbol
is created or cloned or when it's completed. The former is obviously safe,
while the latter is safe as well, because before acquiring init-dependent
state of symbols, the compiler calls `initialize`, which is synchronized.

We can say that symbols can be in four possible states: 1) being created,
2) created, but not yet initialized, 3) initializing, 4) initialized.
in runtime reflection can undergo is init. scala#3 is dangerous and needs protection
xeno-by added a commit to xeno-by/scala that referenced this pull request Aug 9, 2013
First of all, GIL should only apply to runtime reflection, because noone
is going to run toolboxes in multiple threads: a) that's impossible, b/c
the compiler isn't thread safe, b) ToolBox api prevents that.

Secondly, the only things in symbols which require synchronization are:
1) info/validTo (completers aren't thread-safe),
2) rawInfo and its dependencies (it shares a mutable field with info)
3) non-trivial caches like in typeAsMemberOfLock

If you think about it, other things like sourceModule or associatedFile
don't need synchronization, because they are either set up when a symbol
is created or cloned or when it's completed. The former is obviously safe,
while the latter is safe as well, because before acquiring init-dependent
state of symbols, the compiler calls `initialize`, which is synchronized.

We can say that symbols can be in four possible states: 1) being created,
2) created, but not yet initialized, 3) initializing, 4) initialized.
Of those only scala#3 is dangerous and needs protection, which is what this
commit does.
xeno-by added a commit to xeno-by/scala that referenced this pull request Oct 18, 2013
First of all, GIL should only apply to runtime reflection, because noone
is going to run toolboxes in multiple threads: a) that's impossible, b/c
the compiler isn't thread safe, b) ToolBox api prevents that.

Secondly, the only things in symbols which require synchronization are:
1) info/validTo (completers aren't thread-safe),
2) rawInfo and its dependencies (it shares a mutable field with info)
3) non-trivial caches like in typeAsMemberOfLock

If you think about it, other things like sourceModule or associatedFile
don't need synchronization, because they are either set up when a symbol
is created or cloned or when it's completed. The former is obviously safe,
while the latter is safe as well, because before acquiring init-dependent
state of symbols, the compiler calls `initialize`, which is synchronized.

We can say that symbols can be in four possible states: 1) being created,
2) created, but not yet initialized, 3) initializing, 4) initialized.
Of those only scala#3 is dangerous and needs protection, which is what this
commit does.
xeno-by added a commit to xeno-by/scala that referenced this pull request Nov 12, 2013
When an application of a blackbox macro is used as an implicit candidate,
no expansion is performed until the macro is selected as the result of
the implicit search.

This makes it impossible to dynamically calculate availability of
implicit macros.
xeno-by added a commit that referenced this pull request Nov 13, 2013
When an application of a blackbox macro is used as an implicit candidate,
no expansion is performed until the macro is selected as the result of
the implicit search.

This makes it impossible to dynamically calculate availability of
implicit macros.
retronym added a commit that referenced this pull request Dec 12, 2013
Introduce Unliftable for Quasiquotes (take #3)
retronym added a commit that referenced this pull request Feb 12, 2014
Swathes of important logic are duplicated between `findMember`
and `findMembers` after they separated on grounds of irreconcilable
differences about how fast they should run:

    d905558 Variation #10 to optimze findMember
    fcb0c01 Attempt #9 to opimize findMember.
    71d2ceb Attempt #8 to opimize findMember.
    77e5692 Attempty #7 to optimize findMember
    275115e Fixing problem that caused fingerprints to fail in
    e94252e Attemmpt #6 to optimize findMember
    73e61b8 Attempt #5 to optimize findMember.
    04f0b65 Attempt #4 to optimize findMember
    0e3c70f Attempt #3 to optimize findMember
    41f4497 Attempt #2 to optimize findMember
    1a73aa0 Attempt #1 to optimize findMember

This didn't actually bear fruit, and the intervening years have
seen the implementations drift.

Now is the time to reunite them under the banner of `FindMemberBase`.

Each has a separate subclass to customise the behaviour. This is
primarily used by `findMember` to cache member types and to assemble
the resulting list of symbols in an low-allocation manner.

While there I have introduced some polymorphic calls, the call sites
are only bi-morphic, and our typical pattern of compilation involves
far more `findMember` calls, so I expect that JIT will keep the
virtual call cost to an absolute minimum.

Test results have been updated now that `findMembers` correctly
excludes constructors and doesn't inherit privates.

Coming up next: we can actually fix SI-7475!
retronym added a commit that referenced this pull request Apr 10, 2015
Under `-Ydelambdafy:method`, a public, static accessor method is
created to expose the private method containing the body of the
lambda.

Currently this accessor method has its parameters in the same order
structure as those of the lambda body method.

What is this order? There are three categories of parameters:

  1. lambda parameters
  2. captured parameters (added by lambdalift)
  3. self parameters (added to lambda bodies that end up in trait
     impl classes by mixin, and added unconditionally to the static
     accessor method.)

These are currently emitted in order #3, #1, #2.

Here are examples of the current behaviour:

BEFORE (trait):

```
% cat sandbox/test.scala && scalac-hash v2.11.5 -Ydelambdafy:method sandbox/test.scala && javap -private -classpath . 'Test$class'
trait Member; class Capture; trait LambdaParam
trait Test {
  def member: Member
  def foo {
    val local = new Capture
    (arg: LambdaParam) => "" + arg + member + local
  }
}
Compiled from "test.scala"
public abstract class Test$class {
  public static void foo(Test);
  private static final java.lang.String $anonfun$1(Test, LambdaParam, Capture);
  public static void $init$(Test);
  public static final java.lang.String accessor$1(Test, LambdaParam, Capture);
}
```

BEFORE (class):

```
% cat sandbox/test.scala && scalac-hash v2.11.5 -Ydelambdafy:method sandbox/test.scala && javap -private -classpath . Test
trait Member; class Capture; trait LambdaParam
abstract class Test {
  def member: Member
  def foo {
    val local = new Capture
    (arg: LambdaParam) => "" + arg + member + local
  }
}
Compiled from "test.scala"
public abstract class Test {
  public abstract Member member();
  public void foo();
  private final java.lang.String $anonfun$1(LambdaParam, Capture);
  public Test();
  public static final java.lang.String accessor$1(Test, LambdaParam, Capture);
}
```

Contrasting the class case with Java:

```
% cat sandbox/Test.java && javac -d . sandbox/Test.java && javap -private -classpath . Test
public abstract class Test {
  public static class Member {};
  public static class Capture {};
  public static class LambaParam {};
  public static interface I {
    public abstract Object c(LambaParam arg);
  }
  public abstract Member member();
  public void test() {
    Capture local = new Capture();
    I i1 = (LambaParam arg) -> "" + member() + local;
  }
}

Compiled from "Test.java"
public abstract class Test {
  public Test();
  public abstract Test$Member member();
  public void test();
  private java.lang.Object lambda$test$0(Test$Capture, Test$LambaParam);
}
```

We can see that in Java 8 lambda parameters come after captures. If we
want to use Java's LambdaMetafactory to spin up our anoymous FunctionN
subclasses on the fly, our ordering must change.

I can see three options for change:

  1. Adjust `LambdaLift` to always prepend captured parameters,
     rather than appending them. I think we could leave `Mixin` as
     it is, it already prepends the self parameter. This would result
     a parameter ordering, in terms of the list above: #3, #2, #1.
  2. More conservatively, do this just for methods known to hold
     lambda bodies. This might avoid needlessly breaking code that
     has come to depend on our binary encoding.
  3. Adjust the parameters of the accessor method only. The body
     of this method can permute params before calling the lambda
     body method.

This commit implements option #2.

In also prototyped #1, and found it worked so long as I limited it to
non-constructors, to sidestep the need to make corresponding
changes elsewhere in the compiler to avoid the crasher shown
in the enclosed test case, which was minimized from a bootstrap
failure from an earlier a version of this patch.

We would need to defer option #1 to 2.12 in any case, as some of
these lifted methods are publicied by the optimizer, and we must
leave the signatures alone to comply with MiMa.

I've included a test that shows this in all in action. However, that
is currently disabled, as we don't have a partest category for tests
that require Java 8.
@gkossakowski gkossakowski mentioned this pull request Apr 16, 2015
16 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants