Skip to content

Replace custom "holes" AST nodes by extensions to restore binary compatibility#1503

Merged
voodoos merged 4 commits intoocaml:masterfrom
voodoos:holes-in-ppx
Sep 5, 2022
Merged

Replace custom "holes" AST nodes by extensions to restore binary compatibility#1503
voodoos merged 4 commits intoocaml:masterfrom
voodoos:holes-in-ppx

Conversation

@voodoos
Copy link
Copy Markdown
Collaborator

@voodoos voodoos commented Jul 25, 2022

There is currently an issue with Merlin's new typed holes and ppxes.
Since these holes were added as new AST node ppxes crash in there presence.

This PR removes these new parsetree nodes and restore the old extension-based ones.
We still keep the typedtree nodes that improve the typing if these holes.

I added a test with a custom ppx that shows the error. However this requires to have ppxlib as a test-only dependency for the merlin package. @trefis I know you were wary about doing that, but it really looks like the easiest way to test ppxes in the the testsuite, do you see another option ?

cc @lpw25

@voodoos voodoos requested review from let-def and trefis July 25, 2022 13:34
antalsz
antalsz suggested changes Aug 31, 2022
View reviewed changes
Copy link
Copy Markdown
Contributor

@antalsz antalsz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for making this change, it looks great! I went ahead and gave it some review, since we're excited to roll it out internally; I just had a couple small comments.

src/ocaml/typing/untypeast.ml Outdated
Comment on lines +521 to +522
let id = Location.mkloc "merlin.hole" loc in
Pexp_extension (id, PStr [])
Copy link
Copy Markdown
Contributor

@antalsz antalsz Aug 31, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We've imported Ast_helpers, so this can just be Exp.hole (), I believe

Copy link
Copy Markdown
Collaborator Author

@voodoos voodoos Sep 1, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Exp.hole () has type expression but expression_desc is expected here so we cannot use it. It is probably enough to only factor the merlin.hole name out.

src/ocaml/typing/untypeast.ml Outdated
Comment on lines +685 to +686
let id = Location.mkloc "merlin.hole" loc in
Pmod_extension (id, PStr [])
Copy link
Copy Markdown
Contributor

@antalsz antalsz Aug 31, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We've imported Ast_helpers, so this can just be Mod.hole (), I believe

Copy link
Copy Markdown
Collaborator Author

@voodoos voodoos Sep 1, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same.

@voodoos
Add a test illustrating bad behavior of holes with ppxex.
ee08ad9 
- silence the "relocation in read-only section `.text'" warning happening on wome x86_32 systems
@voodoos
Use extension instead of new nodes for holes.
9d186ed 
This restores binary compatibility with external tools like PPX.
@voodoos
Promote fixed PPX test.
3dabe03
@voodoos
Add changelog entry for ocaml#1503
576f3bf
@voodoos voodoos merged commit f34484b into ocaml:master Sep 5, 2022
@voodoos
Copy link
Copy Markdown
Collaborator Author

voodoos commented Sep 5, 2022

@antalsz merged :-)

antalsz added a commit to oxcaml/merlin that referenced this pull request Sep 29, 2022
@antalsz
Backport "Replace custom "holes" AST nodes by extensions to restore b…
291fd67 
…inary compatibility" (ocaml/merlin#1503) by @voodoos
antalsz added a commit to oxcaml/merlin that referenced this pull request Sep 29, 2022
@antalsz
Backport "Replace custom "holes" AST nodes by extensions to restore b…
2d4573a 
…inary compatibility" (ocaml/merlin#1503) by @voodoos
voodoos added a commit to voodoos/merlin that referenced this pull request Nov 23, 2022
@voodoos
Backport ocaml#1503: replace custom "holes" AST nodes by extensions
da17cb8 
from voodoos/holes-in-ppx

Replace custom "holes" AST nodes by extensions to restore binary compatibility
voodoos added a commit that referenced this pull request Nov 24, 2022
@voodoos
Backport #1503: replace custom "holes" AST nodes by extensions
eb37330 
from voodoos/holes-in-ppx

Replace custom "holes" AST nodes by extensions to restore binary compatibility
voodoos added a commit to voodoos/opam-repository that referenced this pull request Nov 24, 2022
@voodoos
[new release] merlin, merlin-lib and dot-merlin-reader (4.7-414)
4c2a6b8 
CHANGES:

Thu Nov 24 13:31:42 CEST 2022

  + merlin binary
    - Replace custom "holes" AST nodes by extensions. This restores binary
      compatibility and fixes issues with PPXs when using typed-holes.
      (ocaml/merlin#1503)
    - Do not change temporarily Merlin's cwd when starting a PPX (ocaml/merlin#1521,
      fixes ocaml/merlin#1420)
    - Fix a parsing issue when declaring the `(??)` custom prefix operator.
      (ocaml/merlin#1507, fixes ocaml/merlin#1506)
    - Fix variant constructors' comments grouping (ocaml/merlin#1516, @mheiber, fixes ocaml/merlin#1513)
    - Filter-out duplicates from the `enclosing` command result (ocaml/merlin#1512)
    - Add a new `verbosity=smart` mode for type enclosing that only expand
      modules' types (ocaml/merlin#1374, @ulugbekna)
    - Improve locate for labels' declarations in the current buffer.
      (ocaml/merlin#1505, fixes ocaml/merlin#1524)
    - Fix locate on module without implementation (ocaml/merlin#1522, fixes ocaml/merlin#1519)
    - Allow program name customization when merlin is used as a library. (ocaml/merlin#1532)
  + editor modes
    - vim: load the plugin when necessary if it wasn't loaded before (ocaml/merlin#1511)
    - emacs: update CI for newer releases and fix some warnings (ocaml/merlin#1454,
      @mattiase)
  + test suite
    - Add tests for constructors' documentation (ocaml/merlin#1511)
    - Add test cases for label comment documentation (ocaml/merlin#1526, @mheiber)
    - Add a test for the `enclosing` command (ocaml/merlin#1512)
    - Add tests for interactions between locate and record labels (ocaml/merlin#1505)
    - Add test showing an issue with locate and implicit transitive deps
voodoos added a commit to voodoos/opam-repository that referenced this pull request Nov 24, 2022
@voodoos
[new release] merlin and dot-merlin-reader (4.7-413)
3060ede 
CHANGES:

Thu Nov 24 13:31:42 CEST 2022

  + merlin binary
    - Replace custom "holes" AST nodes by extensions. This restores binary
      compatibility and fixes issues with PPXs when using typed-holes.
      (ocaml/merlin#1503)
    - Fix a parsing issue when declaring the `(??)` custom prefix operator.
      (ocaml/merlin#1507, fixes ocaml/merlin#1506)
    - Fix variant constructors' comments grouping (ocaml/merlin#1516, @mheiber, fixes ocaml/merlin#1513)
    - Filter-out duplicates from the `enclosing` command result (ocaml/merlin#1512)
  + editor modes
    - vim: load the plugin when necessary if it wasn't loaded before (ocaml/merlin#1511)
  + test suite
    - Add tests for constructors' documentation (ocaml/merlin#1511)
    - Add test cases for label comments documentation (ocaml/merlin#1526, @mheiber)
    - Add a test for the `enclosing` command (ocaml/merlin#1512)
voodoos added a commit to voodoos/opam-repository that referenced this pull request Nov 24, 2022
@voodoos
[new release] merlin and dot-merlin-reader (4.7-412)
1936bf5 
CHANGES:

Thu Nov 24 13:31:42 CEST 2022

  + merlin binary
    - Replace custom "holes" AST nodes by extensions. This restores binary
      compatibility and fixes issues with PPXs when using typed-holes.
      (ocaml/merlin#1503)
    - Fix a parsing issue when declaring the `(??)` custom prefix operator.
      (ocaml/merlin#1507, fixes ocaml/merlin#1506)
    - Fix variant constructors' comments grouping (ocaml/merlin#1516, @mheiber, fixes ocaml/merlin#1513)
    - Filter-out duplicates from the `enclosing` command result (ocaml/merlin#1512)
  + editor modes
    - vim: load the plugin when necessary if it wasn't loaded before (ocaml/merlin#1511)
  + test suite
    - Add tests for constructors' documentation (ocaml/merlin#1511)
    - Add test cases for label comment documentation (ocaml/merlin#1526, @mheiber)
    - Add a test for the `enclosing` command (ocaml/merlin#1512)
voodoos added a commit to voodoos/opam-repository that referenced this pull request Nov 24, 2022
@voodoos
[new release] merlin and merlin-lib (4.7-414)
48727d6 
CHANGES:

  + merlin binary
    - Replace custom "holes" AST nodes by extensions. This restores binary
      compatibility and fixes issues with PPXs when using typed-holes.
      (ocaml/merlin#1503)
    - Do not change temporarily Merlin's cwd when starting a PPX (ocaml/merlin#1521,
      fixes ocaml/merlin#1420)
    - Fix a parsing issue when declaring the `(??)` custom prefix operator.
      (ocaml/merlin#1507, fixes ocaml/merlin#1506)
    - Fix variant constructors' comments grouping (ocaml/merlin#1516, @mheiber, fixes ocaml/merlin#1513)
    - Filter-out duplicates from the `enclosing` command result (ocaml/merlin#1512)
    - Add a new `verbosity=smart` mode for type enclosing that only expand
      modules' types (ocaml/merlin#1374, @ulugbekna)
    - Improve locate for labels' declarations in the current buffer.
      (ocaml/merlin#1505, fixes ocaml/merlin#1524)
    - Fix locate on module without implementation (ocaml/merlin#1522, fixes ocaml/merlin#1519)
    - Allow program name customization when merlin is used as a library. (ocaml/merlin#1532)
  + editor modes
    - vim: load the plugin when necessary if it wasn't loaded before (ocaml/merlin#1511)
    - emacs: update CI for newer releases and fix some warnings (ocaml/merlin#1454,
      @mattiase)
  + test suite
    - Add tests for constructors' documentation (ocaml/merlin#1511)
    - Add test cases for label comment documentation (ocaml/merlin#1526, @mheiber)
    - Add a test for the `enclosing` command (ocaml/merlin#1512)
    - Add tests for interactions between locate and record labels (ocaml/merlin#1505)
    - Add test showing an issue with locate and implicit transitive deps

[new release] merlin (4.7-413)

CHANGES:

  + merlin binary
    - Replace custom "holes" AST nodes by extensions. This restores binary
      compatibility and fixes issues with PPXs when using typed-holes.
      (ocaml/merlin#1503)
    - Fix a parsing issue when declaring the `(??)` custom prefix operator.
      (ocaml/merlin#1507, fixes ocaml/merlin#1506)
    - Fix variant constructors' comments grouping (ocaml/merlin#1516, @mheiber, fixes ocaml/merlin#1513)
    - Filter-out duplicates from the `enclosing` command result (ocaml/merlin#1512)
  + editor modes
    - vim: load the plugin when necessary if it wasn't loaded before (ocaml/merlin#1511)
  + test suite
    - Add tests for constructors' documentation (ocaml/merlin#1511)
    - Add test cases for label comments documentation (ocaml/merlin#1526, @mheiber)
    - Add a test for the `enclosing` command (ocaml/merlin#1512)

[new release] merlin (4.7-412)

CHANGES:

  + merlin binary
    - Replace custom "holes" AST nodes by extensions. This restores binary
      compatibility and fixes issues with PPXs when using typed-holes.
      (ocaml/merlin#1503)
    - Fix a parsing issue when declaring the `(??)` custom prefix operator.
      (ocaml/merlin#1507, fixes ocaml/merlin#1506)
    - Fix variant constructors' comments grouping (ocaml/merlin#1516, @mheiber, fixes ocaml/merlin#1513)
    - Filter-out duplicates from the `enclosing` command result (ocaml/merlin#1512)
  + editor modes
    - vim: load the plugin when necessary if it wasn't loaded before (ocaml/merlin#1511)
  + test suite
    - Add tests for constructors' documentation (ocaml/merlin#1511)
    - Add test cases for label comment documentation (ocaml/merlin#1526, @mheiber)
    - Add a test for the `enclosing` command (ocaml/merlin#1512)
@voodoos voodoos mentioned this pull request Nov 24, 2022
Merged
voodoos added a commit to voodoos/opam-repository that referenced this pull request Nov 24, 2022
@voodoos
[new release] merlin, merlin-lib and dot-merlin-reader (4.7-500)
55c8f20 
CHANGES:

Thu Nov 24 17:49:42 CEST 2022

  + merlin binary
    - Replace custom "holes" AST nodes by extensions. This restores binary
      compatibility and fixes issues with PPXs when using typed-holes.
      (ocaml/merlin#1503)
    - Do not change temporarily Merlin's cwd when starting a PPX (ocaml/merlin#1521,
      fixes ocaml/merlin#1420)
    - Fix a parsing issue when declaring the `(??)` custom prefix operator.
      (ocaml/merlin#1507, fixes ocaml/merlin#1506)
    - Fix variant constructors' comments grouping (ocaml/merlin#1516, @mheiber, fixes ocaml/merlin#1513)
    - Filter-out duplicates from the `enclosing` command result (ocaml/merlin#1512)
    - Add a new `verbosity=smart` mode for type enclosing that only expand
      modules' types (ocaml/merlin#1374, @ulugbekna)
    - Improve locate for labels' declarations in the current buffer.
      (ocaml/merlin#1505, fixes ocaml/merlin#1524)
    - Fix locate on module without implementation (ocaml/merlin#1522, fixes ocaml/merlin#1519)
    - Allow program name customization when merlin is used as a library. (ocaml/merlin#1532)
  + editor modes
    - vim: load the plugin when necessary if it wasn't loaded before (ocaml/merlin#1511)
    - emacs: update CI for newer releases and fix some warnings (ocaml/merlin#1454,
      @mattiase)
  + test suite
    - Add tests for constructors' documentation (ocaml/merlin#1511)
    - Add test cases for label comment documentation (ocaml/merlin#1526, @mheiber)
    - Add a test for the `enclosing` command (ocaml/merlin#1512)
    - Add tests for interactions between locate and record labels (ocaml/merlin#1505)
    - Add test showing an issue with locate and implicit transitive deps
voodoos added a commit to voodoos/opam-repository that referenced this pull request Nov 24, 2022
@voodoos
[new release] merlin and merlin-lib (4.7-500)
8a57619 
CHANGES:

  + merlin binary
    - Replace custom "holes" AST nodes by extensions. This restores binary
      compatibility and fixes issues with PPXs when using typed-holes.
      (ocaml/merlin#1503)
    - Do not change temporarily Merlin's cwd when starting a PPX (ocaml/merlin#1521,
      fixes ocaml/merlin#1420)
    - Fix a parsing issue when declaring the `(??)` custom prefix operator.
      (ocaml/merlin#1507, fixes ocaml/merlin#1506)
    - Fix variant constructors' comments grouping (ocaml/merlin#1516, @mheiber, fixes ocaml/merlin#1513)
    - Filter-out duplicates from the `enclosing` command result (ocaml/merlin#1512)
    - Add a new `verbosity=smart` mode for type enclosing that only expand
      modules' types (ocaml/merlin#1374, @ulugbekna)
    - Improve locate for labels' declarations in the current buffer.
      (ocaml/merlin#1505, fixes ocaml/merlin#1524)
    - Fix locate on module without implementation (ocaml/merlin#1522, fixes ocaml/merlin#1519)
    - Allow program name customization when merlin is used as a library. (ocaml/merlin#1532)
  + editor modes
    - vim: load the plugin when necessary if it wasn't loaded before (ocaml/merlin#1511)
    - emacs: update CI for newer releases and fix some warnings (ocaml/merlin#1454,
      @mattiase)
  + test suite
    - Add tests for constructors' documentation (ocaml/merlin#1511)
    - Add test cases for label comment documentation (ocaml/merlin#1526, @mheiber)
    - Add a test for the `enclosing` command (ocaml/merlin#1512)
    - Add tests for interactions between locate and record labels (ocaml/merlin#1505)
    - Add test showing an issue with locate and implicit transitive deps
@voodoos voodoos mentioned this pull request Nov 24, 2022
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@trefis trefis Awaiting requested review from trefis

@let-def let-def Awaiting requested review from let-def

1 more reviewer

@antalsz antalsz antalsz requested changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@voodoos @antalsz