Skip to content

Allow local form of arbitrary structure items#13835

Closed
nojb wants to merge 15 commits intoocaml:trunkfrom
nojb:let_str_item
Closed

Allow local form of arbitrary structure items#13835
nojb wants to merge 15 commits intoocaml:trunkfrom
nojb:let_str_item

Conversation

@nojb
Copy link
Copy Markdown
Contributor

@nojb nojb commented Feb 27, 2025

Currently OCaml supports local forms (= scoped to an expression) of a few specific structure items: let exception, let open and let module.

However, other structure items that could be useful are not available "locally": let type t =, let type t +=, let class, let module type, etc.

The perspicacious reader will tell me that in fact, this is not quite true and that one can use let open struct to extend the range of items that can be bound locally. For example: let open struct type t = T end in e will declare the type t = T locally in e.

However, the current state of things has a number of downsides:

  • The logic (whether parsing, type checking, or code generation) for let module, let open, let exception is duplicated in various ways, using their own dedicated AST nodes, between the code that deals with structure items in their "global" and "local" forms.
  • The let open struct construct involves the allocation of an extra block (corresponding to the struct) and is a bit heavy syntactically.
  • There is no justification for why let exception exists, but let type t += does not, given that they are essentially the same concept.

The one justification I can think for the last point of is that adding a new local form for an existing structure item is a heavy undertaking (need to modify the compiler in a few dozen places). Instead, this PR proposes to uniformize and generalize the treatment of local forms of structure items, so that this handling is done once (in this PR), and in the future, any new structure item will automatically be supported "locally", without needing any extra implementation effort. More precisely:

  • A new kind of expression is introduced in the abstract syntax: let structure-item in expr. This generalizes the existing let open, let exception and let module. (First four commits, up to and including Add Pexp_stritem, Texp_stritem.) Note that this only involves the middle- and back-end (typechecking and code generation).
  • Reimplement the compiler support for let open, let exception and let module in terms of this one construct. These commits only touch the parser, but indirectly they are switching the typechecking and code generation logic for these three constructs to a shared one for all structure items. (Three commits: Reimplement ....)

At this point, we have neither gained nor lost anything as far as the surface language is concerned. However, we have gained in the implementation level: the old AST nodes Pexp_letmodule, Pexp_open and Pexp_letexception (and their typed avatars) have become dead code, replaced by a single node (Pexp_stritem/Texp_stritem). More importantly, the typechecking and code generation for this one construction is fully shared with that used for ordinary structure items. The next step is to expose this abstract construct in the surface language.

  • A new rule is added for the expression parser: LET structure_item IN seq_expr, and the ad-hoc ones for let exception, let open and let module are removed. Actually, something a bit more complex is done to disallow certain forms (see below), as well as to account for the different way attributes are handled in local forms of structure items1.

After this step (commits parser: propagate ... and Allow local forms ...), the surface language is endowded with the following local forms of structure items:

The first three were already supported:

  • let open ... in expr
  • let exception ... in expr
  • let module ... in expr

The following ones are newly supported:

  • let type t = ... in expr
  • let type t += ... in expr
  • let class c = ... in expr
  • let class type c = ... in expr
  • let module type T = ... in expr
  • let external f : .... in expr
  • let [@@@foo] in expr

The following ones are disallowed:

  • let let p = ... in expr
  • let include ... in expr

The first one is disallowed for usability reasons and the second one because I could not think of any use for it (one would use let open instead). Finally,

  • We remove all the (now dead) code related to the implementation of the ad-hoc let module, let open and let exception. (Last commit: Remove reimplemented ...)

That's it. To summarize: the PR proposes to treat the derivation from structure item to "local structure item" in a systematic way, both at the parser level, AST level, typechecking and code generation, so that any new structure item will be automatically available in a local form. Along the way, we expose in the surface language local forms of existing structure items which were not available today (at least without passing via the let open struct construct).

All opinions warmly welcome! I don't think the review of the PR is very difficult (each individual commit is rather small and self-contained), but we will need to agree first on whether we want to move in this direction (or not).

cc @yallop, whose comment in https://discuss.ocaml.org/t/seq-of-iter-a-contained-use-case-for-effects/16187/5?u=nojb spurred me to look at this again.

What is missing?

  • Update the manual
  • More tests (there are a few ones already committed, but I would add more if we move forward with this PR)

What needs special attention

  • The changes to typecore.ml dealing with the typechecking for the new construct let structure-item in expr. There is some logic to make sure that the type of expr makes sense outside of the scope of the structure-item. However, I do not have a good understanding of it, so an expert will need to take a look and explain it to me :)
  • The changes to translcore.ml dealing with the code-generation for the new construct. I am fairly confident about this part, but I would appreciate an expert pair of eyes here as well.
  • One test shows that the "shape indexer" seems to detect more stuff than before (namely, local modules). This looks like a bug fix to me, but confirmation would be nice (cc @voodoos).

Potential downsides

This PR will involve a fair amount of churn on the ppxlib side and/or compiler-libs users, but it should be fairly manageable, I think (especially because these local forms are infrequently used).

Footnotes

  1. In let exception[@foo] E in expr, the attribute foo is attached to the whole let exception E in expr and not only the exception E structure item.

@nojb nojb added the parsetree-change Track changes to the parsetree for that affects ppxs label Feb 27, 2025
@gasche
Copy link
Copy Markdown
Member

gasche commented Feb 28, 2025

The first part, where you reimplement existing constructions using a new generic construction, sounds like unilaterally good to have, it is easier to review, and it does not require design-consensus-building because it does not change the language. Have you considered sending it as a prefix PR to get the ball rolling?

@nojb
Copy link
Copy Markdown
Contributor Author

nojb commented Feb 28, 2025

The first part, where you reimplement existing constructions using a new generic construction, sounds like unilaterally good to have, it is easier to review, and it does not require design-consensus-building because it does not change the language. Have you considered sending it as a prefix PR to get the ball rolling?

Good idea. I will split that part into its own PR. Thanks!

@nojb
Copy link
Copy Markdown
Contributor Author

nojb commented Feb 28, 2025

The first part, where you reimplement existing constructions using a new generic construction, sounds like unilaterally good to have, it is easier to review, and it does not require design-consensus-building because it does not change the language. Have you considered sending it as a prefix PR to get the ball rolling?

Good idea. I will split that part into its own PR. Thanks!

Done in #13839

@nojb
Copy link
Copy Markdown
Contributor Author

nojb commented Mar 1, 2025

I made a note to discuss the language change during the next caml-devel meeting.

@stedolan
Copy link
Copy Markdown
Contributor

This sounds useful! I have wished for let type t = ... in expr in the past, and been unhappy with settling for let open struct type t = ... end in expr.

What is the intended semantics of let [@@@foo] in expr? Most uses of such attributes have some effect that applies to the rest of the structure that contains them, but it's unclear to me whether you mean to include expr in that. Concretely, would you expect an inexhaustive match warning to be raised by the following code?

let [@@@ocaml.warning "-a"] in
let Some () = Some () in
()

@nojb
Copy link
Copy Markdown
Contributor Author

nojb commented Mar 10, 2025

What is the intended semantics of let [@@@foo] in expr? Most uses of such attributes have some effect that applies to the rest of the structure that contains them, but it's unclear to me whether you mean to include expr in that.

Yes, I didn't mention it explicitly in the description, but the semantics I had in mind was that the scope of the attributes would extend over expr.

Concretely, would you expect an inexhaustive match warning to be raised by the following code?

Accordingly, I would expect the warning not to be raised here.

@nojb
Copy link
Copy Markdown
Contributor Author

nojb commented May 16, 2025

After #13839 and #14009 the last remaining bit of this PR has been split off to #14040. This PR is thus subsumed by those other PRs and can thefore be closed. Thanks everyone who chipped in the dicussions.

@nojb nojb closed this May 16, 2025
@nojb nojb deleted the let_str_item branch May 16, 2025 12:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

devel-discussion parsetree-change Track changes to the parsetree for that affects ppxs

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants