We need to have some syntax to distinguish the barred delimiters {%foo-bar|...|bar} from the nonbarred delimiters {%foo ...}, but I don't really like the choice of - (despite having proposed it myself). Could we consider, for example, {%foo: ... } (non-significant spaces) in the non-barred case and {%foo bar|...|bar} in the barred case? That is, using a space in the barred case, and some other suggestive token (but not a dot) in the non-barred case?

My vote would be for just having the "barred" versions. I think that the simplicity and the consistency with the existing quotation syntax is worth the cost of an extra couple of unobtrusive characters.

In the above example,

let content = {%html <div id="content">some content</div>}

Does the quoted string work when we have the balanced use of } delimiter as mentioned in #8802 (comment)?

Is the syntax below supported?
let div1 = {%jsx <div id="id1">{text("hello world")}</div>}

That feature would enable ppxes to support a large number of dsl. I think it best if we leave the anti-quotation mechanism to ppx authors to exploit and implement.

To provide more context to my example above, here is the jsx syntax I would like to eventually support/implement in ocaml. https://github.com/bikallem/jsx-ssr/blob/master/examples/JsxTester.re

Note that the example is my current experiments with the ReasonML JSX syntax. I would like to implement the same syntax with the addition of {%jsx } bit.

1. OCaml is already full of magical and surprising syntax variations like ; ; in match leafs and whatnot -- in my opinion they make things harder to read/learn, and they make things harder to refactor. I tentatively agree with @lpw25; and if we can avoid introducing syntax that works on single lines but fails across multiple lines, I would be happy. That being said, [%html{|<div id="content">some content</div>|}] is pretty ugly, and I think this PR is a good idea.

2. I find it confusing that div1 and div2 below are essentially equivalent, yet look pretty different:
   Why does div2 have two percentage signs, but `div1 only has one? (copied from @Drup 's example without knowing what I'm doing)

Would replacing div1 with something like the div3 given be too ugly?

let div1 = {%M.foo <hello> <hi> }
let div2 = {%%M.foo| <hello>
 <hi> |}

let div3 = {%M.foo| <hello> <hi> |}

3. Agree with @gasche that - is easily overlooked. I think the {%M.foo bar| hello hi |bar} suggestion seems pleasant.

Some clarifications:

• All forms accept multiline.
• The only difference between the two forms are what the end delimiters look like. either }, |} or |xxx}, by increasing verbosity.
• The difference between % and %% is exactly the same as for regular ppx extensions: one is for structure/signature items, the other is for expression/pattern/types. It's consistent with the rest of the language.
• The shortest form doesn't care about balanced content. Since the (slightly) longer form exists, I don't think it's necessary.

I strongly think we should optimize for the most common form, i.e, the short one. For quoted strings, the long form {xx|....|xx} is almost never used, since the short form {|...|} is sufficient 99% of the time. Given that, I don't see why using dash is a problem. It'll almost never be used anyway.

Some remarks:

• The one use-case that started this discussion (the domain-specific JSX syntax) profusely uses braces, which led to my idea of assuming well-balancing the non-barred case. Requiring well-balancing is not a perfect solution either, due to braces occurring within string literals (typically using single- or double-quotes) in the embedded language). It supports more use-cases without any extra syntactic weight, but it makes the specification more complex and less regular.

• Speaking of regularity, there is no precedent for {...} being used as opaque literals, while there is precedent for the barred forms. We would need a strong justification to add the non-barred form, and I'm not convinced by the gain/risk ratio of the current proposal. (We add one syntax to shorten the short path by one character; but it is non-expected and makes it likely that users run into unexpected-end-of-literal issue, then having to move to a less-discoverable alternative.)

• It's not very nice that someone that would write {%foo bar|...|bar} would have their content silently accepted by your grammar as a non-barred payload. (They probably would get an error from the extension parser.) One more reason to get rid of the non-barred case at first.

Alright, I still would prefer the shortest version, but consistency is more important. I'll change the PR to only add barred versions.

Any opinion on the position question?

Your plan wrt. position sounds very reasonable to me.

Done and done. All the locations should be decent. The patch to add locations to string constants is big, but boring.

@Drup @gasche Will this be merged for ocaml 4.10.0 perhaps?

The implementation is fine, but there hasn't been enough discussion among language maintainers to create a consensus on the new language feature. We may discuss it on the occasion of an upcoming development meeting, or in an online discussion round.

@gasche ping. Any movement on this perhaps? I think this new ppx syntax along with the upcoming ppxlib work (https://discuss.ocaml.org/t/the-future-of-ppx/3766) will make the ppx system in ocaml much more ergonomic and useful for use cases other than UI(jsx/html) such as sql, configs, json and so on. Would really love to see this merged soonish. 🥇

I just rebased. The change entry is still in the 4.10 section, but I suspect I'll have to move it before merging.

Rebased, Short manual added, All tests pass. :)

We had a developer meeting today where the syntax as currently proposed (only the barred form, with a space between the extension and quote name) was approved in principle. We still need someone to do a thorough review of the implementation. I did a review of an earlier version of the patch (without the location changes), this would need to be reviewed again.

Quoted extensions should probably be handled in comments too, such that (* {%foo|*)|} *) is a valid comment.

I would expect {%foo|...|} to be equivalent to [%foo {|...|}] in any fragment of OCaml syntax; this indeed supports your suggestion.

We would like the following extension of the testsuite to work properly. It currently does not:

diff --git a/testsuite/tests/parsing/quotedextensions.ml b/testsuite/tests/parsing/quotedextensions.ml
index 6fcb9e2654..3962cef32d 100644
--- a/testsuite/tests/parsing/quotedextensions.ml
+++ b/testsuite/tests/parsing/quotedextensions.ml
@@ -30,3 +30,15 @@ let {%M.foo bar| <hello>{|x|} |bar}
    {x}
  </hello>
 |}
+
+(* Double quotes inside quoted strings inside comments: *)
+(* {|"|}, and *)
+(* [%foo {|"|}], and *)
+(* {%foo|"|}
+  should be valid inside comments *)
+
+(* Double quotes inside quoted strings inside comments: *)
+(* {|*)|}, and *)
+(* [%foo {|*)|}], and *)
+(* {%foo|*)|}
+  should be valid inside comments *)