Skip to content

Implement remaining sublime-syntax Build 4075 features#612

Merged
keith-hall merged 10 commits into
trishume:masterfrom
stefanobaghino:sublime-syntax-build-4075-remaining
Mar 21, 2026
Merged

Implement remaining sublime-syntax Build 4075 features#612
keith-hall merged 10 commits into
trishume:masterfrom
stefanobaghino:sublime-syntax-build-4075-remaining

Conversation

@stefanobaghino

@stefanobaghino stefanobaghino commented Mar 10, 2026

Copy link
Copy Markdown
Contributor

Make progress on #323:

  • v2 set + clear_scopes: allow set operations to apply clear_scopes
  • v2 embed_scope replacement: embed_scope replaces embedded syntax scope
  • Multiple inheritance: extends accepts a YAML list of syntax paths

- v2 set + clear_scopes: allow set operations to apply clear_scopes
- v2 embed_scope replacement: embed_scope replaces embedded syntax scope
- Multiple inheritance: extends accepts a YAML list of syntax paths
Comment thread src/parsing/syntax_definition.rs Outdated
Remove derived PartialOrd and implement it manually so that
PartialOrd delegates to Ord, following the standard pattern for
types with a total ordering. This resolves a hard lint error
introduced by Rust 1.92's clippy that was not suppressed by the
previous #[allow] attribute placement.
…ring>

None and Some(vec![]) are never semantically distinct: the YAML parser
already converts empty arrays to None, and all consumption sites treat
both identically. Replace the Option wrapper with a plain Vec, using an
empty vec to mean "no extends".
Comment thread src/parsing/scope.rs

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This failed for me on CI, although I couldn't repro locally. Not sure of what's wrong.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To clarify, the only reason why I added this comment is to make sure it was clear why I edited this (seemingly unrelated) file. @jrappen also raised that it appears to be a problem on master as well here. As mentioned elsewhere, this is already take care of in a2d1823. Not sure how this creeped into master TBH, this check passed on #610 IIUC.

@jrappen

This comment was marked as resolved.

@stefanobaghino

Copy link
Copy Markdown
Contributor Author

Right now I don't have time to look into your issue with failed CI.

There's no need, I only added the comment to make sure it was clear why I made the change. The error is the same that you raised in #610 (comment) and that I fixed in a2d1823.

With regard to multiple inheritance think of it like this. Say you have a JSON syntax, and then a bunch of helper syntaxes where the first overrides JSON to add optional trailing commas in arrays and objects, the second adds support for single-quoted strings, the third adds support for null as an additional value, the fourth adds support for line and block comments. Now JSON has syntax flavors like GeoJSON, HJSON, JSON5, JSONC (with Comments), JSON.NET. So for JSONC as an example you would need to do the following: (1) add multiple inheritance where (2) the comments helper syntax goes into the list of syntaxes for the extends key before (3) the trailing commas helper syntax. syntect should produce an error (as does Sublime) if (1) not all entries in the list extend from the same syntax or (2) you add the base syntax to the list, which is the same as (1) but I just wanted to reiterate that explicitely in case it wasn't obvious.

Are you saying this pointing to some specific flaw in the PR?

@jrappen

This comment was marked as resolved.

Cover the remaining Build 4075 compatibility items raised by jrappen:

v1/v2 behavioral differences:
- B: v2 set action should not apply parent meta_content_scope to matched
  text (currently FAILS — implementation gap)
- D: v2 embed escape text should get meta_scope of embed context
  (currently FAILS — not yet implemented)
- E: v2 push with multiple targets applies clear_scopes only from the
  last context (PASSES — confirmed working)
- F: capture group scopes are applied in text position order, not capture
  number order (PASSES — confirmed working)

Multiple inheritance validation gaps:
- G: all parents in extends must derive from the same base syntax
  (currently FAILS — no common-base validation)
- H: parent and its base must share the same version
  (currently FAILS — no version consistency check)
- I: source syntax and its parents must share the same version
  (currently FAILS — no version consistency check)

Failing tests are left as-is to document unimplemented behavior for
follow-up work.
- v2 set: pop parent meta_content_scope before matched text so it is
  not visible on the token, matching Sublime Build 4075 behavior
- v2 embed escape: keep embed_scope (meta_content_scope) visible on the
  escape text itself; fix spurious scope pop caused by the with_prototype
  lookahead popping the embedded context whose mcs was never pushed
- extends validation: reject extends when parent and child versions
  differ (H/I) or when multiple parents share no common base syntax (G)
@stefanobaghino

stefanobaghino commented Mar 12, 2026

Copy link
Copy Markdown
Contributor Author

Thanks for the detailed review!

some behavior with regard to addressing syntax v1 vs. v2 changed. I saw you addressing the stacking of meta_scopes. The complete list is here as a reference. I haven't checked if you addressed all, yet.

I went through the compatibility list after your comment. The latest two commits add tests covering the remaining gaps I found and fix the ones that were failing: specifically, set in v2 no longer applies the parent context's meta_content_scope to the matched text, and the escape pattern in a v2 embed now correctly sees the embed_scope (which is the context's meta_content_scope). The v1/v2 differences that were already handled (e.g. clear_scopes stacking on multi-push, capture group scoping) continue to pass.

I saw you checking if each syntax in an extends list value exists. I didn't see you checking if whatever they extend from themselves: exists, and is the same for all, and has the same syntax version between the parents and their common base, and has the same syntax version between the parents and the source

The "exists" case was already covered: unresolvable parents cause the child to remain in the unresolved set, which produces a warning at the end of the fixed-point loop. The other three were missing — the latest commits add them:

  • Same root for all parents: if parents don't share a common ancestor syntax, the extends is skipped with a warning.
  • Version consistency: the version check is enforced at each resolution step — parents must have the same version as the child extending them. Since this applies transitively throughout the chain, it covers both the parents-vs-common-base and source-vs-parents cases.

In all error cases syntect emits a warning (via eprintln!) and skips applying the extends, leaving the child syntax in its unextended state.

Reformat method chaining and function calls to meet rustfmt standards:
- parser.rs: split ops.push() call across multiple lines
- syntax_set.rs: convert .iter().position() chains to method-chaining style
Per Sublime Text docs, embed_scope applies to text "after the match
and before the escape" — it should not apply to the escape text itself.

The v2 embed_scope_replaces special case in the Pop path of
push_meta_ops incorrectly kept the embed_scope (meta_content_scope) on
the scope stack during escape text processing. This caused downstream
highlighting regressions (e.g. Typst fenced code block closing
backticks getting the embedded language's color instead of the default
foreground).

Remove the special case and let the normal pop logic handle it: pop
meta_content_scope before the escape text (initial=true) and
meta_scope after (initial=false). The skip check for embedded syntax
contexts whose meta_content_scope was never pushed is preserved.
@stefanobaghino stefanobaghino changed the title Implement remaining sublime-syntax Build 4075 features (#323) Implement remaining sublime-syntax Build 4075 features Mar 13, 2026
@jrappen

This comment was marked as resolved.

Verify that extends paths like "syntaxes/Child.sublime-syntax" resolve
correctly against full loaded paths via suffix matching, confirming
relative paths work without any code changes.
@stefanobaghino

Copy link
Copy Markdown
Contributor Author

I hadn't thought about local relative paths explicitly. I looked into it, the current find_parent_index implementation already handles this: it normalizes the extends value and checks whether any loaded syntax path ends with it (after prepending /). So a relative path like syntaxes/json-feature-comments.sublime-syntax will match against the full loaded path Packages/JSON/syntaxes/json-feature-comments.sublime-syntax. I added a test in 74e4476 to make sure this keeps working.

@stefanobaghino

Copy link
Copy Markdown
Contributor Author

Is there some way in which I can facilitate the review process? 🙂

@jrappen

jrappen commented Mar 20, 2026

Copy link
Copy Markdown
Contributor

My comments above were addressed, marked them as "resolved". Looks good so far.

@jrappen

jrappen commented Mar 20, 2026

Copy link
Copy Markdown
Contributor

@keith-hall maybe you can take a look if you have a min?

@keith-hall keith-hall left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@keith-hall keith-hall merged commit 9812399 into trishume:master Mar 21, 2026
5 checks passed
stefanobaghino added a commit to stefanobaghino/syntect that referenced this pull request Apr 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants