@Albab-Hasan thanks for this! I'm a little worried about performance. How expensive is looking up each character every time? How many microseconds does this add to rendering a typical page of code?

Claude also noticed that this defaults to continuing the run in the fallback font, it should probably switch back to the main font more aggressively (e.g. 字a should use the main font for "a").

I am not sure that we should here, but we do use the unicode-segmentation library to do glyph grouping in other places. Should we also use that here?

@ConradIrwin thanks for the review. pushed the bug fix in the latest commit.

as for performance i considered a few approaches before landing on the charmap probe:

• pre-built coverage bitset per font at load time (faster): enumerate all supported codepoints once on load store a roaring bitmap or sparse bitset. lookup becomes O(1) vs the current binary search over cmap segments. obvious win if the charmap probe shows up in profiles.

• unicode block range pre-filter (faster but brittle): skip the probe entirely for chars whose unicode block clearly maps to one font. fast but breaks on cross-script composition and needs maintenance.

• per-(FontId, char) cache (mixed): high hit rate for real text but adds lock contention on the hot path.

• try-shape-and-check (much slower): shape the character, check for .notdef. only useful as last resort.

i started with the charmap probe since its simple and already correct. happy to add a benchmark or switch to bitsets if you want numbers before merging.

for the unicode segmentation yeah i think we should. the mark category heuristic misses hangul jamo, regional indicator pairs and indic prepend chars. switching to .graphemes(true) lets us delete is_inheriting_codepoint entirely and just check the first codepoint of each cluster. the crates already a workspace dep. what do you think?

Sounds good to me. Are you able to run a micro benchmark? (e.g. Load a page full of code and see how many microseconds it takes to render a frame before and after with no font caching). If it's <5% that's seems ok; but if it's 10% or more we should think about optimization (like just ignoring ascii or something).

ran a bench directly on layout_line (criterion, 100 samples, no layout cache):

input is ~3 800 chars of ASCII code in one run (worst case — in practice Zed splits a line into many short runs).

6% is just over your threshold. the cost is one charmap.map() probe per character. skipping codepoints ≤ U+007F would eliminate almost all of it for code text since every latin font covers ASCII — happy to add that if you'd like:

fn pick_covering_slot(...) -> Option<usize> {
    if (ch as u32) <= 0x7F || is_inheriting_codepoint(ch) {
        return current;
    }
    ...
}

the zero-overhead case (no fallbacks configured, the default) is unchanged.

will open the unicode-segmentation follow-up as a separate pr.

👍 Thank you for testing it!

I think we should do both the latin optimization and the unicode segmentation so we can land this in one cohesive chunk. It'd be annoying to merge this and then have to rush follow up changes before the next release ships.

@ConradIrwin any updates?

Sorry I don't get notified when the PR is pushed to (and I'm super behind on my reviews).

Happy to merge this, but looks like the linter is unhappy; what's the overhead with the optimization and unicode segmentation?

@ConradIrwin My bad I'll fix all the issues. And yeah this pr just disappeared for some reason that might explain you not getting the notification.

Benchmarks:

No fallback configured | 3.87 ms | baseline
1 fallback, all ASCII code text | 3.94 ms | ~70 µs (~1.8%)

Down from ~6% overhead before. The ASCII skip (char <= '\u{007F}') removes the charmap probe for the most code text and grapheme segmentation replaced the is_inheriting_codepoint heuristic entirely.

@ConradIrwin done

Thank you!

@zed-industries/approved