Skip to content

Full unicode support #306

@kazimuth

Description

@kazimuth

TL;DR: Supporting unicode is hard, and it might be a good idea to use a library that knows how to do it well. Maybe look into using harfbuzz as well as freetype for font rendering. Unicode-width is good but doesn't do everything.

The problem of translating sequences of unicode codepoints to actual you-can-draw-this-on-screen glyphs, supporting things like character width (#265), ligatures (#50), bidirectional text (like in arabic), and text reordering (!), is called complex text layout. It is, appropriately, complex, and most terminals don't actually do it very well.

Windows and Mac both have systems that perform layout, integrated with their font rasterizers:

  • On windows, there's several different supported libraries, from various periods of windows history: DirectWrite, GDI, and Uniscribe on windows.
  • On OS X, there's Core Text.
  • On linux, there's a stack: freetype for rasterizing, harfbuzz for "text shaping", and pango for full layout. All of these libraries are actually cross-platform, though. See this article for more on the difference between pango and harfbuzz.

There's also ICU, which is cross-platform and supported by IBM (I think?)

Of the available options:

  • OS X Terminal / Iterm2 use Core Text
  • Terminator (and everything else that uses GTK) uses Pango
  • Windows CMD and Powershell use black magic and baby tears
  • Chrome uses harfbuzz
  • Firefox uses pango
  • ...

It seems to me like harfbuzz is the best option in terms of cross-platform support and level of control. You basically hand it a line of text and it tells you all the glyphs to draw in that line. Keep in mind I'm not actually a font rendering person, though, and it's possible I'm missing important details here.

This would probably have a performance cost, but I'm not sure how much of a performance cost. With a clever implementation it might not be too bad, and would be a huge boon to international users.

Other relevant links:

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions