Perform font fallback #6926

nulano · 2023-02-02T17:00:55Z

Add a new type, ImageFont.FreeTypeFontFamily(font1, font2, ..., layout_engine=layout_engine), that can be used with ImageDraw.text*(...) functions performing font fallback. Font fallback is done per cluster with Raqm layout (similar to Chromium) and per codepoint with basic layout.

This PR is far from complete, several TODOs:

Font families have only a minimal API so far, e.g. retrieving metrics or setting font variations should be supported
Maybe add a wrapper similar to ImageFont.truetype(...), perhaps ImageFont.truetype_family(...)?
Lots of tests
Documentation

I would like to get some feedback, both on the API and the implementation, before working on the TODOs above.
A dev build for Windows is available from the artifact here: https://github.com/nulano/Pillow/actions/runs/8583137967

A few examples (click to expand):

All examples use this helper block:

from PIL import Image, ImageDraw, ImageFont

im = Image.new("RGBA", (500, 200), "white")
draw = ImageDraw.Draw(im)
def line(y, string, font, name, **kwargs):
  draw.text((10, y), name, fill="black", font=font, **kwargs)
  draw.text((300, y), string, fill="black", font=font, **kwargs)

example()

im.show()

Combining Latin, symbols, and an emoji:

def example():
  s = "smile ⊗ 😀"

  times = ImageFont.truetype("times.ttf", 24)
  segoe_ui_emoji = ImageFont.truetype("seguiemj.ttf", 24)
  segoe_ui_symbol = ImageFont.truetype("seguisym.ttf", 24)
  family = ImageFont.FreeTypeFontFamily(times, segoe_ui_emoji, segoe_ui_symbol)

  line(30, s, times, "Times New Roman", anchor="ls", embedded_color=True)
  line(80, s, segoe_ui_emoji, "Segoe UI Emoji", anchor="ls", embedded_color=True)
  line(130, s, segoe_ui_symbol, "Segoe UI Symbol", anchor="ls", embedded_color=True)
  line(180, s, family, "Font Family", anchor="ls", embedded_color=True)

Combining Arabic, Greek, Latin, and a symbol:

def example():
  s = "ية↦α,abc"

  scriptin = ImageFont.truetype(r"C:\Users\Nulano\AppData\Local\Microsoft\Windows\Fonts\SCRIPTIN.ttf", 24)
  segoe_ui = ImageFont.truetype("segoeui.ttf", 24)
  segoe_ui_symbol = ImageFont.truetype("seguisym.ttf", 24)
  family = ImageFont.FreeTypeFontFamily(scriptin, segoe_ui, segoe_ui_symbol)

  line(30, s, scriptin, "Scriptina", direction="ltr", anchor="ls")
  line(80, s, segoe_ui, "Segoe UI", direction="ltr", anchor="ls")
  line(130, s, segoe_ui_symbol, "Segoe UI Symbol", direction="ltr", anchor="ls")
  line(180, s, family, "Font Family", direction="ltr", anchor="ls")

Combining characters are treated as part of a single cluster (with Raqm layout):

def example():
  import unicodedata

  s = " ̌,ῶ,ω̃,ώ,ώ, ́,á,č,č"
  for c in s:
    print(unicodedata.name(c))

  le = ImageFont.Layout.RAQM  # or ImageFont.Layout.BASIC
  scriptin = ImageFont.truetype(r"C:\Users\Nulano\AppData\Local\Microsoft\Windows\Fonts\SCRIPTIN.ttf", 24, layout_engine=le)
  dubai = ImageFont.truetype(r"DUBAI-REGULAR.TTF", 24, layout_engine=le)
  gentium = ImageFont.truetype(r"C:\Users\Nulano\AppData\Local\Microsoft\Windows\Fonts\GentiumPlus-Regular.ttf", 24, layout_engine=le)
  family = ImageFont.FreeTypeFontFamily(scriptin, dubai, gentium, layout_engine=le)

  line(30, s, scriptin, "Scriptina", anchor="ls")
  line(80, s, dubai, "Dubai", anchor="ls")
  line(130, s, gentium, "GentiumPlus", anchor="ls")
  line(180, s, family, "Font Family", anchor="ls")

Raqm layout:

Basic layout:

The string s contains:

SPACE
COMBINING CARON
COMMA
GREEK SMALL LETTER OMEGA WITH PERISPOMENI
COMMA
GREEK SMALL LETTER OMEGA
COMBINING TILDE
COMMA
GREEK SMALL LETTER OMEGA WITH TONOS
COMMA
GREEK SMALL LETTER OMEGA
COMBINING ACUTE ACCENT
COMMA
SPACE
COMBINING ACUTE ACCENT
COMMA
LATIN SMALL LETTER A
COMBINING ACUTE ACCENT
COMMA
LATIN SMALL LETTER C WITH CARON
COMMA
LATIN SMALL LETTER C
COMBINING CARON

Yay295 · 2023-02-02T18:34:03Z

src/_imagingft.c

            switch (anchor[1]) {
                case 'a':  // ascender
-                    y_anchor = PIXEL(self->face->size->metrics.ascender);
+                    y_anchor = PIXEL(family->faces[0]->size->metrics.ascender);
                    break;
                case 't':  // top
                    y_anchor = y_max;
                    break;
                case 'm':  // middle (ascender + descender) / 2
                    y_anchor = PIXEL(
-                        (self->face->size->metrics.ascender +
-                         self->face->size->metrics.descender) /
+                        (family->faces[0]->size->metrics.ascender +
+                         family->faces[0]->size->metrics.descender) /
                        2);
                    break;
                case 's':  // horizontal baseline


I think here you would need to get the ascenders and descenders of every font used on this line, not just the first one, and use the greatest value.

Perhaps. I don't think looking just at the fonts used in a given line would be good - that could create an inconsistent layout for the text_multiline functions.

This should use either the first font's metrics or the greatest value across all fonts in a family object, and it should be consistent with family.getmetrics() (not yet implemented).

I've just tested what MS Edge (i.e. Chromium) does - it uses the metrics of all fonts up to the last one that is in use for a given line, even if it causes uneven line spacing.

For example, with three fonts, a paragraph that only uses the second font will have line spacing computed from the first two fonts. If one word in the paragraph then uses the third font, that line will include the third font's line spacing, but the other lines are unaffected.

This is incompatible with how Pillow currently calculates line spacing for multiline text (see #6469 (comment)), so I think I'll just use the maximum over all fonts (i.e. assume that if a user creates a FontFamily object they actually want to use all of the fonts), and leave fixing it for #1646 (+document the limitation).

radarhere · 2023-02-18T07:21:21Z

src/_imagingft.c

+                found = 1;
            }
-        }
+            /* prefer first font's missing glyph if no font support this codepoint */


Suggested change

/* prefer first font's missing glyph if no font support this codepoint */

/* prefer first font's missing glyph if no font supports this codepoint */

radarhere · 2023-02-18T07:22:35Z

src/_imagingft.c


+static PyObject *
+getfamily(PyObject *self_, PyObject *args, PyObject *kw) {
+    /* create a font family object from a list of file names and a sizes (in pixels) */


Suggested change

/* create a font family object from a list of file names and a sizes (in pixels) */

/* create a font family object from a list of file names and sizes (in pixels) */

src/_imagingft.c

radarhere · 2023-06-07T20:48:37Z

src/_imagingft.c

+                (*glyph_info)[i].x_offset = 0;
+                (*glyph_info)[i].y_offset = 0;
+
+                /* This has been broken and had no effect for many years now...


Could you expand on this?

#5191 (comment)

nissansz · 2023-07-16T12:22:18Z

How to download the module or the build with below module?

AttributeError: module 'PIL.ImageFont' has no attribute 'FreeTypeFontFamily'

nulano · 2023-07-16T17:54:00Z

You can use a source install from the branch https://github.com/nulano/Pillow/archive/refs/heads/font-fallback.zip using the installation instructions: https://pillow.readthedocs.io/en/stable/installation.html#building-from-source

The previous Windows dev build has expired so I've re-run the CI for this branch here: https://github.com/nulano/Pillow/actions/runs/5568846705
You can download the dist-font-fallback-build.zip artifact from there and install the relevant wheel for your version of Python.
(I've had to mark the test step as ignored because some crash test files seem to have been moved since.)

nissansz · 2023-07-16T22:02:50Z

Thank you.
I tried. It works now.
Will it be updated to new pillow version 10.0 too?

nissansz · 2023-09-30T01:59:36Z

If I want to use below method to draw text for each char. is it slower than fontfamily?

Iterate all chacters in a string,
judge each character to see whether it is in the char list of a desire font,
if not in the list, specify the backup font to draw the character, then continue

nissansz · 2023-09-30T02:23:20Z

If a character is not in char list of a font, it can be displayed by a backup font.

But some fonts are strange, the backupfont did not work too.

aclark4life · 2024-02-01T17:27:10Z

Hi. Is there any progress?

@Mitchell-kw-Lee Looks like this is still a WIP, are you able to test the branch and report results? That may help …

nulano · 2024-02-01T17:56:35Z

This needs a pretty large rebase before it can cleanly merge with the main branch.

Currently, I see #6926 (comment) as the biggest unresolved issue with this PR. I think I have an idea that could work, but I've not had time to work on this.

However, if you are able to test this PR as is (even though it is made for an older version of Pillow) and report whether it works / causes issues, it could perhaps be helpful.

nulano · 2024-02-02T17:05:40Z

As I wrote above, this branch has a non-trivial merge conflict I haven't had time to resolve, even though I would like to get to it at some point.

If you'd like (keep in mind this is based on Pillow from a year ago), I can re-run the CI to generate new Windows wheels for easier installation. If you are on Linux/macOS, you'll need to figure out source installation (hint: https://pillow.readthedocs.io/en/stable/installation.html#building-from-source)

nulano · 2024-02-03T08:58:51Z

I've rerun the CI: https://github.com/nulano/Pillow/actions/runs/7765749734
You can download dist-font-fallback-build from the list of artifacts at the bottom, unzip it, and install with python -m pip install Pillow-9.5.0.dev0-cp311-cp311-win32.whl --user.

Example usage is at the top of this page (click on A few examples (click to expand) in the top comment) or #4808 (comment).

radarhere · 2024-02-04T05:17:30Z

FreeTypeFontFamily doesn't have getsize() - but that's not a method that needs to be added later, it's a method that has been transitioned out of handling fonts in the latest Pillow versions. I suggest you use getbbox() instead. You can read more at https://pillow.readthedocs.io/en/stable/deprecations.html#font-size-and-offset-methods

nulano · 2024-02-04T09:32:19Z

_font_family = ImageFont.FreeTypeFontFamily(_font, _backup_font) <<-- HERE

This function does not call getsize or any similar function, so I don't see how that could be possible. Please post the full stack trace.

nulano · 2024-02-04T11:48:53Z

Ah right, looks like multiline text is not working for this branch. If you want to use multiline text, you'll have to wait for someone to rebase this branch to main (the current _multiline_spacing function uses getbbox instead of getsize and I didn't implement getsize for FreeTypeFontFamily in anticipation of that change).

aclark4life · 2024-02-04T12:46:49Z

@nulano Questions.

* Is there the 'Someone' already on it? or?

* 'Wait' mean.....how long?

@Mitchell-kw-Lee You can follow related issues and provide comments, questions and feedback just as you did here to find the answers to those questions. Pillow is released quarterly, and your fix may or may not be in the next release, based on the answers to those questions. I suspect we already know most of the involved parties from the discussion in this thread, and I don't think we can give any meaningful answer in this case to the the question of how long you'll have to wait for a new feature, developed in large part by volunteers, or underpaid developers. Sometimes we can answer! But I don't see anything definitive here yet … more than @nulano has already provided at least.

nissansz · 2024-04-05T11:18:43Z

fontfamily (font1 (include some chars)+font2(include most chars)) cannot show some characters

font1: see attachment.

font2: Arial Unicode MS.ttf or any other fonts
str = 'をも資资儲储議议歷历權权個个TextInAaBbCcDdEeFfGgHhIi family PILLOW'

ballpen.zip

    # 写入字体效果的文字
    image = Image.new('RGB', (image_width, image_height), color='white')
    draw = ImageDraw.Draw(image)

     font0 = ImageFont.truetype(font_file, size=image_height-2)
     font1 = ImageFont.truetype(r'C:/F/fonts/tianshiyanti2.0.ttf', size=image_height-2)

        font_family = ImageFont.FreeTypeFontFamily(font0, font1)
        draw.text((0, 0), text+'   family PILLOW', fill='black', font=font_family)

nulano · 2024-04-06T17:34:31Z

Your code is incomplete. What is font_file? What is C:/F/fonts/tianshiyanti2.0.ttf? What is text? Please provide a complete example. See https://stackoverflow.com/help/minimal-reproducible-example

nulano · 2024-04-06T23:02:19Z

What is difference between [dist-windows-x86] and [dist-windows-AMD64]?

dist-windows-x86 is 32-bit x86
dist-windows-AMD64 is 64-bit x86.
dist-windows-ARM64 is 64-bit ARM.

Any new function for fontfamily in new version?

No, except multiline text seems to work now.

I can download from your above link, but I don't see such link on list. How to find above links?

Go to https://github.com/nulano/Pillow/actions/runs/8583137967 and scroll down:

nissansz · 2024-04-06T23:05:18Z

What is difference between [dist-windows-x86] and [dist-windows-AMD64]?

dist-windows-x86 is 32-bit x86 dist-windows-AMD64 is 64-bit x86. dist-windows-ARM64 is 64-bit ARM.

Any new function for fontfamily in new version?

No, except multiline text seems to work now.

I can download from your above link, but I don't see such link on list. How to find above links?

Go to https://github.com/nulano/Pillow/actions/runs/8583137967 and scroll down:

Saw it. Thank you.
How to use multiple lines for font family?

There is still below problem from yesterday.
fontfamily (font1 (include some chars)+font2(include most chars)) cannot show some characters

font_file0: ballpen.zip.

font_file1: tianshiyanti2.0.ttf.zip

str = 'をも資资儲储議议歷历權权個个TextInAaBbCcDdEeFfGgHhIi family PILLOW'

image = Image.new('RGB', (image_width=1000, image_height=25), color='white')
draw = ImageDraw.Draw(image)

font0 = ImageFont.truetype(font_file0, size=20)
font1 = ImageFont.truetype(font_file1, size=20)

font_family = ImageFont.FreeTypeFontFamily(font0, font1)
draw.text((0, 0), textstr, fill='black', font=font_family)

nulano · 2024-04-06T23:12:32Z

How to use multiple lines for font family?

Same as for regular text:

draw.text((0, 0), "line1\nline2", font=family, ...)

There is still below problem from yesterday.

I'll take a look at it when I have the time.

nissansz · 2024-04-06T23:14:37Z

How to use multiple lines for font family?

Same as for regular text:
draw.text((0, 0), "line1\nline2", font=family, ...)
There is still below problem from yesterday.

I'll take a look at it when I have the time.

I wonder whether there is any function to import html to pillow canvas, like rendering a web page.
Write html tables, etc.

radarhere · 2024-04-07T01:09:48Z

I wonder whether there is any function to import html to pillow canvas, like rendering a web page.
Write html tables, etc.

This is off-topic for this discussion of multiple fonts, but no, Pillow doesn't have any functionality for rendering HTML to an image.

PaulOlteanu · 2024-04-18T20:41:47Z

Would it be possible to get a getmask method on FreeTypeFontFamily so that it could be used with TransposedFont?

nulano · 2024-04-21T18:32:40Z

Would it be possible to get a getmask method on FreeTypeFontFamily so that it could be used with TransposedFont?

Sure, I didn't bother for a WIP PR when I wasn't sure it would be useful, but it's quite simple (literally just copy-pasted from FreeTypeFont 😄).

You should be able to download wheels from this workflow in a few hours: https://github.com/nulano/Pillow/actions/runs/8774721560

lavnishhh · 2024-08-25T12:55:31Z

You should be able to download wheels from this workflow in a few hours: https://github.com/nulano/Pillow/actions/runs/8774721560

Hi, the builds are expired. Could you please re run? I'm fairly new to this, so I don't know if there's another way to get the build

…-missing glyph in a single cluster

… instead of last font's missing glyph

for more information, see https://pre-commit.ci

nulano · 2024-08-25T13:39:45Z

I've rebased the PR and triggered a build, you should see the wheels here in a few hours: https://github.com/nulano/Pillow/actions/runs/10547427520

abjugard · 2025-06-12T13:26:28Z

This would be amazing to see completion, I have a project now where I could really use this!

wlt233 · 2025-08-22T06:06:52Z

Hi there, I’m curious if there’s any update on this PR, thanks!

nulano mentioned this pull request Feb 2, 2023

Backup font for missing characters when drawing text #4808

Open

Yay295 reviewed Feb 2, 2023

View reviewed changes

nulano force-pushed the font-fallback branch from 8c4c0d6 to 2b0b5b3 Compare February 8, 2023 02:47

radarhere reviewed Feb 18, 2023

View reviewed changes

rahmcoff mentioned this pull request Feb 19, 2023

Support Emoji in /profile PADD Text jp00p/AGIMUS#386

Closed

radarhere mentioned this pull request May 21, 2023

Corrected error code #7177

Merged

radarhere reviewed Jun 7, 2023

View reviewed changes

src/_imagingft.c Show resolved Hide resolved

radarhere reviewed Jun 7, 2023

View reviewed changes

radarhere added the Needs Rebase label Jun 13, 2023

This was referenced Jul 17, 2023

FreeType error when drawing text #7290

Closed

Is it possible to generate or simulate italic text for any font other than an original italic font? #7292

Closed

radarhere mentioned this pull request Sep 30, 2023

Fontfamily cannot show some characters as browser effect #7429

Closed

Minnozz mentioned this pull request Feb 27, 2024

Preview Card doesn't display non-latin characters properly bookwyrm-social/bookwyrm#1828

Closed

nulano mentioned this pull request Apr 5, 2024

fontfamily (font1 (include some chars)+font2(include most chars)) cannot show some characters #7946

Closed

radarhere added the Needs Rebase label May 22, 2024

nulano mentioned this pull request Jun 3, 2024

Rendering multi line text in bounding box, text bbox coordinates issue #8092

Closed

nulano added 9 commits August 25, 2024 15:12

(WIP) support font-family style fallback with basic text layout

6002b0a

merge single font and font family code paths

3710f2d

font fallback for Raqm layout

9105116

C89 / Lint / Raqm<0.9.0 fixes

3b3cac8

fix Raqm fallback algorithm when a missing glyph is followed by a non…

c244169

…-missing glyph in a single cluster

fix Raqm fallback for first codepoint, use first font's missing glyph…

cbbd9bd

… instead of last font's missing glyph

fix face reference for missing glyphs with Raqm layout

5aab906

consider all fonts when computing font family metrics

9081695

FreeTypeFontFamily: add getmask() method

4f957e1

nulano force-pushed the font-fallback branch from 5a83821 to 4f957e1 Compare August 25, 2024 13:37

[pre-commit.ci] auto fixes from pre-commit.com hooks

9ccec98

for more information, see https://pre-commit.ci

nulano removed the Needs Rebase label Aug 25, 2024

radarhere added the Needs Rebase label Sep 8, 2024

wlt233 mentioned this pull request Sep 14, 2024

Render emojis with zwj #8380

Closed

radarhere mentioned this pull request Nov 4, 2024

How to detect character shown as box for a font? #8528

Closed

radarhere mentioned this pull request Mar 31, 2025

Use unicode characters #8849

Closed

mattblovell mentioned this pull request Apr 12, 2025

Fallback font support mattblovell/kodi_panel#54

Open

	/* prefer first font's missing glyph if no font support this codepoint */
	/* prefer first font's missing glyph if no font supports this codepoint */

	/* create a font family object from a list of file names and a sizes (in pixels) */
	/* create a font family object from a list of file names and sizes (in pixels) */

Uh oh!

Perform font fallback #6926

Are you sure you want to change the base?

Perform font fallback #6926

Uh oh!

Conversation

nulano commented Feb 2, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Yay295 Feb 2, 2023

Choose a reason for hiding this comment

Uh oh!

nulano Feb 2, 2023

Choose a reason for hiding this comment

Uh oh!

nulano Feb 8, 2023

Choose a reason for hiding this comment

Uh oh!

radarhere Feb 18, 2023

Choose a reason for hiding this comment

Uh oh!

radarhere Feb 18, 2023

Choose a reason for hiding this comment

Uh oh!

Uh oh!

radarhere Jun 7, 2023

Choose a reason for hiding this comment

Uh oh!

nulano Jun 13, 2023

Choose a reason for hiding this comment

Uh oh!

nissansz commented Jul 16, 2023

Uh oh!

nulano commented Jul 16, 2023

Uh oh!

nissansz commented Jul 16, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

nissansz commented Sep 30, 2023

Uh oh!

nissansz commented Sep 30, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

aclark4life commented Feb 1, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

nulano commented Feb 1, 2024

Uh oh!

nulano commented Feb 2, 2024

Uh oh!

nulano commented Feb 3, 2024

Uh oh!

radarhere commented Feb 4, 2024

Uh oh!

nulano commented Feb 4, 2024

Uh oh!

nulano commented Feb 4, 2024

Uh oh!

aclark4life commented Feb 4, 2024

Uh oh!

nissansz commented Apr 5, 2024

Uh oh!

nulano commented Apr 6, 2024

Uh oh!

nulano commented Apr 6, 2024

Uh oh!

nissansz commented Apr 6, 2024

Uh oh!

nulano commented Apr 6, 2024

Uh oh!

nissansz commented Apr 6, 2024

Uh oh!

radarhere commented Apr 7, 2024

Uh oh!

PaulOlteanu commented Apr 18, 2024

Uh oh!

nulano commented Apr 21, 2024

Uh oh!

lavnishhh commented Aug 25, 2024

Uh oh!

nulano commented Feb 2, 2023 •

edited

Loading

nissansz commented Jul 16, 2023 •

edited

Loading

nissansz commented Sep 30, 2023 •

edited

Loading

aclark4life commented Feb 1, 2024 •

edited

Loading