Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 93.44%. Comparing base (209371e) to head (bd943e8).

@@           Coverage Diff           @@
##             main     #745   +/-   ##
=======================================
  Coverage   93.44%   93.44%           
=======================================
  Files           7        7           
  Lines        2241     2241           
=======================================
  Hits         2094     2094           
  Misses        147      147           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

We ought a quick benchmark to see if there are meaningful performance gains from building with the later toolchains available in the newer images. Building more than one wheel would be a very easy way to give a free performance boost if there are.

#741 (comment)

Here's benchmark branch and run, uses tests/benchmark.py from this repo:

• https://github.com/hugovk/test/tree/ujson-manylinux-benchmark
• https://github.com/hugovk/test/actions/runs/27496464712/job/81271339436

Uses wheels from:

• manylinux_2_28: https://github.com/ultrajson/ultrajson/actions/runs/27460510633
• manylinux2014: https://github.com/hugovk/ultrajson/actions/runs/27471254973

Here's a Claude summary:

Benchmark: manylinux2014 (gcc 10) vs manylinux_2_28 (gcc 14)

Method: benchmarked ujson wheels built from the same C source with the two images, so the only variable is the compiler. Both wheel sets came straight from CI:

Source parity verified — git diff of lib/, python/, deps/, *.c, *.h, setup.py between the two commits is empty.

Each wheel was loaded into the same interpreter on the same runner and run through tests/benchmark.py, across a matrix of {x86_64, aarch64} × {CPython 3.10–3.14, 3.14t} (12 jobs, all green).

Results — geomean speed-up (new / old), 17 metrics per job

Overall geomean: +8.8%, consistent across both arches and every Python version.

The gains are concentrated on encode paths — string-array encode ~2×, UTF-8 encode ~+50%, dict encode ~+23% — while decode is milder (~3–12%). A handful of metrics dip just under 1.0 (noise); no systematic regression. The pattern matches a separate local aarch64 run, so it's real codegen (newer gcc auto-vectorising the escape/copy loops), not measurement noise.

Takeaway

Re-adding manylinux2014 restores broad compatibility (glibc 2.17) but costs this ~8–9% encode-heavy speed-up. So building both image variants is worth considering: old systems get a working wheel, and modern systems (pip prefers the higher platform tag) automatically pick up the faster manylinux_2_28 build — at the cost of roughly doubling the Linux wheel count.

Yeah, I think that's big enough a difference to warrant doing both.

New benchmark, wheels from https://github.com/hugovk/ultrajson/actions/runs/27506564289, run in https://github.com/hugovk/test/actions/runs/27507091606.

Claude summary:

Benchmark: manylinux2014 (gcc 10) vs manylinux_2_28 (gcc 14)

Geomean speed-up (new / old), 17 metrics per job

Overall geomean: +9.3%, every job between +6% and +12%.

Gains concentrate on encode paths — string-array encode ~2× (per-benchmark max +215%), UTF-8 encode ~+50%, dict encode ~+23% — while decode is milder (~3–12%). A few metrics dip just under 1.0 (noise); no systematic regression. A local aarch64 run (Docker, 3.11) showed the same pattern, so it's real codegen (newer gcc auto-vectorising the escape/copy loops), not measurement noise.

Takeaway

Re-adding manylinux2014 restores glibc 2.17 compatibility but costs this ~9% encode-heavy speed-up. Building both variants gets the best of both: old systems get a working wheel, and modern systems automatically pick up the faster manylinux_2_28 build via the higher platform tag — at the cost of ~16 extra Linux wheels (here 77 → 93 dists).

And the manylinux2014 wheels are similar in size to the others, around 55k each.