Unicode data: reduce size of to_lower/to_upper tables by Kmeakin · Pull Request #152954 · rust-lang/rust

Kmeakin · 2026-02-22T05:21:29Z

Reduces the combined size of to_lower and to_upper from 25,364 bytes to 3,110 bytes. Explained in detail in the doc comments

rustbot · 2026-02-22T05:21:34Z

r? @Mark-Simulacrum

rustbot has assigned @Mark-Simulacrum.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

Why was this reviewer chosen?

The reviewer was selected based on:

Owners of files modified in this PR: bootstrap
bootstrap expanded to 6 candidates
Random selection from Mark-Simulacrum, clubby789, jieyouxu

Mark-Simulacrum

Will need to review the actual implementation as well, but would like more context on the algorithm selection before doing so.

View changes since this review

src/tools/unicode-table-generator/src/case_mapping.rs

rustbot · 2026-02-23T01:17:39Z

Reminder, once the PR becomes ready for a review, use @rustbot ready.

Kobzol · 2026-02-23T10:49:40Z

@bors try @rust-timer queue

Unicode data: reduce size of to_lower/to_upper tables

rust-bors · 2026-02-23T13:01:49Z

☀️ Try build successful (CI)
Build commit: dcce5a6 (dcce5a661ada2ae2006af256bf8e3b0d882272d7, parent: eeb94be79adc9df7a09ad0b2421f16e60e6d932c)

rust-timer · 2026-02-23T13:42:10Z

Finished benchmarking commit (dcce5a6): comparison URL.

Overall result: no relevant changes - no action needed

Benchmarking this pull request means it may be perf-sensitive – we'll automatically label it not fit for rolling up. You can override this, but we strongly advise not to, due to possible changes in compiler perf.

@bors rollup=never
@rustbot label: -S-waiting-on-perf -perf-regression

Instruction count

This benchmark run did not return any relevant results for this metric.

Max RSS (memory usage)

Results (secondary -4.9%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-4.9%	[-5.3%, -4.5%]	2
All ❌✅ (primary)	-	-	0

Cycles

Results (secondary -4.9%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-4.9%	[-4.9%, -4.9%]	1
All ❌✅ (primary)	-	-	0

Binary size

Results (primary -0.2%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-0.2%	[-0.3%, -0.2%]	4
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	-0.2%	[-0.3%, -0.2%]	4

Bootstrap: 481.498s -> 480.969s (-0.11%)
Artifact size: 395.88 MiB -> 397.86 MiB (0.50%)

Kmeakin · 2026-03-05T01:18:31Z

@rustbot ready

Mark-Simulacrum

Probably can squash the commits at least a little? r=me with that and the nit fixed.

View changes since this review

src/tools/unicode-table-generator/src/case_mapping.rs

Instead of generating a standalone executable to test `unicode_data`, generate normal tests in `coretests`. This ensures tests are always generated, and will be run as part of the normal testsuite. Also change the generated tests to loop over lookup tables, rather than generating a separate `assert_eq!()` statement for every codepoint. The old approach produced a massive (20,000 lines plus) file which took minutes to compile!

Add a doc-comment to the top of `case-mapping.rs`.

rustbot · 2026-03-08T22:40:31Z

This PR was rebased onto a different main commit. Here's a range-diff highlighting what actually changed.

Rebasing is a normal part of keeping PRs up to date, so no action is needed—this note is just to help reviewers.

Kmeakin · 2026-03-08T22:41:53Z

r=@Mark-Simulacrum

ognevny · 2026-03-09T05:48:37Z

r=@Mark-Simulacrum

that should be @bors r=Mark-Simulacrum :)

Kmeakin · 2026-03-09T19:05:24Z

@bors r=Mark-Simulacrum

rust-bors · 2026-03-09T19:09:18Z

@Kmeakin: 🔑 Insufficient privileges: not in review users

Mark-Simulacrum · 2026-03-10T02:11:06Z

@bors r+

rust-bors · 2026-03-10T02:11:09Z

📌 Commit 902199b has been approved by Mark-Simulacrum

It is now in the queue for this repository.

rust-bors · 2026-03-10T06:20:11Z

⌛ Testing commit 902199b with merge 3bc6ea5...

Workflow: https://github.com/rust-lang/rust/actions/runs/22889887298

…Simulacrum Unicode data: reduce size of to_lower/to_upper tables Reduces the combined size of to_lower and to_upper from 25,364 bytes to 3,110 bytes. Explained in detail in the doc comments

rustbot assigned Mark-Simulacrum Feb 22, 2026

rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) T-libs Relevant to the library team, which will review and decide on the PR/issue. labels Feb 22, 2026

Mark-Simulacrum requested changes Feb 23, 2026

View reviewed changes

src/tools/unicode-table-generator/src/case_mapping.rs Outdated Show resolved Hide resolved

rustbot added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Feb 23, 2026

This comment has been minimized.

Sign in to view

rustbot added the S-waiting-on-perf Status: Waiting on a perf run to be completed. label Feb 23, 2026

This comment has been minimized.

Sign in to view

rust-bors bot pushed a commit that referenced this pull request Feb 23, 2026

Auto merge of #152954 - Kmeakin:km/unicode-data/case-mapping, r=<try>

dcce5a6

Unicode data: reduce size of to_lower/to_upper tables

This comment has been minimized.

Sign in to view

rustbot removed the S-waiting-on-perf Status: Waiting on a perf run to be completed. label Feb 23, 2026

Kmeakin force-pushed the km/unicode-data/case-mapping branch 2 times, most recently from 51870c1 to abe23a5 Compare March 4, 2026 00:26

This comment has been minimized.

Sign in to view

Kmeakin force-pushed the km/unicode-data/case-mapping branch 2 times, most recently from 23fdbc4 to 3f4a072 Compare March 5, 2026 01:13

rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Mar 5, 2026

Kmeakin requested a review from Mark-Simulacrum March 6, 2026 13:23

Kmeakin force-pushed the km/unicode-data/case-mapping branch from 3f4a072 to eb10a54 Compare March 6, 2026 21:33

Mark-Simulacrum approved these changes Mar 8, 2026

View reviewed changes

src/tools/unicode-table-generator/src/case_mapping.rs Outdated Show resolved Hide resolved

Kmeakin added 2 commits March 8, 2026 22:38

Document the case-mapping lookup tables

724a36d

Add a doc-comment to the top of `case-mapping.rs`.

Compress case-mapping tables

902199b

Kmeakin force-pushed the km/unicode-data/case-mapping branch from eb10a54 to 902199b Compare March 8, 2026 22:40

rust-bors bot added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Mar 10, 2026

Uh oh!

Conversation

Kmeakin commented Feb 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

rustbot commented Feb 22, 2026

Uh oh!

Mark-Simulacrum left a comment • edited by rustbot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

rustbot commented Feb 23, 2026

Uh oh!

Kobzol commented Feb 23, 2026

Uh oh!

This comment has been minimized.

This comment has been minimized.

rust-bors bot commented Feb 23, 2026

Uh oh!

This comment has been minimized.

rust-timer commented Feb 23, 2026

Overall result: no relevant changes - no action needed

Instruction count

Max RSS (memory usage)

Cycles

Binary size

Uh oh!

This comment has been minimized.

Kmeakin commented Mar 5, 2026

Uh oh!

Mark-Simulacrum left a comment • edited by rustbot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

rustbot commented Mar 8, 2026

Uh oh!

Kmeakin commented Mar 8, 2026

Uh oh!

ognevny commented Mar 9, 2026

Uh oh!

Kmeakin commented Mar 9, 2026

Uh oh!

rust-bors bot commented Mar 9, 2026

Uh oh!

Mark-Simulacrum commented Mar 10, 2026

Uh oh!

rust-bors bot commented Mar 10, 2026

Uh oh!

rust-bors bot commented Mar 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

Kmeakin commented Feb 22, 2026 •

edited

Loading

Mark-Simulacrum left a comment •

edited by rustbot

Loading

Mark-Simulacrum left a comment •

edited by rustbot

Loading

rust-bors bot commented Mar 10, 2026 •

edited

Loading