Skip to content

refactor: reduce omega's dependency on fvar IDs#9723

Merged
datokrat merged 5 commits intomasterfrom
paul/temp/big-omega
Aug 11, 2025
Merged

refactor: reduce omega's dependency on fvar IDs#9723
datokrat merged 5 commits intomasterfrom
paul/temp/big-omega

Conversation

@datokrat
Copy link
Copy Markdown
Contributor

@datokrat datokrat commented Aug 5, 2025

This PR replaces some HashSet Expr-typed collections of facts in omega's implementation with plain lists. This change makes some omega calls faster, some slower, but the advantage is that omega's performance is more independent the state of the name generator that produces fvar IDs.

I've created this PR for discussion and am happy to hear opinions on whether this should be merged or not. A good reason not to merge is that it causes regressions in some places and grind is expected to supersede omega either way. A good reason to merge is that omega is used all over the place and its flaky performance increases the noise in future benchmarks.

@datokrat
Copy link
Copy Markdown
Contributor Author

datokrat commented Aug 5, 2025

!bench

@leanprover-bot
Copy link
Copy Markdown
Collaborator

Here are the benchmark results for commit ef1cb71.
There were significant changes against commit 95e753c:

  Benchmark                              Metric                  Change
  ================================================================================
- Init.Data.BitVec.Lemmas                branches                  2.2%   (80.5 σ)
- Init.Data.BitVec.Lemmas                instructions              2.0%   (66.0 σ)
- Init.Data.List.Sublist async           branches                  1.5%   (52.5 σ)
- Init.Data.List.Sublist async           instructions              1.1%   (34.0 σ)
- Init.Prelude async                     branches                  1.2%   (89.3 σ)
- Std.Data.DHashMap.Internal.RawLemmas   branches                  1.5%  (151.7 σ)
+ Std.Data.DHashMap.Internal.RawLemmas   task-clock              -16.6%  (-42.6 σ)
+ Std.Data.DHashMap.Internal.RawLemmas   wall-clock              -14.6%  (-27.1 σ)
- Std.Data.Internal.List.Associative     branches                  1.5%  (103.5 σ)
+ Std.Data.Internal.List.Associative     wall-clock              -15.4%  (-21.5 σ)
- big_do                                 branches                  1.1%  (133.1 σ)
- big_omega.lean                         branches                  1.2%  (221.6 σ)
- big_omega.lean MT                      branches                  1.2%  (219.2 σ)
- binarytrees                            instructions              1.1%  (190.5 σ)
- binarytrees.st                         instructions              1.1%  (527.6 σ)
+ bv_decide_inequality.lean              task-clock               -6.9%  (-30.3 σ)
+ bv_decide_inequality.lean              wall-clock               -6.8%  (-21.0 σ)
- bv_decide_large_aig.lean               wall-clock                3.9%   (30.0 σ)
+ bv_decide_mod                          task-clock               -6.8%  (-52.4 σ)
+ bv_decide_mod                          wall-clock               -6.6%  (-35.8 σ)
- bv_decide_rewriter.lean                branches                  1.0%   (45.8 σ)
+ channel.lean                           bounded0_mpsc            -9.3%  (-58.5 σ)
+ channel.lean                           bounded1_mpmc           -34.8% (-107.0 σ)
+ channel.lean                           bounded1_mpsc           -26.6%  (-28.4 σ)
+ channel.lean                           boundedn_seq             -2.7%
+ channel.lean                           task-clock              -22.5%  (-30.2 σ)
+ channel.lean                           wall-clock              -22.6%  (-26.6 σ)
- grind_bitvec2.lean                     branches                  1.5%   (76.7 σ)
- grind_bitvec2.lean                     instructions              1.3%   (57.7 σ)
+ grind_bitvec2.lean                     task-clock               -7.7% (-169.4 σ)
- grind_list2.lean                       branches                  1.6%   (79.1 σ)
- grind_list2.lean                       instructions              1.2%   (51.9 σ)
- grind_ring_5.lean                      branches                  2.5%  (804.3 σ)
- grind_ring_5.lean                      instructions              3.1% (1204.3 σ)
- grind_ring_5.lean                      task-clock                8.0%   (24.0 σ)
- grind_ring_5.lean                      wall-clock                8.0%   (23.5 σ)
- hashmap.lean                           branch-misses             3.5%   (28.6 σ)
- hashmap.lean                           branches                  1.6%  (941.3 σ)
+ hashmap.lean                           insertIfNewHit           -2.3%
- lake startup                           instructions              1.0%   (22.8 σ)
- language server startup                wall-clock                2.3%   (37.8 σ)
+ nat_repr                               branches                 -5.6% (-805.4 σ)
+ nat_repr                               instructions             -5.5% (-740.4 σ)
- omega_stress.lean async                branches                  1.7%   (45.0 σ)
- omega_stress.lean async                instructions              1.7%   (40.4 σ)
- phashmap.lean                          branches                  1.3%  (720.9 σ)
+ phashmap.lean                          containsMiss             -5.3%
+ phashmap.lean                          insertHit                -4.5%
+ phashmap.lean                          iterate                  -2.4%
+ phashmap.lean                          task-clock               -5.1%  (-24.8 σ)
+ phashmap.lean                          wall-clock               -5.1%  (-24.7 σ)
+ rbmap_1                                instructions             -1.4% (-478.1 σ)
- rbmap_fbip                             instructions              1.8%  (196.1 σ)
- reduceMatch                            instructions              2.0%  (235.9 σ)
- riscv-ast.lean                         branches                  1.3%   (95.1 σ)
- riscv-ast.lean                         instructions              1.2%  (520.4 σ)
- simp_bubblesort_256                    branches                  1.3%   (40.8 σ)
- simp_local                             branches                  1.3%  (249.3 σ)
- simp_subexpr                           branches                  1.2%   (29.5 σ)
- stdlib                                 attribute application     5.9%  (120.0 σ)
- unionfind                              instructions              1.2%  (399.7 σ)

@github-actions github-actions bot added the toolchain-available A toolchain is available for this PR, at leanprover/lean4-pr-releases:pr-release-NNNN label Aug 5, 2025
@leanprover-bot
Copy link
Copy Markdown
Collaborator

leanprover-bot commented Aug 5, 2025

Reference manual CI status:

  • ❗ Reference manual CI can not be attempted yet, as the nightly-testing-2025-07-27 tag does not exist there yet. We will retry when you push more commits. If you rebase your branch onto nightly-with-manual, reference manual CI should run now. You can force reference manual CI using the force-manual-ci label. (2025-08-05 09:54:04)
  • ❗ Reference manual CI will not be attempted unless your PR branches off the nightly-with-manual branch. Try git rebase 09e8079ea3a7dd9a57cae5b417c7f2d84c8fcb81 --onto d3dda9f6d4428a906c096067ecb75e432afc4615. You can force reference manual CI using the force-manual-ci label. (2025-08-05 13:25:28)
  • ❗ Reference manual CI will not be attempted unless your PR branches off the nightly-with-manual branch. Try git rebase ed860dfa23acab2178686d85f463101f5adbddf1 --onto d3dda9f6d4428a906c096067ecb75e432afc4615. You can force reference manual CI using the force-manual-ci label. (2025-08-05 16:17:02)

ghost pushed a commit to leanprover-community/batteries that referenced this pull request Aug 5, 2025
ghost pushed a commit to leanprover-community/mathlib4-nightly-testing that referenced this pull request Aug 5, 2025
@ghost ghost added the builds-mathlib CI has verified that Mathlib builds against this PR label Aug 5, 2025
@ghost
Copy link
Copy Markdown

ghost commented Aug 5, 2025

Mathlib CI status (docs):

  • ✅ Mathlib branch lean-pr-testing-9723 has successfully built against this PR. (2025-08-05 10:46:34) View Log
  • ❗ Batteries/Mathlib CI will not be attempted unless your PR branches off the nightly-with-mathlib branch. Try git rebase 09e8079ea3a7dd9a57cae5b417c7f2d84c8fcb81 --onto c260435913353a138e3d463b533d1449de0d0062. You can force Mathlib CI using the force-mathlib-ci label. (2025-08-05 13:25:27)
  • ❗ Batteries/Mathlib CI will not be attempted unless your PR branches off the nightly-with-mathlib branch. Try git rebase ed860dfa23acab2178686d85f463101f5adbddf1 --onto c260435913353a138e3d463b533d1449de0d0062. You can force Mathlib CI using the force-mathlib-ci label. (2025-08-05 16:17:01)

@datokrat datokrat marked this pull request as ready for review August 5, 2025 12:25
@datokrat datokrat requested a review from kim-em as a code owner August 5, 2025 12:25
@datokrat datokrat force-pushed the paul/temp/big-omega branch from ef1cb71 to b0f3f1e Compare August 5, 2025 12:37
@datokrat
Copy link
Copy Markdown
Contributor Author

datokrat commented Aug 5, 2025

!bench

@leanprover-bot
Copy link
Copy Markdown
Collaborator

Here are the benchmark results for commit b0f3f1e.Found no runs to compare against.

@datokrat
Copy link
Copy Markdown
Contributor Author

datokrat commented Aug 5, 2025

No benchmarks have been run for the merge base commit, so here's a comparison against the previous commit: https://speed.lean-lang.org/lean4/compare/c7cfda2e-49a6-4e52-a268-f106880e0cde/to/bab50d33-13a1-40c4-85a0-fda5f93f8b22?hash2=b0f3f1eb42cc228d10c0519b6e5e9cea3c8fa2a2

@datokrat datokrat force-pushed the paul/temp/big-omega branch from b0f3f1e to 1559cd4 Compare August 5, 2025 15:32
@datokrat
Copy link
Copy Markdown
Contributor Author

datokrat commented Aug 5, 2025

!bench

@leanprover-bot
Copy link
Copy Markdown
Collaborator

Here are the benchmark results for commit 1559cd4.
There were significant changes against commit ed860df:

  Benchmark                 Metric         Change
  =========================================================
- big_omega.lean            branches         6.9% (673.6 σ)
- big_omega.lean            instructions     7.2% (789.0 σ)
- big_omega.lean MT         branches         7.0% (190.8 σ)
- big_omega.lean MT         instructions     7.2% (175.6 σ)
- channel.lean              boundedn_seq     1.1%
+ omega_stress.lean async   branches        -2.0% (-30.9 σ)
+ omega_stress.lean async   instructions    -2.0% (-29.6 σ)
+ treemap.lean              containsHit     -5.6%
+ treemap.lean              containsMiss    -4.1%

@datokrat datokrat changed the title chore: reduce omega's dependency on fvar IDs refactor: reduce omega's dependency on fvar IDs Aug 6, 2025
@datokrat datokrat added this pull request to the merge queue Aug 11, 2025
Merged via the queue into master with commit f15d531 Aug 11, 2025
19 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

builds-mathlib CI has verified that Mathlib builds against this PR toolchain-available A toolchain is available for this PR, at leanprover/lean4-pr-releases:pr-release-NNNN

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants