Skip to content

fix: search path related bugs#7873

Merged
mhuisi merged 10 commits intoleanprover:masterfrom
mhuisi:mhuisi/search-path-mess
Apr 9, 2025
Merged

fix: search path related bugs#7873
mhuisi merged 10 commits intoleanprover:masterfrom
mhuisi:mhuisi/search-path-mess

Conversation

@mhuisi
Copy link
Copy Markdown
Contributor

@mhuisi mhuisi commented Apr 8, 2025

This PR fixes a number of bugs related to the handling of the source search path in the language server, where deleting files could cause several features to stop functioning and both untitled files and files that don't exist on disc could have conflicting module names.

In detail, it makes the following adjustments:

  • The URI <-> module name conversion was adjusted to produce no name collisions.
    • File URIs in the search path yield a module name relative to the search path, as before.
    • File URIs not in the search path, non-file URIs and non-.lean files yield a «external:<full uri>» module name.
  • To avoid the issue of the URI -> module name conversion failing when a file is deleted from disc, we now cache the result of this conversion in the watchdog and the file worker when the file is first opened.
  • All of the URI <-> module name conversions now consistently go through Server.documentUriFromModule? and moduleFromDocumentUri to ensure that we don't have minor deviations for this conversion all over the place.
  • The threading of the source search path through the file worker (from lake setup-file) is removed. It turns out that lake serve already sets the correct source search path in the environment, so we can just always use the search path from the environment.
  • Since we can now answer more requests that need the .ileans in untitled files, a lot of the tests that test 'Go to definition' needed to be adjusted so that they use the information from the watchdog, not the file worker. As we load references asynchronously, this PR adds an internal $/lean/waitForILeans request that tests can use to wait for all .ilean files to be loaded and for the ilean references from the file worker for the current document version to be finalized.
  • As part of this PR, we noticed that the .ileans aren't available in the NixOS setup, so @Kha adjusted the Nix CI to fix this.

Breaking changes

  • Server.documentUriFromModule has been renamed to Server.documentUriFromModule? and doesn't take a SearchPath argument anymore, as the SearchPath is now computed from the LEAN_SRC_PATH environment variable. It has also been moved from Lean.Server.GoTo to Lean.Server.Utils.
  • Server.moduleFromDocumentUri does not take a SearchPath argument anymore and won't return an Option anymore. It has also been moved from Lean.Server.GoTo to Lean.Server.Utils.
  • The System.SearchPath.searchModuleNameOfUri function has been removed. It is recommended to use Server.moduleFromDocumentUri instead.
  • The initSrcSearchPath function has been renamed to getSrcSearchPath and has been moved from Lean.Util.Paths to Lean.Util.Path. It also doesn't need to take a pkgSearchPath argument anymore.

@github-actions github-actions bot added the toolchain-available A toolchain is available for this PR, at leanprover/lean4-pr-releases:pr-release-NNNN label Apr 8, 2025
@ghost
Copy link
Copy Markdown

ghost commented Apr 8, 2025

Mathlib CI status (docs):

  • ❗ Batteries/Mathlib CI will not be attempted unless your PR branches off the nightly-with-mathlib branch. Try git rebase acd6b13d7692c2bf2314c09e406a178407d02743 --onto bfed22330660045ea0641543a969ce79d1ef65b0. You can force Mathlib CI using the force-mathlib-ci label. (2025-04-08 14:31:30)
  • ❗ Batteries/Mathlib CI will not be attempted unless your PR branches off the nightly-with-mathlib branch. Try git rebase acd6b13d7692c2bf2314c09e406a178407d02743 --onto 388b6f045bd6d79df9b503d55eef6688ceb8f647. You can force Mathlib CI using the force-mathlib-ci label. (2025-04-09 14:40:12)
  • 🟡 Mathlib branch lean-pr-testing-7873 build this PR didn't complete normally. (2025-04-10 06:44:43) View Log

@mhuisi mhuisi force-pushed the mhuisi/search-path-mess branch from 797129d to 5607111 Compare April 8, 2025 15:55
@mhuisi mhuisi added changelog-server Language server, widgets, and IDE extensions merge-ci Enable merge queue CI checks for PR. In particular, produce artifacts for all major platforms. labels Apr 8, 2025
@mhuisi
Copy link
Copy Markdown
Contributor Author

mhuisi commented Apr 9, 2025

!bench

@leanprover-bot
Copy link
Copy Markdown
Collaborator

Here are the benchmark results for commit e1c9ee4.
There were significant changes against commit acd6b13:

  Benchmark                            Metric                    Change
  ===============================================================================
- Init.Data.List.Sublist async         task-clock                  2.8%  (14.2 σ)
- Init.Data.List.Sublist re-elab -j4   wall-clock                  3.2%  (10.2 σ)
- Init.Prelude async                   wall-clock                  2.8%  (33.1 σ)
+ bv_decide_mul                        maxrss                     -2.2% (-27.5 σ)
- bv_decide_realworld                  wall-clock                  1.7%  (13.5 σ)
+ lake build no-op                     wall-clock                 -1.6% (-10.7 σ)
- omega_stress.lean async              task-clock                  3.5%  (10.5 σ)
- omega_stress.lean async              wall-clock                  3.9%  (13.4 σ)
- qsort                                task-clock                 10.4%  (69.3 σ)
- qsort                                wall-clock                 10.4%  (68.9 σ)
+ rbmap                                task-clock                 -3.5% (-10.7 σ)
+ rbmap                                wall-clock                 -3.5% (-11.0 σ)
- rbmap_fbip                           task-clock                  2.7%  (10.9 σ)
- rbmap_fbip                           wall-clock                  2.7%  (10.7 σ)
- reduceMatch                          maxrss                      2.8%  (35.9 σ)
- reduceMatch                          task-clock                  1.5%  (32.6 σ)
+ simp_arith1                          task-clock                 -3.4% (-11.2 σ)
- stdlib                               instantiate metavars        2.8%  (10.7 σ)
+ stdlib                               maxrss                     -2.2% (-62.4 σ)
- stdlib                               process pre-definitions     1.0%  (10.2 σ)
- stdlib                               type checking               1.6%  (10.9 σ)
- stdlib                               wall-clock                  1.8%  (20.0 σ)
+ unionfind                            task-clock                 -7.7% (-12.5 σ)
+ unionfind                            wall-clock                 -7.7% (-12.3 σ)

@mhuisi mhuisi enabled auto-merge April 9, 2025 15:20
@mhuisi mhuisi added this pull request to the merge queue Apr 9, 2025
Merged via the queue into leanprover:master with commit 2ede81f Apr 9, 2025
16 checks passed
kim-em added a commit to leanprover-community/batteries that referenced this pull request Apr 10, 2025
kim-em added a commit to leanprover-community/import-graph that referenced this pull request Apr 10, 2025
kim-em added a commit to leanprover-community/mathlib4 that referenced this pull request Apr 10, 2025
kmill added a commit to leanprover-community/mathlib4 that referenced this pull request Apr 19, 2025
* fixes

* fix test

* fix

* chore: adaptations for leanprover/lean4#7797

* stricter rfl check

* fix merge

* Trigger CI for leanprover/lean4#7797

* Trigger CI for leanprover/lean4#7797

* Trigger CI for leanprover/lean4#7797

* inverse construction

* Update lean-toolchain for testing leanprover/lean4#7791

* lake update

* Fix

* Fix

* Fix

* chore: bump to nightly-2025-04-03

* more fixes

* merge lean-pr-testing-6325

* bump proofwidgets

* Update lean-toolchain for testing leanprover/lean4#7812

* shake

* fix shake

* fix tests

* import mathlib.init

* Update lean-toolchain for testing leanprover/lean4#7816

* Trigger CI for leanprover/lean4#7816

* Update lean-toolchain for testing leanprover/lean4#7802

* Fix

* fixes

* Fix (pending upstream changes)

* Trigger CI for leanprover/lean4#7802

* fix

* chore: bump to nightly-2025-04-04

* Fix

* Fix

* Fix

* Fix

* Fix

* Fix

* Update lean-toolchain for testing leanprover/lean4#7818

* Fix

* Fix

* Fix

* Merge master into nightly-testing

* Fix

* Fix comments

* fixes

* Trigger CI for leanprover/lean4#7816

* fix

* fix

* missing doc-string?

* chore: bump to nightly-2025-04-05

* chore: bump to nightly-2025-04-06

* move Batteries to nightly-testing

* lake update

* lint

* fix

* chore: bump to nightly-2025-04-07

* sprinkle noncomputable

* fix: relativize file paths

fix for leanprover/lean4#7822

* chore: reset cache

* chore: bump to nightly-2025-04-07

* shake

* fix: relativize more

* Update lean-toolchain for testing leanprover/lean4#7856

* Update lean-toolchain for testing leanprover/lean4#7851

* Fix

* Fix

* Fix

* Fix

* chore: bump to nightly-2025-04-08

* Update lean-toolchain for testing leanprover/lean4#7870

* Trigger CI for leanprover/lean4#7870

* Trigger CI for leanprover/lean4#7870

* Trigger CI for leanprover/lean4#7870

* wip

* Trigger CI for leanprover/lean4#7870

* .

* chore: bump to nightly-2025-04-09

* Trigger CI for leanprover/lean4#7870

* .

* bump

* lake update

* fixes for leanprover/lean4#7873

* fixes

* deprecations

* chore: bump to nightly-2025-04-10

* chore: `Option.zipWith` -> `Option.merge`

* lake update

* toolchain

* fix

* fix

* Update lean-toolchain for testing leanprover/lean4#7897

* Trigger CI for leanprover/lean4#7897

* fix

* lake update and fix

* Fix

* Fix

* Trigger CI for leanprover/lean4#7897

* fix merge

* chore: adaptations for nightly-2025-04-10

* chore: bump to nightly-2025-04-14

* fix

* .

* annotate changed goal state

* fixes (adaptation notes)

* cleanup imports

* cleanup

* Update lean-toolchain for testing leanprover/lean4#7933

* Fix

* Fix

* Update lean-toolchain for testing leanprover/lean4#7975

* remove adaptation notes

* chore: bump to nightly-2025-04-16

---------

Co-authored-by: leanprover-community-mathlib4-bot <leanprover-community-mathlib4-bot@users.noreply.github.com>
Co-authored-by: Kim Morrison <kim@tqft.net>
Co-authored-by: Markus Himmel <markus@lean-fro.org>
Co-authored-by: Rob23oba <robin.arnez@web.de>
Co-authored-by: Mario Carneiro <di.gama@gmail.com>
Co-authored-by: Johan Commelin <johan@commelin.net>
Co-authored-by: github-actions <github-actions@github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

changelog-server Language server, widgets, and IDE extensions merge-ci Enable merge queue CI checks for PR. In particular, produce artifacts for all major platforms. toolchain-available A toolchain is available for this PR, at leanprover/lean4-pr-releases:pr-release-NNNN

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants