Skip to content

fix: only return new mvars from refine, elabTermWithHoles, and withCollectingNewGoalsFrom#2502

Merged
kim-em merged 13 commits intoleanprover:masterfrom
thorimur:refine-filter-old-mvars-strict
Sep 21, 2023
Merged

fix: only return new mvars from refine, elabTermWithHoles, and withCollectingNewGoalsFrom#2502
kim-em merged 13 commits intoleanprover:masterfrom
thorimur:refine-filter-old-mvars-strict

Conversation

@thorimur
Copy link
Copy Markdown
Contributor

@thorimur thorimur commented Sep 1, 2023

Fixes #2495 and #2434 by making sure that withCollectingNewGoals only returns mvarIds created during elaboration. This changes the behavior of elabTermWithHoles and refine accordingly: refine e now replaces the main goal only with goals that were created during elaboration of e (and does not include goals that appear in e but existed prior to its elaboration).

Consequences of this change:

  • This prevents duplication of goals in the infoview by refine (bug: refine duplicates goals in the infoview #2495): pre-existing goals are now excluded from the resulting tactic state, and thus don't appear twice.
  • It also prevents "goal tunneling", wherein a goal that was created before a given refine statement manages to be included in its resulting goals unexpectedly, due to being present in e.g. an implicit argument.
  • Previously, refine was inconsistent in which pre-existing goals were returned; this is the content of issue bug: refine loses track of natural goals #2434, which points out that pre-existing natural mvars are not returned, while synthetic mvars are. refine's occurs check relied on all mvars appearing in the expression being returned, and so goals were erroneously closed. This renders that issue moot by uniformly not returning any old mvars, and updating the occurs check accordingly.
  • This PR breaks mathlib, and requires fixes to a few proofs that use refine and convert (which relies on withCollectingNewGoalsFrom). All breakages are instances of goal tunneling, and are easily fixed. These fixes are done in bump: lean4: account for leanprover/lean4#2502 leanprover-community/mathlib4#6928. Note, however, that it breaks mathlib less than an incremental fix of bug: refine loses track of natural goals #2434 alone, which required us to filter out old mvars anyway in change.
  • Finally, metaprogrammers likely expect elabTermWithHoles and withCollectingNewGoalsFrom to behave in this way already—see zulip for an expression of this sentiment.

Note also that this streamlines the code of withCollectingNewGoalsFrom, replacing

    let newMVarIds ← if allowNaturalHoles then
      pure newMVarIds.toList
    else
      let naturalMVarIds ← newMVarIds.filterM fun mvarId => return (← mvarId.getKind).isNatural
      let syntheticMVarIds ← newMVarIds.filterM fun mvarId => return !(← mvarId.getKind).isNatural
      let naturalMVarIds ← filterOldMVars naturalMVarIds mvarCounterSaved
      logUnassignedAndAbort naturalMVarIds
      pure syntheticMVarIds.toList

with simply

    let newMVarIds ← filterOldMVars newMVarIds mvarCounterSaved
    unless allowNaturalHoles do
      let naturalMVarIds ← newMVarIds.filterM fun mvarId => return (← mvarId.getKind).isNatural
      logUnassignedAndAbort naturalMVarIds

(plus comments).

Changes made by this PR

  • prevents withCollectingNewGoalsFrom, and therefore elabTermWithHoles, refine, and specialize from returning old goals

  • updates the occurs check in refine to compensate for this change (note that specialize does not have an occurs check; this is a separate issue)

  • adds a missing docstring to elabTermWithHoles

  • adds tests for bug: refine duplicates goals in the infoview #2495, goal tunneling, and bug: refine loses track of natural goals #2434

  • changes test run/492.lean, which made incidental use of goal tunneling


@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Sep 1, 2023

Thanks for your contribution! Please make sure to follow our Commit Convention.

@thorimur
Copy link
Copy Markdown
Contributor Author

thorimur commented Sep 1, 2023

WIP

@github-actions github-actions bot added WIP This is work in progress, and only a PR for the sake of CI or sharing. No review yet. toolchain-available A toolchain is available for this PR, at leanprover/lean4-pr-releases:pr-release-NNNN labels Sep 1, 2023
ghost pushed a commit to leanprover-community/mathlib4 that referenced this pull request Sep 1, 2023
@ghost ghost added the breaks-mathlib This is not necessarily a blocker for merging: but there needs to be a plan label Sep 1, 2023
@ghost
Copy link
Copy Markdown

ghost commented Sep 1, 2023

@kim-em kim-em removed the breaks-mathlib This is not necessarily a blocker for merging: but there needs to be a plan label Sep 1, 2023
@thorimur thorimur changed the title fix: make refine, elabTermWithHoles only return new mvars fix: only return new mvars from refine, elabTermWithHoles, and withCollectingNewGoalsFrom Sep 2, 2023
ghost pushed a commit to leanprover-community/mathlib4 that referenced this pull request Sep 2, 2023
@ghost ghost added the breaks-mathlib This is not necessarily a blocker for merging: but there needs to be a plan label Sep 2, 2023
@thorimur
Copy link
Copy Markdown
Contributor Author

thorimur commented Sep 3, 2023

awaiting-review

@github-actions github-actions bot added awaiting-review Waiting for someone to review the PR and removed WIP This is work in progress, and only a PR for the sake of CI or sharing. No review yet. labels Sep 3, 2023
@kim-em
Copy link
Copy Markdown
Collaborator

kim-em commented Sep 4, 2023

Could you merge your Mathlib fixes to the lean-pr-testing-2502 branch, so we can hopefully get an automatically generated builds-mathlib label on this PR?

@ghost ghost added builds-mathlib CI has verified that Mathlib builds against this PR and removed breaks-mathlib This is not necessarily a blocker for merging: but there needs to be a plan labels Sep 4, 2023
ghost pushed a commit to leanprover-community/mathlib4 that referenced this pull request Sep 4, 2023
@thorimur thorimur marked this pull request as ready for review September 6, 2023 00:23
ghost pushed a commit to leanprover-community/mathlib4 that referenced this pull request Sep 20, 2023
@kim-em kim-em merged commit e79370a into leanprover:master Sep 21, 2023
thorimur added a commit to thorimur/lean4 that referenced this pull request Sep 30, 2023
thorimur added a commit to thorimur/lean4 that referenced this pull request Sep 30, 2023
thorimur added a commit to thorimur/lean4 that referenced this pull request Sep 30, 2023
thorimur added a commit to thorimur/lean4 that referenced this pull request Sep 30, 2023
thorimur added a commit to thorimur/lean4 that referenced this pull request Sep 30, 2023
thorimur added a commit to thorimur/lean4 that referenced this pull request Sep 30, 2023
kim-em pushed a commit that referenced this pull request Oct 2, 2023
github-actions bot pushed a commit that referenced this pull request Oct 2, 2023
kim-em pushed a commit that referenced this pull request Oct 2, 2023
(cherry picked from commit 8c0f0b5)

Co-authored-by: thorimur <68410468+thorimur@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

awaiting-review Waiting for someone to review the PR builds-mathlib CI has verified that Mathlib builds against this PR toolchain-available A toolchain is available for this PR, at leanprover/lean4-pr-releases:pr-release-NNNN

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bug: refine duplicates goals in the infoview bug: refine loses track of natural goals

2 participants