Fix file reloading in dmypy with --export-types#16359
Merged
JukkaL merged 5 commits intopython:masterfrom Nov 1, 2023
Merged
Conversation
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
JukkaL
reviewed
Oct 30, 2023
| changed += [ | ||
| (bs.module, bs.path) | ||
| for bs in sources | ||
| if bs.path and (bs.module, bs.path) not in changed |
Collaborator
There was a problem hiding this comment.
This is quadratic, and it could be slow if there are many files and many that are changed. Maybe construct a temporary set from changed and use it in the not in changed check?
Member
Author
There was a problem hiding this comment.
Yes, good point, fixed now.
| changed += [ | ||
| (bs.module, bs.path) | ||
| for bs in sources | ||
| if bs.path and (bs.module, bs.path) not in changed |
Collaborator
There was a problem hiding this comment.
Similar to above. Maybe move this to a helper function to avoid code duplication?
Member
Author
There was a problem hiding this comment.
Moved to a helper function.
| options.use_fine_grained_cache = self.use_cache and not build_cache | ||
| options.cache_fine_grained = self.use_cache | ||
| options.local_partial_types = True | ||
| options.export_types = True |
Collaborator
There was a problem hiding this comment.
What's the purpose of this change?
Member
Author
There was a problem hiding this comment.
Starting the daemon with this option set to True is cleaner for inspect unit tests, I made this explicit now (so it will not slow down other fine grained tests).
|
|
||
| def run_check(self, server: Server, sources: list[BuildSource]) -> list[str]: | ||
| response = server.check(sources, export_types=True, is_tty=False, terminal_width=-1) | ||
| response = server.check(sources, export_types=False, is_tty=False, terminal_width=-1) |
Collaborator
There was a problem hiding this comment.
Hmm why do we have export_types=False here, but true above?
Member
Author
There was a problem hiding this comment.
Passing it explicitly on each run will force reloading files. Putting it in options from the start is IMO cleaner.
Contributor
|
Diff from mypy_primer, showing the effect of this PR on open source code: discord.py (https://github.com/Rapptz/discord.py): typechecking got 1.09x faster (138.5s -> 127.5s)
(Performance measurements are based on a single noisy sample)
|
JukkaL
pushed a commit
that referenced
this pull request
Nov 1, 2023
Fixes #15794 Unfortunately, this requires to pass `--export-types` to `dmypy run` if one wants to inspect a file that was previously kicked out of the build.
JukkaL
added a commit
that referenced
this pull request
Nov 9, 2023
The script format changelog entries based on commit history and has some rules to filter out some changes, such as typeshed sync and changes cherry-picked to the previous release branch. Example of how to run it: ``` $ python misc/generate_changelog.py 1.7 Generating changelog for 1.7 Previous release was 1.6 Merge base: d7b2451 NOTE: Drop "Fix crash on ParamSpec unification (for real)", since it was in previous release branch NOTE: Drop "Fix crash on ParamSpec unification", since it was in previous release branch NOTE: Drop "Fix mypyc regression with pretty", since it was in previous release branch NOTE: Drop "Clear cache when adding --new-type-inference", since it was in previous release branch NOTE: Drop "Match note error codes to import error codes", since it was in previous release branch NOTE: Drop "Make PEP 695 constructs give a reasonable error message", since it was in previous release branch NOTE: Drop "Fix ParamSpec inference for callback protocols", since it was in previous release branch NOTE: Drop "Try upgrading tox", since it was in previous release branch NOTE: Drop "Optimize Unpack for failures", since it was in previous release branch * Fix crash on unpack call special-casing (Ivan Levkivskyi, PR [16381](#16381)) * Fix file reloading in dmypy with --export-types (Ivan Levkivskyi, PR [16359](#16359)) * Fix daemon crash caused by deleted submodule (Jukka Lehtosalo, PR [16370](#16370)) ... ```
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #15794
Unfortunately, this requires to pass
--export-typestodmypy runif one wants to inspect a file that was previously kicked out of the build.