Skip to content

Document formatter error shrinking#5915

Merged
konstin merged 1 commit intomainfrom
Don_t_minimize_files_that_don_t_match_in_the_first_place
Jul 21, 2023
Merged

Document formatter error shrinking#5915
konstin merged 1 commit intomainfrom
Don_t_minimize_files_that_don_t_match_in_the_first_place

Conversation

@konstin
Copy link
Copy Markdown
Member

@konstin konstin commented Jul 20, 2023 •
edited
Loading

Summary

Don't minimize files that don't match in the first place This adds a sanity check to the minimizer script that the
input matches the condition (e.g. unstable formatting). Otherwise we run
through all checks with the whole file, which is extremely slow. It's
more reasonable for downstream usage to write an empty string to the
output file instead.

@konstin
Copy link
Copy Markdown
Member Author

konstin commented Jul 20, 2023 •
edited
Loading

Current dependencies on/for this PR:

This comment was auto-generated by Graphite.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Jul 20, 2023 •
edited
Loading

PR Check Results

Benchmark

Linux

group                                      main                                   pr
-----                                      ----                                   --
formatter/large/dataset.py                 1.00      9.8±0.02ms     4.2 MB/sec    1.00      9.8±0.02ms     4.2 MB/sec
formatter/numpy/ctypeslib.py               1.00   1905.9±5.15µs     8.7 MB/sec    1.00   1903.8±5.67µs     8.7 MB/sec
formatter/numpy/globals.py                 1.00    207.6±0.83µs    14.2 MB/sec    1.00    208.1±0.28µs    14.2 MB/sec
formatter/pydantic/types.py                1.00      4.2±0.01ms     6.1 MB/sec    1.00      4.2±0.00ms     6.0 MB/sec
linter/all-rules/large/dataset.py          1.00     13.5±0.06ms     3.0 MB/sec    1.00     13.5±0.04ms     3.0 MB/sec
linter/all-rules/numpy/ctypeslib.py        1.01      3.4±0.00ms     4.8 MB/sec    1.00      3.4±0.01ms     4.9 MB/sec
linter/all-rules/numpy/globals.py          1.01    370.9±0.81µs     8.0 MB/sec    1.00    368.7±1.00µs     8.0 MB/sec
linter/all-rules/pydantic/types.py         1.00      6.1±0.01ms     4.2 MB/sec    1.00      6.1±0.01ms     4.2 MB/sec
linter/default-rules/large/dataset.py      1.00      7.0±0.01ms     5.8 MB/sec    1.00      7.0±0.01ms     5.8 MB/sec
linter/default-rules/numpy/ctypeslib.py    1.00   1428.5±3.26µs    11.7 MB/sec    1.00   1422.1±3.82µs    11.7 MB/sec
linter/default-rules/numpy/globals.py      1.00    149.5±0.20µs    19.7 MB/sec    1.00    149.4±0.24µs    19.7 MB/sec
linter/default-rules/pydantic/types.py     1.00      3.1±0.00ms     8.2 MB/sec    1.00      3.1±0.01ms     8.2 MB/sec

Windows

group                                      main                                   pr
-----                                      ----                                   --
formatter/large/dataset.py                 1.00      9.0±0.07ms     4.5 MB/sec    1.05      9.4±0.11ms     4.3 MB/sec
formatter/numpy/ctypeslib.py               1.00  1724.6±13.52µs     9.7 MB/sec    1.03  1777.1±10.01µs     9.4 MB/sec
formatter/numpy/globals.py                 1.00    179.4±2.87µs    16.4 MB/sec    1.04    186.4±4.49µs    15.8 MB/sec
formatter/pydantic/types.py                1.00      3.8±0.03ms     6.7 MB/sec    1.04      4.0±0.04ms     6.4 MB/sec
linter/all-rules/large/dataset.py          1.00     12.4±0.09ms     3.3 MB/sec    1.00     12.3±0.07ms     3.3 MB/sec
linter/all-rules/numpy/ctypeslib.py        1.00      3.2±0.01ms     5.2 MB/sec    1.01      3.2±0.01ms     5.2 MB/sec
linter/all-rules/numpy/globals.py          1.00    338.1±3.74µs     8.7 MB/sec    1.01    342.1±2.32µs     8.6 MB/sec
linter/all-rules/pydantic/types.py         1.00      5.5±0.03ms     4.6 MB/sec    1.00      5.5±0.03ms     4.6 MB/sec
linter/default-rules/large/dataset.py      1.00      6.6±0.04ms     6.2 MB/sec    1.01      6.7±0.03ms     6.1 MB/sec
linter/default-rules/numpy/ctypeslib.py    1.00   1303.7±7.40µs    12.8 MB/sec    1.02   1323.3±7.28µs    12.6 MB/sec
linter/default-rules/numpy/globals.py      1.00    138.8±1.04µs    21.3 MB/sec    1.02    142.1±1.60µs    20.8 MB/sec
linter/default-rules/pydantic/types.py     1.00      2.9±0.01ms     8.9 MB/sec    1.01      2.9±0.02ms     8.8 MB/sec

@konstin konstin force-pushed the Don_t_minimize_files_that_don_t_match_in_the_first_place branch from b0c9d41 to de466b9 Compare July 20, 2023 13:02
@konstin konstin requested a review from MichaReiser July 20, 2023 15:21
@konstin konstin force-pushed the Don_t_minimize_files_that_don_t_match_in_the_first_place branch from 6d4ca8e to 963a352 Compare July 20, 2023 15:49
@konstin konstin added the formatter Related to the formatter label Jul 20, 2023
MichaReiser
MichaReiser approved these changes Jul 21, 2023
View reviewed changes
Copy link
Copy Markdown
Member

@MichaReiser MichaReiser left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The documentation looks good to me.

I recommend splitting the other changes into their own PR, it otherwise reading the Git history. Who would suspect that a PR Documenting... contains code changes.

Splitting should be straightforward as each change is its own commit. You can split the PR by using gt branch split

@konstin
Copy link
Copy Markdown
Member Author

konstin commented Jul 21, 2023

I recommend splitting the other changes into their own PR, it otherwise reading the Git history. Who would suspect that a PR Documenting... contains code changes.

good point, i didn't consider how this interacts with squash merge

@konstin
Don't minimize files that don't match in the first place
547a176 
**Summary** This adds a sanity check to the minimizer script that the
input matches the condition (e.g. unstable formatting). Otherwise we run
through all checks with the whole file, which is extremely slow. It's
more reasonable for downstream usage to write an empty string to the
output file instead. It also prints the minimized code in the end
@konstin konstin force-pushed the Don_t_minimize_files_that_don_t_match_in_the_first_place branch from 963a352 to 547a176 Compare July 21, 2023 08:24
This was referenced Jul 21, 2023
Merged
Merged
@konstin konstin merged commit b56e8ad into main Jul 21, 2023
@konstin konstin deleted the Don_t_minimize_files_that_don_t_match_in_the_first_place branch July 21, 2023 09:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@MichaReiser MichaReiser MichaReiser approved these changes

Assignees

No one assigned

Labels

formatter Related to the formatter

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@konstin @MichaReiser