Skip to content

formatter: Remove CST and old formatting#4730

Merged
MichaReiser merged 1 commit intomainfrom
formatter-remove-cst
May 31, 2023
Merged

formatter: Remove CST and old formatting#4730
MichaReiser merged 1 commit intomainfrom
formatter-remove-cst

Conversation

@MichaReiser
Copy link
Member

@MichaReiser MichaReiser commented May 30, 2023 •
edited
Loading

Summary

This PR removes the formatter's CST implementation and the existing formatting code. I intend of reusing the formatter code by going back to this PR and copying the logic over.

The reason for deleting the code now is that updating to RustPython/Parser#76 causes some tests to panic and I don't believe it's still worth investing time into fixing the panics, considering that we intend replacing the code anyway.

I added some dummy formatting that simply prints the source code as is (it's not a 100% correct because RustPython's generated ranges do not include trailing trivia).

Why:
The new architecture avoids re-building the AST and collecting the children. This should result in a significantly faster formatter.

Test Plan

cargo test

@MichaReiser
Copy link
Member Author

MichaReiser commented May 30, 2023

Current dependencies on/for this PR:

This comment was auto-generated by Graphite.

@MichaReiser MichaReiser added internal An internal refactor or improvement formatter Related to the formatter labels May 30, 2023
@MichaReiser MichaReiser requested a review from charliermarsh May 30, 2023 16:23
@MichaReiser MichaReiser marked this pull request as ready for review May 30, 2023 16:25
@MichaReiser MichaReiser force-pushed the formatter-remove-cst branch from c41a345 to 05d43e6 Compare May 30, 2023 16:29
charliermarsh
charliermarsh approved these changes May 30, 2023
View reviewed changes
Copy link
Member

@charliermarsh charliermarsh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🥲

@MichaReiser
Copy link
Member Author

MichaReiser commented May 30, 2023

🥲

Yeah I'm sorry. We'll bring back some of it.

@github-actions
Copy link
Contributor

github-actions bot commented May 30, 2023 •
edited
Loading

PR Check Results

Ecosystem

✅ ecosystem check detected no changes.

Benchmark

Linux

group                                      main                                   pr
-----                                      ----                                   --
linter/all-rules/large/dataset.py          1.02     15.0±0.17ms     2.7 MB/sec    1.00     14.8±0.19ms     2.8 MB/sec
linter/all-rules/numpy/ctypeslib.py        1.02      3.6±0.07ms     4.6 MB/sec    1.00      3.5±0.08ms     4.7 MB/sec
linter/all-rules/numpy/globals.py          1.01    428.2±0.93µs     6.9 MB/sec    1.00    422.2±1.20µs     7.0 MB/sec
linter/all-rules/pydantic/types.py         1.03      6.3±0.12ms     4.1 MB/sec    1.00      6.1±0.13ms     4.2 MB/sec
linter/default-rules/large/dataset.py      1.03      7.1±0.07ms     5.7 MB/sec    1.00      6.9±0.08ms     5.9 MB/sec
linter/default-rules/numpy/ctypeslib.py    1.02   1522.0±4.03µs    10.9 MB/sec    1.00   1488.8±6.48µs    11.2 MB/sec
linter/default-rules/numpy/globals.py      1.01    171.4±2.29µs    17.2 MB/sec    1.00    168.9±1.90µs    17.5 MB/sec
linter/default-rules/pydantic/types.py     1.02      3.2±0.03ms     8.0 MB/sec    1.00      3.1±0.02ms     8.2 MB/sec
parser/large/dataset.py                    1.00      5.2±0.02ms     7.8 MB/sec    1.00      5.2±0.02ms     7.8 MB/sec
parser/numpy/ctypeslib.py                  1.00   1025.1±7.56µs    16.2 MB/sec    1.00   1023.2±1.24µs    16.3 MB/sec
parser/numpy/globals.py                    1.00    105.8±0.36µs    27.9 MB/sec    1.00    106.3±4.36µs    27.8 MB/sec
parser/pydantic/types.py                   1.00      2.2±0.00ms    11.4 MB/sec    1.00      2.2±0.00ms    11.4 MB/sec

Windows

group                                      main                                    pr
-----                                      ----                                    --
linter/all-rules/large/dataset.py          1.01     16.5±0.15ms     2.5 MB/sec     1.00     16.4±0.14ms     2.5 MB/sec
linter/all-rules/numpy/ctypeslib.py        1.01      4.2±0.05ms     4.0 MB/sec     1.00      4.2±0.05ms     4.0 MB/sec
linter/all-rules/numpy/globals.py          1.03   511.2±11.40µs     5.8 MB/sec     1.00    496.6±6.88µs     5.9 MB/sec
linter/all-rules/pydantic/types.py         1.03      7.1±0.13ms     3.6 MB/sec     1.00      6.9±0.06ms     3.7 MB/sec
linter/default-rules/large/dataset.py      1.03      8.5±0.16ms     4.8 MB/sec     1.00      8.3±0.38ms     4.9 MB/sec
linter/default-rules/numpy/ctypeslib.py    1.09  1918.9±114.45µs     8.7 MB/sec    1.00  1755.9±29.13µs     9.5 MB/sec
linter/default-rules/numpy/globals.py      1.04    211.4±9.24µs    14.0 MB/sec     1.00    204.1±4.66µs    14.5 MB/sec
linter/default-rules/pydantic/types.py     1.05      3.9±0.14ms     6.6 MB/sec     1.00      3.7±0.04ms     6.9 MB/sec
parser/large/dataset.py                    1.12      7.4±0.04ms     5.5 MB/sec     1.00      6.6±0.31ms     6.2 MB/sec
parser/numpy/ctypeslib.py                  1.12  1366.6±14.94µs    12.2 MB/sec     1.00  1221.1±12.96µs    13.6 MB/sec
parser/numpy/globals.py                    1.09    136.1±2.55µs    21.7 MB/sec     1.00    124.8±1.76µs    23.6 MB/sec
parser/pydantic/types.py                   1.12      3.1±0.02ms     8.3 MB/sec     1.00      2.7±0.04ms     9.3 MB/sec

@MichaReiser MichaReiser merged commit 06bcb85 into main May 31, 2023
@MichaReiser MichaReiser deleted the formatter-remove-cst branch May 31, 2023 06:27
@MichaReiser MichaReiser mentioned this pull request Jun 2, 2023
Closed
72 tasks
konstin pushed a commit that referenced this pull request Jun 13, 2023
@MichaReiser @konstin
formatter: Remove CST and old formatting (#4730)
2b77d3e
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@charliermarsh charliermarsh charliermarsh approved these changes

Assignees

No one assigned

Labels

formatter Related to the formatter internal An internal refactor or improvement

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@MichaReiser @charliermarsh