Skip to content

Add auto-formatting via isort and black#3062

Closed
Eric-Arellano wants to merge 11 commits intopython:masterfrom
Eric-Arellano:autoformat
Closed

Add auto-formatting via isort and black#3062
Eric-Arellano wants to merge 11 commits intopython:masterfrom
Eric-Arellano:autoformat

Conversation

@Eric-Arellano
Copy link
Contributor

@Eric-Arellano Eric-Arellano commented Jun 16, 2019

Will close #3058.

Autoformatting makes it easier for readers to have a consistent code style and easier for developers to not waste time dealing with formatting.

We achieve this via isort and black, with a new pyproject.toml to configure both to use a line limit of 130.

Followup: script to run tests and autoformatters

This change adds two additional steps that we expect contributors to do before submitting a PR. Instead, to make it easier for contributors, we can automate much of the validation flow via a new script, as proposed in #3058 (comment).

We run both isort and black on the same shard because there is a nontrivial startup cost to adding an additional shard. It reduces the available worker pool and each shard takes some time to clone the repository and get set up properly.
@Eric-Arellano
Copy link
Contributor Author

Reviewers: GitHub's review page puts all of the non-automated changes at the top of the review. Alternatively, each commit may be meaningfully reviewed independently.

They handle comments in the import section differently. The best solution for now is to turn off Black, because isort has the more desirable behavior.
This complies more with the original style. The only reason I didn't have this on was that I didn't know it was an option.
@Eric-Arellano
Copy link
Contributor Author

I'm not able to figure out the Pytype issue. It doesn't appear to be due to trailing commas, which is what I was suspecting, as setting black's target_python to py34 (which doesn't add trailing comma to kwargs) does not fix the issue.

I've been struggling to understand pytype_test.py. Would it be okay for me to refactor the file a bit in a separate precursor PR, e.g. add type hints, use subprocess.run()? I think it will help me to better understand what's going on and hopefully improve the file for everyone else. Would try to refrain from getting too carried away. CC @JelleZijlstra @srittau.

(On that note, thoughts on requiring Python 3.6 for these scripts so that we can use f-strings?)

--

I also can't figure out why isort is trying to change three files in CI: https://travis-ci.org/python/typeshed/jobs/546514620#L265. Locally, isort isn't making the same changes.

@rchen152
Copy link
Collaborator

Regarding the pytype issue - I wasn't able to tell from the Travis logs what the specific problem(s) are this time, but pytype has a custom pyi parser that only recognizes a subset of the Python grammar. (Someday we'll find time to to replace the custom parser with something more maintainable like typed_ast...) Feel free to open issues against https://github.com/google/pytype if you can figure out what constructs pytype is barfing on.

@srittau
Copy link
Collaborator

srittau commented Jun 17, 2019

I think we should split this PR:

  • The procedural files (pyproject and requirements). These are uncontroversial, I think, and useful even when the rest of the migration takes a bit more time.
  • The documentation (README, CONTRIBUTING), since it is independent of the Big Reformat and it's easier to discuss this separately. Useful to have separate, if we decide to handle this differently.
  • The Big Reformat. Since this is quite disruptive, it is probably best handled by a maintainer, who can do the reformat against master, open a PR, wait for the checks to run and merge. I don't think we need a second review if the tests pass, since this is a purely automatic change.
  • The Travis change, which can be merged when all of the above have been merged.

@Eric-Arellano
Copy link
Contributor Author

I think we should split this PR.

Agreed, I prefer a smaller diff. I'll close this PR so that we have it still for reference.

I would tweak the order to go 1) procedural, 2) Big Reformat, 3) documentation (and/or bot and/or script to simplify running tests and autormatters) + Travis. Shouldn't document how to run for users until it's guaranteed that them running the tool won't cause unrelated files to change, which is a jarring experience.

The Big Reformat. Since this is quite disruptive, it is probably best handled by a maintainer

Sounds good. Before we can do that, we must fix the pytype issues. The feedback is much more useful with --print-stderr, although haven't had time to dive in yet. I'll try to grok the issue, potentially open upstream issues, and update you all if I'm blocked.

Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Use black and isort for automatic formatting

3 participants