Skip to content

Use pre-commit#11945

Merged
pllim merged 8 commits into
astropy:mainfrom
saimn:precommit
Oct 22, 2021
Merged

Use pre-commit#11945
pllim merged 8 commits into
astropy:mainfrom
saimn:precommit

Conversation

@saimn

@saimn saimn commented Jul 14, 2021

Copy link
Copy Markdown
Contributor

I was looking for a way to run checks for the changelog fragment filenames and pre-commit seems a good option. Actually tox already has such a check so I adapted theirs. With this pre-commit will check that the fragments have one of the correct categories (bugfix|feature|api|other), and that files with the "other" category are put only in the root directory (following #11856 (comment)).

I also added some basic hooks for trailing whitespace and end of lines (@Cadair proposed a similar config in #9890)
About flake8: I added a hook to run flake8 via tox -e codestyle. In #9890 @Cadair did it another way by running pre-commit from tox. After playing with this, I think we should remove flake8 / codestyle from the tox config and do it directly with pre-commit, there is no real benefit in having the possibility to run flake8 with both. And since pre-commit is about running code style checks, flake8 should be part it.

If we go with this, we will need to run it once to fix the existing files, I can do that in this PR later (after 4.3 is out and fragments have been deleted in main).

@pllim

pllim commented Jul 14, 2021

Copy link
Copy Markdown
Member

@pllim pllim added this to the v5.0 milestone Jul 14, 2021
Comment thread .pre-commit-config.yaml
# - repo: https://github.com/asottile/pyupgrade
# rev: v2.21.0
# hooks:
# - id: pyupgrade

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

2 additional checks we could use:

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Other good built-in ones (v3.4.0>) are:

check-added-large-files
check-case-conflict
check-merge-conflict
check-symlinks
check-toml
check-yaml
debug-statements
destroyed-symlinks
detect-private-key
end-of-file-fixer
fix-encoding-pragma
requirements-txt-fixer
trailing-whitespace

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

isort config, should we run it

👍

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And https://github.com/pre-commit/pygrep-hooks is great for rst checks.

  • rst-backticks
  • rst-directive-colons
  • rst-inline-touching-normal

I also use it for checking that I give the right error message in noqa e.g. noqa: F403

  • python-check-blanket-noqa

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Other good built-in ones (v3.4.0>) are:

I already added most of them:

  - id: check-added-large-files
  - id: check-case-conflict
  - id: check-yaml
  - id: debug-statements
  - id: end-of-file-fixer
  - id: trailing-whitespace

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like they are working! The CI has found oodles of corrections in the code.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is "oodles" good or bad? 😅

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Both! 😆

@saimn

saimn commented Jul 14, 2021

Copy link
Copy Markdown
Contributor Author

@pllim - No, it just runs on all changelog fragments, and cannot check that a given PR actually has a fragment. And it couldn't be skipped with some label.

@pllim pllim requested review from a team and Cadair July 14, 2021 17:41
@saimn

saimn commented Jul 14, 2021

Copy link
Copy Markdown
Contributor Author

For the changelog check (@bsipocz @eteq), here is what it gives:

❯ pre-commit run changelogs-rst --all-files
changelog filenames......................................................Failed
- hook id: changelogs-rst
- exit code: 1

changelog files must be named <sub-package>/####.(bugfix|feature|api).rst or ####.other.rst (in the root directory only)

docs/changes/coordinates/11069.other.rst
docs/changes/coordinates/11073.other.rst
docs/changes/coordinates/11090.other.rst
docs/changes/coordinates/11449.other.rst
docs/changes/coordinates/11485.other.rst
docs/changes/coordinates/11601.other.rst
docs/changes/coordinates/11753.other.rst
docs/changes/cosmology/11530.other.rst
docs/changes/io.ascii/11919.other.rst
docs/changes/nddata/11107.other.rst
docs/changes/nddata/11553.other.rst
docs/changes/stats/11219.other.rst
docs/changes/stats/11356.other.rst
docs/changes/utils/11490.other.rst

❯ pre-commit run changelogs-rst-other --all-files
changelog filenames for other category...................................Failed
- hook id: changelogs-rst-other
- exit code: 1

only "other" changelog files must be placed in the root directory

docs/changes/11232.bugfix.rst

@bsipocz

bsipocz commented Jul 26, 2021

Copy link
Copy Markdown
Member

minor idea: maybe this should also check whether the changelog is too wordy, e.g. notify the reviewer to double-check the entry if it's longer than ~5 lines?

@pllim pllim requested a review from a team July 26, 2021 20:13

@pllim pllim left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of adding all sorts of checks in this PR, perhaps we should go with what is already here and defer the rest for follow-up PRs.

Release managers should carefully review the change log check, as it will affect them the most at release time.

@saimn

saimn commented Jul 26, 2021

Copy link
Copy Markdown
Contributor Author

minor idea: maybe this should also check whether the changelog is too wordy, e.g. notify the reviewer to double-check the entry if it's longer than ~5 lines?

I guess I could make it fail if a fragment has more than 5 lines, but then with the way it works it would always fail until you fix that file. So it would prevent having exceptions unless adding the file to the exceptions in pre-commit's config, which would not be practical.

@bsipocz

bsipocz commented Jul 26, 2021

Copy link
Copy Markdown
Member

it works it would always fail until you fix that file.

what about a new special label? :)

@saimn

saimn commented Jul 26, 2021

Copy link
Copy Markdown
Contributor Author

pre-commit does not access github labels and does not know anything about the current PR. It just runs a number of checks on a selection of files. So for the changelog fragments I just tell it to check that files in doc/changes/ respect some rules, with a regex.

@nstarman

Copy link
Copy Markdown
Member

I checked out this branch and ran pre-commit run --all-files twice. I can confirm that the second run produced no errors, except for the expected error of misplaced / misnamed "other" changelog fragments.

@nstarman

Copy link
Copy Markdown
Member

@saimn, #12193 introduced a conflict.

@pllim pllim added the skip-changelog-checks Tells bot to skip changlog checks label Oct 21, 2021
@nstarman

nstarman commented Oct 21, 2021

Copy link
Copy Markdown
Member

Rerunning the CI to see if the code style check job error is real...

@saimn it appears to be real.

@saimn

saimn commented Oct 22, 2021

Copy link
Copy Markdown
Contributor Author

Oops, in the initial version I had pre-commit calling tox for flake8, and then I updated tox to call pre-commit 😁

@pllim pllim left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a good improvement to help maintainers. Let's get it in! We can always tweak it later, as needed.

@pllim pllim merged commit 4a146f7 into astropy:main Oct 22, 2021
@saimn saimn deleted the precommit branch October 22, 2021 15:17
@nstarman nstarman mentioned this pull request Oct 22, 2021
9 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants