At some point, pytest and flake8 stopped working well together. See tholo/pytest-flake8#81 and related links.

This may get worse with pytest 7

On this thread, see a comment around running flake8 as part of pytest is considered an anti pattern.

I think that the more tooling we try to bake into a given pytest run, the harder it is to split them apart, so instead, let's adopt the pre-commit framework for executing any stylistic checks we want prior to a commit - flake8, black, isort, and anything else we want, like editorconfig...

This check would need to also run on CI so that if an author doesn't run it locally, it'll catch anything missing.