We currently run a large suite of tests in CI, which includes testing with different tools across different toolchains and different targets. It also involves miscellaneous things like confirming that README.md is up to date, that code is properly formatted with rustfmt, etc. In other words, it's not as simple as just running cargo test.

As a result, it's easy to make changes that appear fine locally but break CI, requiring an edit-commit-push cycle just to see if all of the CI tests will pass. We should make it easier to run most or all of these tests locally. For especially fast tests (like README.md and code formatting), we could even provide them as commit hooks.

@djkoloski also reports finding this to be painful (#69 (comment)):

I struggled specifically with formatting and forgetting to make stderr match in tests. Adding a way to format test files and an easier way to update all the stderr files for tests might be helpful. 🙂

The challenge here is going to be making these tests available without either a) moving everything out of our ci.yml and into a script (which means we don't get nice job output and error messages) or, b) duplicating everything (which is brittle).