Add pytest-xdist to speed up the CI#6170
Conversation
|
@kAIto47802 |
|
How about using |
nabenabe0928
left a comment
There was a problem hiding this comment.
Thank you for the PR, I left some comments:)
pyproject.toml
Outdated
| markers = [ | ||
| "skip_coverage: marks tests are skipped when calculating the coverage", | ||
| "slow: marks tests as slow (deselect with '-m \"not slow\"')", | ||
| "serial: marks tests as fragile when executed in parallel.", |
There was a problem hiding this comment.
Do we still need this line?
There was a problem hiding this comment.
I forgot to delete this line.
| if storage_mode == "grpc_journal_file": | ||
| pytest.skip("gRPC journal file storage does not support multi-threading") |
There was a problem hiding this comment.
I am not confident if it is fine to skip this test.
There was a problem hiding this comment.
As I referred to in #6172, this test with grpc_journal_file is potentially fragile, so I think it's better to skip it.
This test with thread sizes more than 20 tends to fail on grpc_journal_file, and with xdist exceeding this thread size.
Regarding the Footnotes |
| fi | ||
|
|
||
| pytest tests -m "$target" | ||
| pytest tests -m "$target" -n 8 |
There was a problem hiding this comment.
Probably, you wanted to do something like this?
pytest tests -m "not serial" -n auto
pytest tests -m "serial"Co-authored-by: Shuhei Watanabe <47781922+nabenabe0928@users.noreply.github.com> Co-authored-by: kAIto47802 <115693559+kAIto47802@users.noreply.github.com>
I think the reason the CI test is so slow isn’t the computational load but the tests that include sleep (such as heartbeat tests). In this case, increasing parallelism beyond the number of CPU cores can be effective. |
kAIto47802
left a comment
There was a problem hiding this comment.
Thank you for the PR. LGTM!
pytest-xdist to speed up the CI
Motivation
Current ci test takes around 15 minutes. To improve the speed of ci tests, I added the
pytest-xdistto parallelize.Description of the changes
pytest-xdisttopyproject.toml-n 8to every pytest execution.Additional context
To determine best worker number for the ci test, I ran a lot of test on my repository, and I found it seems the 8 workers are enough.
Following figures are the mean and stddev of 4 ci-tests varying the worker number from 1 to 10. The measurement was done by my repository. The elapsed time is just pytest execution. So this graph DOES NOT contain the pip install or other setup time.
