Skip to content

MAINT: Show test timings#8240

Merged
larsoner merged 3 commits intomne-tools:masterfrom
larsoner:timings
Sep 11, 2020
Merged

MAINT: Show test timings#8240
larsoner merged 3 commits intomne-tools:masterfrom
larsoner:timings

Conversation

@larsoner
Copy link
Copy Markdown
Member

@larsoner larsoner commented Sep 11, 2020

Hopefully should help diagnose recent Travis timeouts. For example locally:

$ pytest mne/utils/ mne/preprocessing/ mne/io/
...
=================================================================== slowest 20 test modules ====================================================================
74.66s total   mne/preprocessing/tests
37.59s total   mne/io/fieldtrip/tests
33.01s total   mne/io/fiff/tests
26.45s total   mne/io/eeglab/tests
12.42s total   mne/io/egi/tests
 7.85s total   mne/io/tests
 7.00s total   mne/io/edf/tests
 6.49s total   mne/io/bti/tests
 6.07s total   mne/io/ctf/tests
 3.04s total   mne/io/brainvision/tests
 2.82s total   mne/io/kit/tests
 2.77s total   mne/io/artemis123/tests
 1.86s total   mne/utils/tests
 0.92s total   mne/io/snirf/tests
 0.76s total   mne/io/nirx/tests
 0.65s total   mne/io/curry/tests
 0.61s total   mne/io/cnt/tests
 0.59s total   mne/io/eximia/tests
 0.48s total   mne/io/array/tests
 0.24s total   mne/io/persyst/tests

Ideally smarie/python-pytest-harvest#41 will be merged and a new release made so we could just do pytest-harvest in the testing requirements, but this is probably good enough already (easy enough to update later).

Let's see if it works on the full test suite.

@larsoner larsoner mentioned this pull request Sep 11, 2020
Closed
@larsoner
Copy link
Copy Markdown
Member Author

larsoner commented Sep 11, 2020

Pushed a commit to split our main mne/tests, since they are basically submodule tests. Locally for example:

$ pytest -m "not ultraslowtest" mne/tests/ mne/utils/ mne/io/
...
29.86s total   mne/tests/test_rank.py
26.41s total   mne/io/fiff/
25.40s total   mne/tests/test_epochs.py
24.54s total   mne/tests/test_morph.py
21.55s total   mne/tests/test_report.py
20.99s total   mne/io/fieldtrip/
19.53s total   mne/io/egi/
17.33s total   mne/tests/test_source_space.py
15.81s total   mne/tests/test_chpi.py
11.79s total   mne/io/eeglab/
11.61s total   mne/tests/test_source_estimate.py
 9.76s total   mne/tests/test_cov.py
 8.45s total   mne/tests/test_label.py
 6.10s total   mne/io/tests/
 5.99s total   mne/tests/test_coreg.py
 5.81s total   mne/io/bti/
 5.77s total   mne/tests/test_filter.py
 5.33s total   mne/io/edf/
 5.25s total   mne/tests/test_docstring_parameters.py
 4.92s total   mne/io/ctf/

@larsoner
Copy link
Copy Markdown
Member Author

larsoner commented Sep 11, 2020

In it goes, should help with follow-up PRs to reduce numbers

@larsoner larsoner merged commit e63190c into mne-tools:master Sep 11, 2020
@larsoner larsoner deleted the timings branch September 11, 2020 16:37
marsipu pushed a commit to marsipu/mne-python that referenced this pull request Oct 14, 2020
@larsoner @marsipu
MAINT: Show test timings (mne-tools#8240)
a09a031 
* MAINT: Show test timings

* FIX: Need full checkout

* FIX: Better paths
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@agramfort agramfort agramfort approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@larsoner @agramfort