[MRG] ENH Current Source Density Feature Add#6910
[MRG] ENH Current Source Density Feature Add#6910agramfort merged 16 commits intomne-tools:masterfrom
Conversation
agramfort
left a comment
There was a problem hiding this comment.
ok I stopped. Tell me when ready to review
Codecov Report
@@ Coverage Diff @@
## master #6910 +/- ##
==========================================
+ Coverage 89.72% 89.75% +0.02%
==========================================
Files 438 440 +2
Lines 77484 77715 +231
Branches 12579 12607 +28
==========================================
+ Hits 69523 69751 +228
- Misses 5155 5157 +2
- Partials 2806 2807 +1 |
|
@sherdim, thanks for the gist, I used it to base the example of off. |
|
@agramfort now would be okay to review |
I do not mind. It is honorable for me I had to do it myself, but I didn’t. So go ahead! |
@sherdim I can add it but I don't know your name and email... |
This comment has been minimized.
This comment has been minimized.
|
These are not timeouts. This is your version of pytest not knowing about the "timeout" mark:
E pytest.PytestUnknownMarkWarning: Unknown pytest.mark.timeout
Also I noticed you're in the"base" conda env, maybe that's the problem...
…On October 5, 2019 5:34:50 PM PDT, Alex Rockhill ***@***.***> wrote:
Hey I'm getting error when I run "make test" locally but they're all
timeout errors. Anyone know how to fix that?
(base) Alexs-MacBook-Pro:mne-python alexrockhill$ make test
python setup.py build_ext -i
running build_ext
rm -f .coverage
py.test -m 'not ultraslowtest' mne
========================================== test session starts
===========================================
platform darwin -- Python 3.6.8, pytest-5.2.0, py-1.8.0, pluggy-0.13.0
rootdir: /Users/alexrockhill/software/mne-python, inifile: setup.cfg
plugins: cov-2.7.1, arraydiff-0.3, remotedata-0.3.2, doctestplus-0.4.0,
openfiles-0.4.0
collected 1302 items / 20 errors / 1282 selected
================================================= ERRORS
=================================================
__________________________ ERROR collecting
mne/commands/tests/test_commands.py __________________________
mne/commands/tests/test_commands.py:206: in <module>
@pytest.mark.timeout(600) # took ~400 sec on a local test
../anaconda3/lib/python3.6/site-packages/_pytest/mark/structures.py:324:
in __getattr__
PytestUnknownMarkWarning,
E pytest.PytestUnknownMarkWarning: Unknown pytest.mark.timeout - is
this a typo? You can register custom marks to avoid this warning - for
details, see https://docs.pytest.org/en/latest/mark.html
__________________________ ERROR collecting
mne/commands/tests/test_commands.py __________________________
mne/commands/tests/test_commands.py:206: in <module>
@pytest.mark.timeout(600) # took ~400 sec on a local test
../anaconda3/lib/python3.6/site-packages/_pytest/mark/structures.py:324:
in __getattr__
PytestUnknownMarkWarning,
E pytest.PytestUnknownMarkWarning: Unknown pytest.mark.timeout - is
this a typo? You can register custom marks to avoid this warning - for
details, see https://docs.pytest.org/en/latest/mark.html
_________________ ERROR collecting
mne/datasets/sleep_physionet/tests/test_physionet.py __________________
mne/datasets/sleep_physionet/tests/test_physionet.py:62: in <module>
@pytest.mark.timeout(60)
../anaconda3/lib/python3.6/site-packages/_pytest/mark/structures.py:324:
in __getattr__
PytestUnknownMarkWarning,
E pytest.PytestUnknownMarkWarning: Unknown pytest.mark.timeout - is
this a typo? You can register custom marks to avoid this warning - for
details, see https://docs.pytest.org/en/latest/mark.html
_________________ ERROR collecting
mne/datasets/sleep_physionet/tests/test_physionet.py __________________
mne/datasets/sleep_physionet/tests/test_physionet.py:62: in <module>
@pytest.mark.timeout(60)
../anaconda3/lib/python3.6/site-packages/_pytest/mark/structures.py:324:
in __getattr__
PytestUnknownMarkWarning,
E pytest.PytestUnknownMarkWarning: Unknown pytest.mark.timeout - is
this a typo? You can register custom marks to avoid this warning - for
details, see https://docs.pytest.org/en/latest/mark.html
________________________ ERROR collecting
mne/forward/tests/test_make_forward.py _________________________
mne/forward/tests/test_make_forward.py:120: in <module>
@pytest.mark.timeout(60) # can take longer than 30 sec on Travis
../anaconda3/lib/python3.6/site-packages/_pytest/mark/structures.py:324:
in __getattr__
PytestUnknownMarkWarning,
E pytest.PytestUnknownMarkWarning: Unknown pytest.mark.timeout - is
this a typo? You can register custom marks to avoid this warning - for
details, see https://docs.pytest.org/en/latest/mark.html
________________________ ERROR collecting
mne/forward/tests/test_make_forward.py _________________________
mne/forward/tests/test_make_forward.py:120: in <module>
@pytest.mark.timeout(60) # can take longer than 30 sec on Travis
../anaconda3/lib/python3.6/site-packages/_pytest/mark/structures.py:324:
in __getattr__
PytestUnknownMarkWarning,
E pytest.PytestUnknownMarkWarning: Unknown pytest.mark.timeout - is
this a typo? You can register custom marks to avoid this warning - for
details, see https://docs.pytest.org/en/latest/mark.html
_____________________ ERROR collecting
mne/inverse_sparse/tests/test_mxne_inverse.py _____________________
mne/inverse_sparse/tests/test_mxne_inverse.py:44: in <module>
@pytest.mark.timeout(120) # ~30 sec on AppVeyor and Travis Linux
../anaconda3/lib/python3.6/site-packages/_pytest/mark/structures.py:324:
in __getattr__
PytestUnknownMarkWarning,
E pytest.PytestUnknownMarkWarning: Unknown pytest.mark.timeout - is
this a typo? You can register custom marks to avoid this warning - for
details, see https://docs.pytest.org/en/latest/mark.html
_____________________ ERROR collecting
mne/inverse_sparse/tests/test_mxne_inverse.py _____________________
mne/inverse_sparse/tests/test_mxne_inverse.py:44: in <module>
@pytest.mark.timeout(120) # ~30 sec on AppVeyor and Travis Linux
../anaconda3/lib/python3.6/site-packages/_pytest/mark/structures.py:324:
in __getattr__
PytestUnknownMarkWarning,
E pytest.PytestUnknownMarkWarning: Unknown pytest.mark.timeout - is
this a typo? You can register custom marks to avoid this warning - for
details, see https://docs.pytest.org/en/latest/mark.html
______________________ ERROR collecting
mne/inverse_sparse/tests/test_mxne_optim.py ______________________
mne/inverse_sparse/tests/test_mxne_optim.py:158: in <module>
@pytest.mark.timeout(60) # ~30 sec on Travis OSX and Linux OpenBLAS
../anaconda3/lib/python3.6/site-packages/_pytest/mark/structures.py:324:
in __getattr__
PytestUnknownMarkWarning,
E pytest.PytestUnknownMarkWarning: Unknown pytest.mark.timeout - is
this a typo? You can register custom marks to avoid this warning - for
details, see https://docs.pytest.org/en/latest/mark.html
______________________ ERROR collecting
mne/inverse_sparse/tests/test_mxne_optim.py ______________________
mne/inverse_sparse/tests/test_mxne_optim.py:158: in <module>
@pytest.mark.timeout(60) # ~30 sec on Travis OSX and Linux OpenBLAS
../anaconda3/lib/python3.6/site-packages/_pytest/mark/structures.py:324:
in __getattr__
PytestUnknownMarkWarning,
E pytest.PytestUnknownMarkWarning: Unknown pytest.mark.timeout - is
this a typo? You can register custom marks to avoid this warning - for
details, see https://docs.pytest.org/en/latest/mark.html
______________________ ERROR collecting
mne/io/artemis123/tests/test_artemis123.py _______________________
mne/io/artemis123/tests/test_artemis123.py:45: in <module>
@pytest.mark.timeout(60) # ~25 sec on Travis Linux OpenBLAS
../anaconda3/lib/python3.6/site-packages/_pytest/mark/structures.py:324:
in __getattr__
PytestUnknownMarkWarning,
E pytest.PytestUnknownMarkWarning: Unknown pytest.mark.timeout - is
this a typo? You can register custom marks to avoid this warning - for
details, see https://docs.pytest.org/en/latest/mark.html
______________________ ERROR collecting
mne/io/artemis123/tests/test_artemis123.py _______________________
mne/io/artemis123/tests/test_artemis123.py:45: in <module>
@pytest.mark.timeout(60) # ~25 sec on Travis Linux OpenBLAS
../anaconda3/lib/python3.6/site-packages/_pytest/mark/structures.py:324:
in __getattr__
PytestUnknownMarkWarning,
E pytest.PytestUnknownMarkWarning: Unknown pytest.mark.timeout - is
this a typo? You can register custom marks to avoid this warning - for
details, see https://docs.pytest.org/en/latest/mark.html
__________________________ ERROR collecting
mne/io/eeglab/tests/test_eeglab.py ___________________________
mne/io/eeglab/tests/test_eeglab.py:183: in <module>
@pytest.mark.timeout(60) # ~60 sec on Travis OSX
../anaconda3/lib/python3.6/site-packages/_pytest/mark/structures.py:324:
in __getattr__
PytestUnknownMarkWarning,
E pytest.PytestUnknownMarkWarning: Unknown pytest.mark.timeout - is
this a typo? You can register custom marks to avoid this warning - for
details, see https://docs.pytest.org/en/latest/mark.html
__________________________ ERROR collecting
mne/io/eeglab/tests/test_eeglab.py ___________________________
mne/io/eeglab/tests/test_eeglab.py:183: in <module>
@pytest.mark.timeout(60) # ~60 sec on Travis OSX
../anaconda3/lib/python3.6/site-packages/_pytest/mark/structures.py:324:
in __getattr__
PytestUnknownMarkWarning,
E pytest.PytestUnknownMarkWarning: Unknown pytest.mark.timeout - is
this a typo? You can register custom marks to avoid this warning - for
details, see https://docs.pytest.org/en/latest/mark.html
____________________________ ERROR collecting
mne/tests/test_source_space.py _____________________________
mne/tests/test_source_space.py:306: in <module>
@pytest.mark.timeout(60) # can be slow on OSX Travis
../anaconda3/lib/python3.6/site-packages/_pytest/mark/structures.py:324:
in __getattr__
PytestUnknownMarkWarning,
E pytest.PytestUnknownMarkWarning: Unknown pytest.mark.timeout - is
this a typo? You can register custom marks to avoid this warning - for
details, see https://docs.pytest.org/en/latest/mark.html
____________________________ ERROR collecting
mne/tests/test_source_space.py _____________________________
mne/tests/test_source_space.py:306: in <module>
@pytest.mark.timeout(60) # can be slow on OSX Travis
../anaconda3/lib/python3.6/site-packages/_pytest/mark/structures.py:324:
in __getattr__
PytestUnknownMarkWarning,
E pytest.PytestUnknownMarkWarning: Unknown pytest.mark.timeout - is
this a typo? You can register custom marks to avoid this warning - for
details, see https://docs.pytest.org/en/latest/mark.html
___________________________ ERROR collecting
mne/utils/tests/test_fetching.py ____________________________
mne/utils/tests/test_fetching.py:9: in <module>
@pytest.mark.timeout(60)
../anaconda3/lib/python3.6/site-packages/_pytest/mark/structures.py:324:
in __getattr__
PytestUnknownMarkWarning,
E pytest.PytestUnknownMarkWarning: Unknown pytest.mark.timeout - is
this a typo? You can register custom marks to avoid this warning - for
details, see https://docs.pytest.org/en/latest/mark.html
___________________________ ERROR collecting
mne/utils/tests/test_fetching.py ____________________________
mne/utils/tests/test_fetching.py:9: in <module>
@pytest.mark.timeout(60)
../anaconda3/lib/python3.6/site-packages/_pytest/mark/structures.py:324:
in __getattr__
PytestUnknownMarkWarning,
E pytest.PytestUnknownMarkWarning: Unknown pytest.mark.timeout - is
this a typo? You can register custom marks to avoid this warning - for
details, see https://docs.pytest.org/en/latest/mark.html
_______________________________ ERROR collecting
mne/viz/tests/test_3d.py ________________________________
mne/viz/tests/test_3d.py:425: in <module>
@pytest.mark.timeout(60) # can sometimes take > 60 sec
../anaconda3/lib/python3.6/site-packages/_pytest/mark/structures.py:324:
in __getattr__
PytestUnknownMarkWarning,
E pytest.PytestUnknownMarkWarning: Unknown pytest.mark.timeout - is
this a typo? You can register custom marks to avoid this warning - for
details, see https://docs.pytest.org/en/latest/mark.html
_______________________________ ERROR collecting
mne/viz/tests/test_3d.py ________________________________
mne/viz/tests/test_3d.py:425: in <module>
@pytest.mark.timeout(60) # can sometimes take > 60 sec
../anaconda3/lib/python3.6/site-packages/_pytest/mark/structures.py:324:
in __getattr__
PytestUnknownMarkWarning,
E pytest.PytestUnknownMarkWarning: Unknown pytest.mark.timeout - is
this a typo? You can register custom marks to avoid this warning - for
details, see https://docs.pytest.org/en/latest/mark.html
------------- generated xml file:
/Users/alexrockhill/software/mne-python/junit-results.xml
--------------
======================================== short test summary info
=========================================
FAILED mne/commands/tests/test_commands.py -
pytest.PytestUnknownMarkWarning: Unknown pytest.mark.timeo...
FAILED mne/commands/tests/test_commands.py -
pytest.PytestUnknownMarkWarning: Unknown pytest.mark.timeo...
FAILED mne/datasets/sleep_physionet/tests/test_physionet.py -
pytest.PytestUnknownMarkWarning: Unknown ...
FAILED mne/datasets/sleep_physionet/tests/test_physionet.py -
pytest.PytestUnknownMarkWarning: Unknown ...
FAILED mne/forward/tests/test_make_forward.py -
pytest.PytestUnknownMarkWarning: Unknown pytest.mark.ti...
FAILED mne/forward/tests/test_make_forward.py -
pytest.PytestUnknownMarkWarning: Unknown pytest.mark.ti...
FAILED mne/inverse_sparse/tests/test_mxne_inverse.py -
pytest.PytestUnknownMarkWarning: Unknown pytest....
FAILED mne/inverse_sparse/tests/test_mxne_inverse.py -
pytest.PytestUnknownMarkWarning: Unknown pytest....
FAILED mne/inverse_sparse/tests/test_mxne_optim.py -
pytest.PytestUnknownMarkWarning: Unknown pytest.ma...
FAILED mne/inverse_sparse/tests/test_mxne_optim.py -
pytest.PytestUnknownMarkWarning: Unknown pytest.ma...
FAILED mne/io/artemis123/tests/test_artemis123.py -
pytest.PytestUnknownMarkWarning: Unknown pytest.mar...
FAILED mne/io/artemis123/tests/test_artemis123.py -
pytest.PytestUnknownMarkWarning: Unknown pytest.mar...
FAILED mne/io/eeglab/tests/test_eeglab.py -
pytest.PytestUnknownMarkWarning: Unknown pytest.mark.timeou...
FAILED mne/io/eeglab/tests/test_eeglab.py -
pytest.PytestUnknownMarkWarning: Unknown pytest.mark.timeou...
FAILED mne/tests/test_source_space.py -
pytest.PytestUnknownMarkWarning: Unknown pytest.mark.timeout - ...
FAILED mne/tests/test_source_space.py -
pytest.PytestUnknownMarkWarning: Unknown pytest.mark.timeout - ...
FAILED mne/utils/tests/test_fetching.py -
pytest.PytestUnknownMarkWarning: Unknown pytest.mark.timeout ...
FAILED mne/utils/tests/test_fetching.py -
pytest.PytestUnknownMarkWarning: Unknown pytest.mark.timeout ...
FAILED mne/viz/tests/test_3d.py - pytest.PytestUnknownMarkWarning:
Unknown pytest.mark.timeout - is thi...
FAILED mne/viz/tests/test_3d.py - pytest.PytestUnknownMarkWarning:
Unknown pytest.mark.timeout - is thi...
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 20 errors during
collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
=========================================== 20 error in 9.92s
============================================
make: *** [test] Error 2
|
|
Thanks, I could have figured that out my bad |
|
@alexrockhill somehow you have a bunch of conflicts in unrelated files |
8c708de to
37b75cd
Compare
|
I'm having some weird issues with tests failing... I'm not sure why files not related to the files being changed for this PR weirdly but I think I fixed that. When I run "make test" most of the tests pass but I get a few errors like E FileNotFoundError: [Errno 2] No such file or directory: 'mne_compensate_data': 'mne_compensate_data' /Users/alexrockhill/software/mne-python/mne/preprocessing/_peak_finder.py:45: DocTestFailure E AssertionError: SNR (worst 2.55) < 3.50 for 18/306 channels (/Users/alexrockhill/mne_data/MNE-testing-data/SSS/test_move_anon_raw.fif) I'm not sure what these are about. Using a fresh install of the mnedev environment. Here's just the first part of the stack trace: (mnedev) alexrockhill@Alexs-MacBook-Pro mne-python % make test ―――――――――――――――――― [doctest] mne.filter.construct_iir_filter ――――――――――――――――――― /Users/alexrockhill/software/mne-python/mne/filter.py:112: DocTestFailure mne/filter.py ⨯ 1% ▏ ――――――――――――――――――――――――― [doctest] mne.filter.detrend ――――――――――――――――――――――――― |
|
@alexrockhill are all the changes here related to this PR? |
|
They are or should be now. For some reason there were other files changed but I checked the file change diffs and that should be fixed. Now they are all just the files I had to change for csd |
drammock
left a comment
There was a problem hiding this comment.
There are changes to what's new and to _optical_density.py that appear unrelated at first glance. Also maybe the spline stuff? Can you double check?
|
Tests pass locally now, it was missing a |
|
I saw that it had to do with a false positive meg_ref, I just couldn't figure out at which level (tests, epochs, viz.epochs or viz.utils) so I'm glad you figured it out :) |
agramfort
left a comment
There was a problem hiding this comment.
sorry for taking so long to review
| for t, rules in get_channel_types().items(): | ||
| # go in order from most specific (most rules entries) to least specific | ||
| channel_types = sorted( | ||
| get_channel_types().items(), key=lambda x: len(x[1]))[::-1] |
There was a problem hiding this comment.
I add a comment here to explain this.
added some tests WIP CSD MVP example working with example weird cherry-picking error again Update examples/preprocessing/plot_csd.py Co-Authored-By: Alexandre Gramfort <alexandre.gramfort@m4x.org> Update examples/preprocessing/plot_csd.py Co-Authored-By: Alexandre Gramfort <alexandre.gramfort@m4x.org> Update examples/preprocessing/plot_csd.py Co-Authored-By: Alexandre Gramfort <alexandre.gramfort@m4x.org> Update mne/preprocessing/_csd.py Co-Authored-By: Alexandre Gramfort <alexandre.gramfort@m4x.org> Update examples/preprocessing/plot_csd.py Co-Authored-By: Alexandre Gramfort <alexandre.gramfort@m4x.org> typos/small fixes incorporated Eric comments more Eric comments refactoring core comp per Eric realized I did that wrong Eric comments 3 plot fix Fix julian date conversions (mne-tools#6911) * Fix julian date conversions * remove tabs * use UTC and private functions. closes mne-tools#6901 [skip travis][skip azp] (mne-tools#6914) ENH: Add source space SNR estimate (mne-tools#6908) * source-snr-edit * FIX move estimate_snr to be a method * ENH: name change * Make it work * SNR works, now clean up code * fixed unnecessary stc copy, returns stc object * added plot source space SNR tutorial file * updated author names * fixed period for pydocsty and suggested documenation edit * edits suggested from @drammock * made edits suggested by reviewers * made edits suggested by reviewers * N -> n_channels * Fixed formatting of Reference in comment * fixed formatting in doc and in code * fixed snr_stc creation * further minor edits * fixed formatting issues and colormap choice in eg * removed colon after Reference * Corrected SNR calculation to take unwhitened sensor signal b instead of whitened, updated tutorial file * Minor changes to formatting * FIX: transparent * FIX: Rename * FIX? Added dSPM (mne-tools#6913) * Added dSPM * removed term signal covariance * Minor modification of dSPM definition * Corrected according to Mattis suggestions [MRG]: read *.dat electrode position files (mne-tools#6909) * ADD: channels.read_dig_dat() * DOC * DOC * TEST * DOC, __all__ * DOC: intro * DOC * pep & py3.5 compatibility * py3.5 MRG: allow users to annotate epochs with colors (mne-tools#6855) * ENH: copy over diff from autoreject * Make it work * More fixes * More fixes * Make horizontal patch * Cleanup * TST: add tests * Address Eric's comments * Address alex comments * DOC: update whats new * FIX travis + circle * FIX For ICA plot Fix infomax (mne-tools#6921) * fix extended infomax * update whats new * cleaner way, using setdefault DOC: Document authors better [ci skip] FIX: Comments and order umlauts MRG, MAINT: Simplify code for vol ravel (mne-tools#6920) * MAINT: Simplify code for vol ravel * STY: Flake ENH: Convert NIRS raw data to optical density (mne-tools#6827) * Initial framework for fnirs raw to optical density conversion * Add optical density to docs * Update mne/preprocessing/tests/test_optical_density.py Co-Authored-By: Eric Larson <larson.eric.d@gmail.com> * Update mne/preprocessing/tests/test_optical_density.py Co-Authored-By: Eric Larson <larson.eric.d@gmail.com> * Import testing * Put optical density docs in preprocessing section * Implement optical density conversion * Fix optical density code style issues * Add plotting of optical density data * Add defaults for optical density plotting * Update mne/preprocessing/optical_density.py Co-Authored-By: Eric Larson <larson.eric.d@gmail.com> * Handle negative intensities in optical density conversion * Add in -1* that was removed in previous commit * Add tests and fix imports for optical density * Add tests for optical density conversion * Use assert_allclose for testing optical density * Update test_optical_density.py * Update mne/preprocessing/_optical_density.py Co-Authored-By: Alexandre Gramfort <alexandre.gramfort@m4x.org> * Do not operate in place for optical density * Force data to load and modify test to include preload=False condition import fix tidying up weird file conflict errors more unexpected file changes fix weird off target changes doc docstring fixes docstyle reformatted plots changes formatting fix Update mne/preprocessing/_csd.py Co-Authored-By: Eric Larson <larson.eric.d@gmail.com> Update examples/preprocessing/plot_csd.py Co-Authored-By: Eric Larson <larson.eric.d@gmail.com> Update mne/preprocessing/tests/test_csd.py Co-Authored-By: Eric Larson <larson.eric.d@gmail.com> Update mne/preprocessing/_csd.py Co-Authored-By: Eric Larson <larson.eric.d@gmail.com> Update mne/preprocessing/_csd.py Co-Authored-By: Eric Larson <larson.eric.d@gmail.com> Update mne/preprocessing/_csd.py Co-Authored-By: Eric Larson <larson.eric.d@gmail.com> halfway through Eric suggested changes getting on plane WIP Eric comments almost done testing fixes updated fif constants csd formatting small change references
|
@agramfort I'm not sure if there was a versioning issue but there weren't any evo in the last one, they had all been changed to evoked... not sure what happened there. Anyway, I did as you suggested, and it should be good to go now. |
agramfort
left a comment
There was a problem hiding this comment.
we've never been so close ! thx for your patience and energy @alexrockhill !
| """Test stim_channel.""" | ||
| TYPE_LUT = {v[0]: k for k, v in _KIND_DICT.items()} | ||
| # This is fragile for EEG/EEG-CSD, so just omit csd | ||
| TYPE_LUT = {v[0]: k for k, v in _KIND_DICT.items() if k != 'csd'} |
There was a problem hiding this comment.
having to change something outside seems like a code smell. @larsoner any thought? why is this needed?
| for t, rules in get_channel_types().items(): | ||
| # go in order from most specific (most rules entries) to least specific | ||
| channel_types = sorted( | ||
| get_channel_types().items(), key=lambda x: len(x[1]))[::-1] |
There was a problem hiding this comment.
what specific means in this context was not clear to me.
|
thx heaps @alexrockhill for leading this effort ! |
Fixes #4480
Adds a current source density estimate function that converts EEG data using the Laplacian of the sensor data.