FIX: Montage Index error when importing CNT files by massich · Pull Request #6025 · mne-tools/mne-python

massich · 2019-03-04T17:09:07Z

codecov · 2019-03-04T17:49:39Z

Codecov Report

Merging #6025 into master will increase coverage by 6.03%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master    #6025      +/-   ##
==========================================
+ Coverage    81.3%   87.34%   +6.03%     
==========================================
  Files         402      402              
  Lines       73181    73162      -19     
  Branches    12163    12159       -4     
==========================================
+ Hits        59503    63904    +4401     
+ Misses      11021     6411    -4610     
- Partials     2657     2847     +190

massich · 2019-03-11T14:35:32Z

Just for the record, testing with some confidential files + some files on the web running the following

import mne
import pytest
import os.path as op
from datetime import datetime
from mne import __file__ as _mne_file
from mne.tests.test_annotations import _assert_annotations_equal

regular_path = op.join(op.dirname(_mne_file), '..', 'sandbox', 'data')
confidential_path = op.join(regular_path, 'confidential', 'cnt')

flankers_path = op.join(regular_path, '914flankers.cnt')

cnt_files_with_eeglab_pair = [
    op.join(confidential_path, 'BoyoAEpic1_16bit.cnt'),
    op.join(confidential_path, 'cont_67chan_resp_32bit.cnt'),
    op.join(confidential_path, 'SampleCNTFile_16bit.cnt')
]


def test_ensure_meas_date(recwarn):
    raw = mne.io.read_raw_cnt(flankers_path, montage=None,
                              date_format='dd/mm/yy')
    meas_date = (datetime
                 .fromtimestamp(raw.info['meas_date'][0])
                 .strftime('%d/%m/%y %H:%M:%S'))

    assert meas_date == '23/09/07 12:22:15'


@pytest.mark.parametrize('fname', cnt_files_with_eeglab_pair, ids=op.basename)
def test_check_cnt_eeglab_pairs(fname, recwarn):
    raw_eeglab = mne.io.read_raw_eeglab(fname.replace('.cnt', '.set'),
                                        montage=None)
    raw_cnt = mne.io.read_raw_cnt(fname, montage=None)

    assert raw_cnt.info.keys() == raw_eeglab.info.keys()
    xx = object_diff(raw_cnt.info, raw_eeglab.info)
    print(xx)

I get:

The date work properly,
the info difference between cnt and its transformed eeglab version are
- the subject_info anonymization (eeglab is none cnt has empty fields)
- digitization mismatch (but this is beeing addressed somwhere else)
- for one file cnt has a trigger channel marked as bad that does not appear in the eeglab version of the file.

['chs'][63]['cal'] type mismatch (<class 'numpy.ndarray'>, <class 'float'>)
['chs'][63]['coord_frame'] value mismatch (4, 0)
['chs'][63]['loc'] array mismatch
['chs'][63]['unit_mul'] type mismatch (<class 'float'>, <class 'int'>)
['subject_info'] type mismatch (<class 'dict'>, <class 'NoneType'>)

massich

Most probably I'm breaking something here.

massich · 2019-03-13T12:46:25Z

reading 914flankers.cnt in master:

>               stim_channel[event_time - 1] = event_id
E               IndexError: index 583988 is out of bounds for axis 0 with size 582560

in this PR:

 sandbox/mwe/5493_montage_index_error.py ✓                                                100% ██████████
-------------------- generated xml file: /home/sik/code/mne-python/junit-results.xml --------------------
======================================= slowest 20 test durations =======================================
1.34s setup    sandbox/mwe/5493_montage_index_error.py::test_foo
0.02s call     sandbox/mwe/5493_montage_index_error.py::test_foo

(0.00 durations hidden.  Use -vv to show these durations.)

Results (1.55s):
       1 passed

massich · 2019-03-13T13:23:37Z

This PR goes after #6047 and misses whatsnew

massich · 2019-03-18T14:11:51Z

rootdir: /home/sik/code/mne-python, inifile: setup.cfg
plugins: timeout-1.3.3, sugar-0.9.2, mock-1.10.1, faulthandler-1.5.0, cov-2.6.1
timeout: 30.0s
timeout method: signal
timeout func_only: False
collecting ... 1526

 sandbox/mwe/5493_montage_index_error.py ✓                                       100% ██████████
--------------- generated xml file: /home/sik/code/mne-python/junit-results.xml ----------------
================================== slowest 20 test durations ===================================
1.29s setup    sandbox/mwe/5493_montage_index_error.py::test_foo
0.01s call     sandbox/mwe/5493_montage_index_error.py::test_foo

(0.00 durations hidden.  Use -vv to show these durations.)

Results (1.62s):
       1 passed

Compilation finished at Mon Mar 18 15:10:42

It reads without error and recovers 1526 events

agramfort · 2019-03-18T16:51:27Z

thx @massich

* Fix: use annotations to create the backward compatible events * Fix: use the dataformat to parse int32 annotations * update whats_new

massich force-pushed the fix_montage_index_error branch from 839b993 to 49098a1 Compare March 4, 2019 17:10

massich mentioned this pull request Mar 5, 2019

add IO support of .cnt file from Neuroscan #2815

Closed

massich force-pushed the fix_montage_index_error branch 4 times, most recently from e35487f to f58a20f Compare March 8, 2019 18:46

massich commented Mar 11, 2019

View reviewed changes

massich mentioned this pull request Mar 12, 2019

Add Annotations to CNT + add code to deprecate stim_channel #6047

Merged

4 tasks

massich force-pushed the fix_montage_index_error branch 3 times, most recently from 56dc942 to f08f77b Compare March 13, 2019 12:41

massich force-pushed the fix_montage_index_error branch from f08f77b to 50533ce Compare March 18, 2019 10:33

Joan Massich added 3 commits March 18, 2019 15:00

Fix: use annotations to create the backward compatible events

ffb1866

Fix: use the dataformat to parse int32 annotations

203a563

update whats_new

4f73d67

massich marked this pull request as ready for review March 18, 2019 14:06

massich force-pushed the fix_montage_index_error branch from 50533ce to 4f73d67 Compare March 18, 2019 14:06

agramfort approved these changes Mar 18, 2019

View reviewed changes

agramfort merged commit f15e4bc into mne-tools:master Mar 18, 2019

massich deleted the fix_montage_index_error branch May 27, 2019 12:12

massich mentioned this pull request Jul 10, 2019

FIX, CNT: Use a more robust event_table_pos && n_bytes inference #6550

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

FIX: Montage Index error when importing CNT files#6025

FIX: Montage Index error when importing CNT files#6025
agramfort merged 3 commits intomne-tools:masterfrom
massich:fix_montage_index_error

massich commented Mar 4, 2019 •

edited

Loading

Uh oh!

codecov bot commented Mar 4, 2019 •

edited

Loading

Uh oh!

massich commented Mar 11, 2019

Uh oh!

massich left a comment

Uh oh!

massich commented Mar 13, 2019

Uh oh!

massich commented Mar 13, 2019

Uh oh!

massich commented Mar 18, 2019

Uh oh!

agramfort commented Mar 18, 2019

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

massich commented Mar 4, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codecov bot commented Mar 4, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

massich commented Mar 11, 2019

Uh oh!

massich left a comment

Choose a reason for hiding this comment

Uh oh!

massich commented Mar 13, 2019

Uh oh!

massich commented Mar 13, 2019

Uh oh!

massich commented Mar 18, 2019

Uh oh!

agramfort commented Mar 18, 2019

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

massich commented Mar 4, 2019 •

edited

Loading

codecov bot commented Mar 4, 2019 •

edited

Loading