WIP: Adds support for fNIRS data to set_montage and new standard artinis montage #9141

rob-luke · 2021-03-18T06:52:45Z

Reference issue

Add montage for Artinis OctaMon and Brite23 Fixes #9093

What does this implement/fix?

This adds support for fNIRS data to set_montage
Add a new standard montage for the artinis brand device as used in @HanBnrd's lab

Additional information

This PR has had feature creep and now contains a suggestion for importing data as csv files, to support the reading of unstructured data this PR now contains new code to check the naming of channels, etc. I have suggested reverting this PR back to its original goal of adding fNIRS support to .set_montage() and providing two sample montage files.

rob-luke · 2021-03-18T06:56:05Z

mne/channels/data/montages/artinis-brite23.txt

@@ -0,0 +1,58 @@
+MNI Coordinates


Once we are sure the elc file is correct we can delete this. Must be done before merge.

For those playing along at home...

In MNE we use the terminology source (S) and detector (D). These artinis files use the terminology transmitter (Tx) and receiver (Rx).

rob-luke · 2021-03-18T06:56:14Z

mne/channels/data/montages/artinis-octamon.txt

@@ -0,0 +1,35 @@
+MNI Coordinates


Once we are sure the elc file is correct we can delete these. Must be done before merge.

tutorials/io/plot_30_reading_fnirs_data.py

rob-luke · 2021-03-18T07:21:03Z

Hi @HanBnrd,

I cant make the sprint in your time zone today so I have tidied up our fork a little so I could share my thoughts with you. I have added a temporary change to one of the docs so we can discuss some questions I had. You can view the render of the doc change here: https://26156-1301584-gh.circle-artifacts.com/0/dev/auto_tutorials/io/plot_30_reading_fnirs_data.html#view-location-of-sensors-over-brain-surface

Now to my questions to you...

Is this the API/workflow that you were thinking about?

We have ended up with this workflow...

raw_intensity = load_some_data()
montage = make_standard_montage("artinis-octamon")
raw_intensity.set_montage(montage)
# ... processing as usual

will this work for you? If not, what's an alternative API we could use?

Locations look wrong

Currently the sources and detectors in 3d look like...

From my understanding the sources and detectors should be over the frontal cortex, and the arrangement should be in a dual cross configuration. So this looks close (its frontal and two nice distinct crosses), but its getting squished in to the brain.

I think we have a coordinate system mismatch going on somewhere, either in the raw files or in the loading, storage, or plotting.

Could you dig in to this a bit further? If we can get the sensor positions plotting correctly we can then go back and clean up our functions, make them more generic, add tests etc. @larsoner has helped me with coord system issues before, but he isn't on your timezone either. You might need to ask around in the available mentors room for someone who can help.

HanBnrd · 2021-03-18T10:27:56Z

Hi @rob-luke,

Is this the API/workflow that you were thinking about?

Yes the API looks good to me :)

Locations look wrong

At the moment I'm having a look at info['dig'] to see if it may come from that, because at the moment we just modify the info['chs'][i]['loc'].
Having a look at _set_montage() it is updating in place 3 things as far as I understand:

info['chs'][i]['loc']
info['dig']
info['dev_head_t'] (which seems to be only useful for MEG so not our case)

rob-luke · 2021-03-18T12:00:17Z

For reference here is a picture of the location of optodes as provided by the vendor.

HanBnrd · 2021-03-18T12:54:15Z

It's probably not an issue related to info['dig'] but I guess it's better to update it as well.

larsoner · 2021-03-18T20:02:40Z

mne/channels/montage.py


+        fnirs_picks = _picks_to_idx(info, 'fnirs', allow_empty=True)
+        if len(fnirs_picks) > 0:
+            info = _set_montage_fnirs(info, montage)


To avoid the pain of the next person errantly using montage instead of mnt_head, after conversion above we could del montage so that it's no longer in the namespace as a strong hint that the montage-in-head should be used by the subsequent code.

HanBnrd · 2021-03-18T20:44:01Z

This is how it looks now with the fix thanks to @larsoner's help :)

mne/channels/montage.py

rob-luke · 2021-03-18T20:50:48Z

Nice! I had a feeling it was the coord frame 🤔 Now that things are correct we just need to make the code nice and add tests, docs etc.

But before that, can you try this on your local data and confirm it helps to load your labs files, that's why we are doing this :)

HanBnrd · 2021-03-18T21:18:54Z

Now that things are correct we just need to make the code nice and add tests, docs etc.
But before that, can you try this on your local data and confirm it helps to load your labs files, that's why we are doing this :)

Yes, I'll do that tomorrow :)

HanBnrd · 2021-03-18T21:21:20Z

Direct link to the doc:
https://26236-1301584-gh.circle-artifacts.com/0/dev/auto_tutorials/io/plot_30_reading_fnirs_data.html#fake-some-data

tutorials/io/plot_30_reading_fnirs_data.py

HanBnrd · 2021-03-19T08:31:56Z

Hi @rob-luke,

So it works with data from the actual OctaMon. I used a file from an experiment where I exported data in real time from OxySoft (Artinis' software) into a CSV thanks to TCP.
See the experiment interface here.

import mne
import numpy as np

from mne.channels import make_standard_montage
from mne.channels.tests.test_standard_montage import _simulate_artinis_octamon


data = []
f = open(PATH + 'raw.csv', 'r', encoding='utf8')
f.readline()  # skip headers
for line in f:
    data.append(line.split('\n')[0].split(',')[1:])  # first col is timestamp
data = np.array(data, dtype='float64')
data = data * 1e-6  # uM to M
data = np.swapaxes(data, 0, 1)

ch_names = ['D1_S1 hbo', 'D1_S1 hbr', 'D1_S2 hbo', 'D1_S2 hbr',
            'D1_S3 hbo', 'D1_S3 hbr', 'D1_S4 hbo', 'D1_S4 hbr',
            'D2_S5 hbo', 'D2_S5 hbr', 'D2_S6 hbo', 'D2_S6 hbr',
            'D2_S7 hbo', 'D2_S7 hbr', 'D2_S8 hbo', 'D2_S8 hbr']
ch_types = ['hbo', 'hbr', 'hbo', 'hbr',
            'hbo', 'hbr', 'hbo', 'hbr',
            'hbo', 'hbr', 'hbo', 'hbr',
            'hbo', 'hbr', 'hbo', 'hbr']
sfreq = 10.  # Hz
info = mne.create_info(ch_names=ch_names, ch_types=ch_types, sfreq=sfreq)
raw = mne.io.RawArray(data, info, verbose=True)

raw.set_montage('artinis-octamon')

raw.plot_sensors()

subjects_dir = mne.datasets.sample.data_path() + '/subjects'
mne.datasets.fetch_fsaverage(subjects_dir=subjects_dir, verbose=True)

fig = mne.viz.create_3d_figure(size=(800, 600), bgcolor='white')
fig = mne.viz.plot_alignment(raw.info, show_axes=True,
                             subject='fsaverage', coord_frame='mri',
                             trans='fsaverage', surfaces=['brain', 'head'],
                             fnirs=['channels', 'pairs',
                                    'sources', 'detectors'],
                             dig=True, mri_fiducials=True,
                             subjects_dir=subjects_dir, fig=fig)
mne.viz.set_3d_view(figure=fig, azimuth=70, elevation=100, distance=0.4,
                    focalpoint=(0., -0.01, 0.02))

Unfortunately I cannot share the data used here with the above code.

Also, once there are loaders for oxy3 and oxy4 files (the proprietary format) it'll be more straight forward hopefully.

I'll try to see if I can make some artificial evoked topo like this.

HanBnrd · 2021-03-19T09:11:12Z

epochs = mne.make_fixed_length_epochs(raw, duration=5.)
epochs.baseline = (None, 1)
times = np.arange(0., 4.9, 1.)
topomap_args = dict(extrapolate='local')
epochs.average(picks='hbo').plot_joint(times=times, topomap_args=topomap_args)

Evoked plot working fine :)

HanBnrd · 2021-03-19T09:32:32Z

Just so we have this somewhere:
The vendor confirmed that the montages are for adult's head.

rob-luke

If we are to demonstrate how to generate raw and info from unsuported data, should we somewhere put in a description of the mandatory formatting? I described the mandatory rules at d42951d

And I assume the issue with the missaligned brite23 montage is fixed?

And I cant see a render of the docs in the jobs list, is it missing or have I lost my mind?

mne/channels/montage.py

rob-luke · 2021-04-22T03:35:40Z

mne/channels/montage.py

+                                      _check_channels_ordered)
+    # Modify info['chs'][#]['loc'] in place
+    num_ficiduals = len(montage.dig) - len(montage.ch_names)
+    freqs = np.unique(_channel_frequencies(info))


Suggested change

freqs = np.unique(_channel_frequencies(info))

# First we must validate that the fNIRS info is correctly formatted

freqs = np.unique(_channel_frequencies(info))

rob-luke · 2021-04-22T03:36:33Z

mne/channels/montage.py


        for name, use in zip(info_names, info_names_use):
            _loc_view = info['chs'][info['ch_names'].index(name)]['loc']
+            # XXX info['chs'][#]['loc'] modified in place


Is this related to a change we made?

It is not directly related but I wrote those comments to pinpoint all the lines where the in-place modifications were happening as it wasn't always obvious at first glance for example for this line 907.

Should I remove them or is it okay?

such comments are useful, but it would be clearer if it didn't start with XXX as this is usually a signal for "something to be fixed later / eventually". Can you change them to say "next line modifies info['chs'][#]['loc'] in-place" or similar?

rob-luke · 2021-04-22T03:36:54Z

mne/channels/montage.py

            # in the old dig
            if ref_dig_point in old_dig:
                digpoints.append(ref_dig_point)
+        # XXX info['dig'] modified in place


rob-luke · 2021-04-22T03:37:13Z

mne/channels/montage.py

+            info = _set_montage_fnirs(info, mnt_head)
+
    else:  # None case
+        # XXX info['dig'] modified in place


rob-luke · 2021-04-22T03:59:40Z

tutorials/io/plot_30_reading_fnirs_data.py

+by MNE and cannot be converted easily into SNIRF. However if it is possible to
+store this data in a legacy csv or tsv format, we show here a way to load it


Suggested change

by MNE and cannot be converted easily into SNIRF. However if it is possible to

store this data in a legacy csv or tsv format, we show here a way to load it

by MNE and cannot be converted easily into SNIRF. However if it is possible to

This legacy data is often in csv or tsv format, we show here a way to load it

this suggestion leaves the previous sentence beginning dangling... be sure to remove the "however if it is possible to" part

rob-luke · 2021-04-22T04:00:06Z

tutorials/io/plot_30_reading_fnirs_data.py

+standardisation of the file format (the naming and ordering of channels, the
+type and scaling of data, and specification of sensor positions varies between
+each vendor). You will likely have to adapt this depending on the system from
+which your CSV originated.


sometimes its csv, other times CSV. which is correct?

All changed to capital letters as they are acronyms.

Co-authored-by: Robert Luke <748691+rob-luke@users.noreply.github.com>

HanBnrd · 2021-04-22T08:31:46Z

Hi @rob-luke, thanks for reviewing.

If we are to demonstrate how to generate raw and info from unsuported data, should we somewhere put in a description of the mandatory formatting? I described the mandatory rules at d42951d

It was in an in-line comment but I just made it a warning now.

And I assume the issue with the missaligned brite23 montage is fixed?

Yes it is solved by not using trans='fsaverage' with Artinis montages (cf. this comment).

And I cant see a render of the docs in the jobs list, is it missing or have I lost my mind?

I think it was because of the checks timing out, hopefully this will work now. Will paste the link here when it's built.

rob-luke · 2021-04-22T08:49:27Z

Ok your turn @larsoner 😃

HanBnrd · 2021-04-22T10:22:48Z

And I cant see a render of the docs in the jobs list, is it missing or have I lost my mind?

I think it was because of the checks timing out, hopefully this will work now. Will paste the link here when it's built.

Actually CircleCI is not running it seems, I'm not sure where to find the built docs.

HanBnrd · 2021-04-22T11:34:08Z

Hi @rob-luke, an important point related to #9141 (comment): I just realised that it is actually possible to read any elc file (see #9309 (comment)) and our PR will make it work for fNIRS too:

montage = read_custom_montage('path/to/artinis-brite23.elc', coord_frame='mri')
raw.set_montage(montage)

(This is probably what has been tried in #8149)

We just need to have coord_frame='mri' so that we can compute the head_t later for plot_alignment (because plot_alignment cannot use 'fsaverge' otherwise it's shifted):

trans = compute_native_head_t(montage)

Should we write something about this in the example?

I think you can add this in the PR description, and also that our PR will close #8149.

drammock · 2021-04-22T15:18:01Z

I'm not sure why CircleCI isn't running here. The first thing I'd try is resovling the merge conflict on latest.inc and see if that triggers it to run.

larsoner · 2021-04-22T15:47:26Z

This recently happened to @cbrnr , I think because he revoked some permissions for CircleCI on his user (?). You could look at your settings.

Also see if you can view https://app.circleci.com/pipelines/github/mne-tools/mne-python/7826/workflows/c965a22a-19cf-402a-b92f-633247c4b948/jobs/27243, I hope it asks you to log in or for permission, and then you can view it. This might fix things

HanBnrd · 2021-04-22T18:48:50Z

Docs:

drammock

this is looking very good @HanBnrd. Just 2 comments.

drammock · 2021-04-22T19:40:40Z

tutorials/io/30_reading_fnirs_data.py

+
+
+###############################################################################
+# Storing of optode locations


this whole section seems almost like a "note" that applies to all the other sections. Maybe this info belongs at the very top of the tutorial, before the NIRx section?

drammock · 2021-04-22T19:43:52Z

tutorials/io/30_reading_fnirs_data.py

+# We simulate 16 channels with 100 samples of data and save this to a file
+# called fnirs.csv.
+
+pd.DataFrame(np.random.normal(size=(16, 100))).to_csv("fnirs.csv")


I wonder if it is really necessary to write out the CSV and then re-load it? Would it be just as good to say something like "here we'll make some fake data using pd.DataFrame; in practice you'd use pd.read_csv instead to load your file"?

I think that some users won't know how to read a csv. And this will demonstrate it.

fair enough.

larsoner · 2021-04-23T14:23:16Z

@HanBnrd please click "Ready for review" if this is good to go, and let me know

It sounds like @rob-luke and @drammock are happy so if it is good to go I'll give it one more look and hopefully merge!

HanBnrd · 2021-04-23T15:57:09Z

I just pushed one more commit implementing @drammock's suggestion and @rob-luke's comment.

Here is the new tutorial: https://27748-1301584-gh.circle-artifacts.com/0/dev/auto_tutorials/io/30_reading_fnirs_data.html

That's ready for review now @larsoner.

larsoner · 2021-04-23T18:56:46Z

Great, thanks @HanBnrd @rob-luke !

HanBnrd · 2021-04-23T19:01:56Z

Thanks everybody! 🎉

HanBnrd and others added 9 commits March 17, 2021 08:21

Add raw montage files sent by Artinis

e0f8cc6

Initial pseudo code

2159fb3

Tweaks

44cbe02

Draft _set_montage_fnirs

5dae7fa

Tweaks

e55859a

Example with optode montage as elc file

cebdf10

Clean up and add Brite 23

9064883

Merge branch 'main' into artinis-montages

4091332

Tweak skeleton code and push example

9bbf7a8

rob-luke commented Mar 18, 2021

View reviewed changes

Add TODOs

d65d8f8

Update also dig

7428812

larsoner reviewed Mar 18, 2021

View reviewed changes

Fix optodes in the brain

599bbfd

HanBnrd reviewed Mar 18, 2021

View reviewed changes

mne/channels/montage.py Show resolved Hide resolved

HanBnrd added 2 commits March 18, 2021 21:07

Add fetch_fsaverage

b77459c

Merge branch 'main' into artinis-montages

801c27f

larsoner reviewed Mar 19, 2021

View reviewed changes

tutorials/io/plot_30_reading_fnirs_data.py Outdated Show resolved Hide resolved

Set montage with str in tutorial

28ff382

rob-luke commented Apr 22, 2021

View reviewed changes

HanBnrd and others added 3 commits April 22, 2021 09:10

Apply suggestions from code review

4690830

Co-authored-by: Robert Luke <748691+rob-luke@users.noreply.github.com>

Merge branch 'main' into artinis-montages

726e282

Fix ref

20aa229

HanBnrd mentioned this pull request Apr 22, 2021

SNIRF loader not working without the optional sourcePos3D #9308

Closed

HanBnrd added 3 commits April 22, 2021 18:47

Update in-line comments

de264a0

Merge branch 'main' into artinis-montages

bade068

Fix refs

c695f04

drammock reviewed Apr 22, 2021

View reviewed changes

larsoner added this to the 0.23 milestone Apr 23, 2021

rob-luke marked this pull request as ready for review April 23, 2021 14:25

Move channel info and add read_custom_montage

0ea038a

larsoner merged commit e4a1571 into mne-tools:main Apr 23, 2021

HanBnrd mentioned this pull request Apr 23, 2021

Custom montage object not loading to NIRS info #8149

Closed

HanBnrd deleted the artinis-montages branch April 23, 2021 21:29

This was referenced May 6, 2021

MRG: Add more checks to ensure fNIRS bads are set correctly for optode pairs #9375

Merged

MRG, ENH: Add support for Hitachi fNIRS #9391

Merged

larsoner mentioned this pull request Feb 11, 2022

BUG: Fix bug with scaling and fids for standard montages #10324

Merged

rob-luke mentioned this pull request May 7, 2022

Add get_montage support for fNIRS #10611

Merged

	freqs = np.unique(_channel_frequencies(info))
	# First we must validate that the fNIRS info is correctly formatted
	freqs = np.unique(_channel_frequencies(info))

		by MNE and cannot be converted easily into SNIRF. However if it is possible to
		store this data in a legacy csv or tsv format, we show here a way to load it



		###############################################################################
		# Storing of optode locations

Uh oh!

WIP: Adds support for fNIRS data to set_montage and new standard artinis montage #9141

WIP: Adds support for fNIRS data to set_montage and new standard artinis montage #9141

Uh oh!

Conversation

rob-luke commented Mar 18, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Reference issue

What does this implement/fix?

Additional information

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

rob-luke commented Mar 18, 2021

Is this the API/workflow that you were thinking about?

Locations look wrong

Uh oh!

HanBnrd commented Mar 18, 2021

Uh oh!

rob-luke commented Mar 18, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

HanBnrd commented Mar 18, 2021

Uh oh!

Choose a reason for hiding this comment

Uh oh!

HanBnrd commented Mar 18, 2021

Uh oh!

Uh oh!

rob-luke commented Mar 18, 2021

Uh oh!

HanBnrd commented Mar 18, 2021

Uh oh!

HanBnrd commented Mar 18, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

HanBnrd commented Mar 19, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

HanBnrd commented Mar 19, 2021

Uh oh!

HanBnrd commented Mar 19, 2021

Uh oh!

rob-luke left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

HanBnrd commented Apr 22, 2021

Uh oh!

rob-luke commented Apr 22, 2021

Uh oh!

HanBnrd commented Apr 22, 2021

rob-luke commented Mar 18, 2021 •

edited

Loading

rob-luke commented Mar 18, 2021 •

edited

Loading

HanBnrd commented Mar 18, 2021 •

edited

Loading

HanBnrd commented Mar 19, 2021 •

edited

Loading