MRG: Add Brain class based on _Renderer by GuillaumeFavelier · Pull Request #6460 · mne-tools/mne-python

GuillaumeFavelier · 2019-06-17T08:55:43Z

EDIT: In the last commit, Brain has been renamed into _Brain from mne.viz._brain. The code snippets below are not up to date.

While work is still in progress in #6232 to fix transparency issues, I thought it would be great to move forward the refactoring of the source code based on PySurfer. This PR introduces the class Brain using the 3d abstraction layer _Renderer.

This PR is just the first step, the goal after is to refactor/test and replace parts of the source code relying on PySurfer.

Now I show some visualizations and code snippets using the latest 3d backend merged recently PyVista:

Basic Visualization

import mne
import os.path as path
from mne.viz import Brain, use_3d_backend
data_path = mne.datasets.sample.data_path()

subject_id = 'sample'
subjects_dir = path.join(data_path, 'subjects')
hemi = 'both'
surf = 'inflated'

with use_3d_backend("pyvista"):
    brain = Brain(subject_id, hemi, surf, size=300, subjects_dir=subjects_dir)
    brain.show()

Activation Data Visualization

import mne
import os.path as path
from mne.viz import Brain, set_3d_backend
data_path = mne.datasets.sample.data_path()

subject_id = 'sample'
subjects_dir = path.join(data_path, 'subjects')
surf = 'inflated'
act_data = path.join(data_path, 'MEG/sample/sample_audvis-meg-eeg')
stc = mne.read_source_estimate(act_data)
hemi = 'lh'
hemi_data = stc.data[:len(stc.vertices[0]), 10]
hemi_vertices = stc.vertices[0]
fmin = stc.data.min()
fmax = stc.data.max()

set_3d_backend("pyvista")
brain = Brain(subject_id, hemi, surf, size=300, subjects_dir=subjects_dir)
brain.add_data(hemi_data, fmin=fmin, hemi=hemi, fmax=fmax, colormap='hot',
               vertices=hemi_vertices, colorbar=False)

brain.show()

Display ROI Values

import numpy as np
import os.path as path
import mne
from mne.viz import Brain, set_3d_backend
import os
import nibabel as nib

subject_id = "fsaverage"
data_path = mne.datasets.sample.data_path()
subjects_dir = path.join(data_path, 'subjects')
hemi = "lh"
surf = "inflated"

set_3d_backend("pyvista")
brain = Brain(subject_id, hemi, surf, background=(0, 0, 0),
              subjects_dir=subjects_dir)

aparc_file = os.path.join(subjects_dir,
                          subject_id, "label",
                          hemi + ".aparc.a2009s.annot")
labels, ctab, names = nib.freesurfer.read_annot(aparc_file)
rs = np.random.RandomState(4)
roi_data = rs.uniform(.5, .8, size=len(names))
vtx_data = roi_data[labels]
vtx_data[labels == -1] = -1

brain.add_data(vtx_data, .5, .75, colormap="jet", alpha=.8)
brain.show()

GuillaumeFavelier · 2019-06-17T09:48:20Z

Most of the code comes from #6232 stripped from ipyvolume dependencies. What do you think @larsoner , @agramfort ?

codecov · 2019-06-17T15:22:00Z

Codecov Report

Merging #6460 into master will decrease coverage by 80.82%.
The diff coverage is 13.01%.

@@            Coverage Diff             @@
##           master   #6460       +/-   ##
==========================================
- Coverage   89.27%   8.44%   -80.83%     
==========================================
  Files         411     402        -9     
  Lines       74697   73403     -1294     
  Branches    12342   12248       -94     
==========================================
- Hits        66684    6200    -60484     
- Misses       5149   67148    +61999     
+ Partials     2864      55     -2809

codecov · 2019-06-17T15:37:51Z

Codecov Report

Merging #6460 into master will decrease coverage by 0.04%.
The diff coverage is 81.74%.

@@            Coverage Diff             @@
##           master    #6460      +/-   ##
==========================================
- Coverage   89.27%   89.22%   -0.05%     
==========================================
  Files         411      416       +5     
  Lines       74697    74371     -326     
  Branches    12342    12287      -55     
==========================================
- Hits        66684    66356     -328     
- Misses       5149     5154       +5     
+ Partials     2864     2861       -3

GuillaumeFavelier · 2019-06-17T16:51:19Z

Thanks to the discussion with the PyVista team, the latest fix allows us now to use the Brain object in a Jupyter Notebook interactively!

agramfort · 2019-06-17T19:27:07Z

nice ! it works with add_data etc?

…

GuillaumeFavelier · 2019-06-18T06:44:38Z

I can't upload a GIF right now but basically the three examples I shared in this thread works so basic usage of add_data() is indeed supported.

larsoner · 2019-06-18T15:36:33Z

How close are we then to having stc.plot() work?

Maybe the plan should be:

Review/merge this PR
Refactor stc.plot to use this new object
Add time_viewer=True support for each backend (where possible)

WDYT?

GuillaumeFavelier · 2019-06-19T09:03:58Z

How close are we then to having stc.plot() work?

Not everything is available yet but we can definitely add the missing features incrementally. I can also create a RST table to improve visibility on this work. Benchmarking this Brain object against the PySurfer example gallery will help as well.

Maybe the plan should be:

1. Review/merge this PR

2. Refactor `stc.plot` to use this new object

3. Add `time_viewer=True` support for each backend (where possible)

Sounds good to me!

larsoner · 2019-06-19T12:05:20Z

Benchmarking this Brain object against the PySurfer example gallery will help as well.

We probably do not need all of those examples, but we can go case by case. Let's aim to make sure stc.plot(...) works to start. Can I already try examples that use stc.plot and expect them to work with this PR?

Not everything is available yet but we can definitely add the missing features incrementally. I can also create a RST table to improve visibility on this work.

+1 for both.

GuillaumeFavelier · 2019-06-19T12:33:00Z

Can I already try examples that use stc.plot and expect them to work with this PR?

This PR adds the class Brain but stc.plot still uses PySurfer . I wanted to continue refactoring with another PR to be honest.

larsoner · 2019-06-19T13:05:46Z

Okay sounds good.

Running the code snippet from the top post with latest pyvista I see:

Scroll wheeling a lot to zoom out I get something more reasonable:

Do you see the same thing?

GuillaumeFavelier · 2019-06-19T13:10:26Z

Yes indeed!

GuillaumeFavelier · 2019-06-19T13:14:18Z

I can find a suitable value for the distance parameter in this PR if you want? It will fix the scrolling issue.

larsoner · 2019-06-19T13:15:56Z

You can just add it to a todo list in a follow-up PR. It will quickly become annoying to test stc.plot if this value is not set properly so I trust it will be fixed very soon :)

larsoner · 2019-06-19T13:18:58Z

One thing that I'd like to do for now, though, is make it _Brain and remove it from python_reference.rst.

The reason for this is that we might want to:

Rename it. Something like BrainSurfaces is more apt if we want to have volumetric and surface source spaces on the same footing.
Change the API. Right now it mimics PySurfer, which is nice to start. But this is our chance to fix things that annoy us about the API.

Okay with you to keep it private for a bit @GuillaumeFavelier ?

GuillaumeFavelier · 2019-06-19T13:22:04Z

Yes sure, I'll rename it _Brain for now.

larsoner · 2019-06-19T16:01:41Z

Thanks @GuillaumeFavelier, keep up the progress!

GuillaumeFavelier added 3 commits June 14, 2019 19:05

Add brain object

8921aa2

Update 3d backends

db76b04

Add minor edits

e1d2153

GuillaumeFavelier added the VIZ label Jun 17, 2019

GuillaumeFavelier self-assigned this Jun 17, 2019

GuillaumeFavelier changed the title ~~Add Brain object based on _Renderer~~ WIP: Add Brain class based on _Renderer Jun 17, 2019

GuillaumeFavelier changed the title ~~WIP: Add Brain class based on _Renderer~~ MRG: Add Brain class based on _Renderer Jun 17, 2019

Fix pyvista rendering bugs inside notebook

244f2a0

Remove clear_arrays() call

4cf82d5

Rename Brain into _Brain

1fa7878

larsoner merged commit ad305e0 into mne-tools:master Jun 19, 2019

GuillaumeFavelier deleted the brain_renderer branch June 19, 2019 16:01

GuillaumeFavelier mentioned this pull request Jun 20, 2019

MRG, FIX: Fix default camera settings for _Brain object #6475

Merged

GuillaumeFavelier mentioned this pull request Jul 1, 2019

Add brain functions table for overview #6515

Merged

GuillaumeFavelier mentioned this pull request Sep 3, 2019

MRG: Use _Brain in plot_source_estimates #6731

Merged

31 tasks

Uh oh!

Conversation

GuillaumeFavelier commented Jun 17, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Basic Visualization

Activation Data Visualization

Display ROI Values

Uh oh!

GuillaumeFavelier commented Jun 17, 2019

Uh oh!

codecov bot commented Jun 17, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

codecov bot commented Jun 17, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

GuillaumeFavelier commented Jun 17, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

agramfort commented Jun 17, 2019 via email

Uh oh!

GuillaumeFavelier commented Jun 18, 2019

Uh oh!

larsoner commented Jun 18, 2019

Uh oh!

GuillaumeFavelier commented Jun 19, 2019

Uh oh!

larsoner commented Jun 19, 2019

Uh oh!

GuillaumeFavelier commented Jun 19, 2019

Uh oh!

larsoner commented Jun 19, 2019

Uh oh!

GuillaumeFavelier commented Jun 19, 2019

Uh oh!

GuillaumeFavelier commented Jun 19, 2019

Uh oh!

larsoner commented Jun 19, 2019

Uh oh!

larsoner commented Jun 19, 2019

Uh oh!

GuillaumeFavelier commented Jun 19, 2019

Uh oh!

larsoner commented Jun 19, 2019

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

GuillaumeFavelier commented Jun 17, 2019 •

edited

Loading

codecov bot commented Jun 17, 2019 •

edited

Loading

codecov bot commented Jun 17, 2019 •

edited

Loading

GuillaumeFavelier commented Jun 17, 2019 •

edited

Loading