Skip to content

[MRG] Handle nested structures#904

Merged
larsoner merged 17 commits intosphinx-gallery:masterfrom
alexisthual:feature/nested_sections
Mar 8, 2022
Merged

[MRG] Handle nested structures#904
larsoner merged 17 commits intosphinx-gallery:masterfrom
alexisthual:feature/nested_sections

Conversation

@alexisthual
Copy link
Copy Markdown
Contributor

@alexisthual alexisthual commented Jan 12, 2022

Fixes #855.

Let's assume the gallery structure is the following (note the new readme.rst files in each subsection):

- top_level
   - readme.rst
   - subdir
      - readme.rst
      - example.rst
      - example2.rst
   - subdir2
      - readme.rst
      - example1.rst
      - example2.rst

This PR:

  • generates an index.rst file for each subsection; it contains readme.rst, the gallery items, and a hidden toctree of the gallery items
  • includes each subsection's index file in a hidden toctree appended at the end of the main sections index file

On-going tasks:

  • passing tests
  • try more levels of nesting (EDIT: outcome is it currently works for 2, doesn't work for more)

@alexisthual alexisthual marked this pull request as draft January 12, 2022 13:24
@alexisthual alexisthual marked this pull request as ready for review January 12, 2022 15:10
@alexisthual
Copy link
Copy Markdown
Contributor Author

tests/test_full/test_rebuild() currently fails.

I incremented N_RST which fixes one of the errors (I don't understand what it was set to 15 + N_TOT in the first place though).
Now _assert_mtimes(generated_rst_0, generated_rst_1, ignore=ignore) fails.

I don't really understand what these tests do, could someone please help me? 🙂

@larsoner
Copy link
Copy Markdown
Contributor

I don't really understand what these tests do, could someone please help me? 

It ensures that only things are rebuilt that are changed.

I have not looked at your code, but if your PR does something like generate an output file, it should effectively only do this if it needs to, i.e., if it has changed. In practice we currently take care of this by using a system based on MD5 hashes: 1) create the new potential file with a whatever.rst.new extension (I think?), 2) call some private function to decide if it should mv to whatever.rst based on whether the MD5 hashes of the original and the .new file match or not.

@alexisthual
Copy link
Copy Markdown
Contributor Author

I don't really understand what these tests do, could someone please help me? 

It ensures that only things are rebuilt that are changed.

I have not looked at your code, but if your PR does something like generate an output file, it should effectively only do this if it needs to, i.e., if it has changed. In practice we currently take care of this by using a system based on MD5 hashes: 1) create the new potential file with a whatever.rst.new extension (I think?), 2) call some private function to decide if it should mv to whatever.rst based on whether the MD5 hashes of the original and the .new file match or not.

My PR indeed creates new files: it creates one new index.rst per subsection.
Where should I add these .new files?

@larsoner
Copy link
Copy Markdown
Contributor

Where should I add these .new files?

Take a look at how it's done elsewhere, e.g.:

https://github.com/sphinx-gallery/sphinx-gallery/blob/master/sphinx_gallery/gen_rst.py#L987-L989

But IIRC it's the same name as the old just plus the extension. So if you want to create an index.rst you create instead an index.rst.new.

@alexisthual
Copy link
Copy Markdown
Contributor Author

It is not clear to me why two tests are failing.

build_docs seems to run alright and finally returns

Exception occurred:
  File "/home/circleci/.local/lib/python3.8/site-packages/sphinx/setup_command.py", line 177, in run
    raise DistutilsExecError(
distutils.errors.DistutilsExecError: caused by html builder.
The full traceback has been saved in /tmp/sphinx-err-09hjf9gr.log, if you want to report the issue to the developers.
Please also report this if it was a user error, so that a better error message can be provided next time.
A bug report can be filed in the tracker at <https://github.com/sphinx-doc/sphinx/issues>. Thanks!

but I can't access the log file.

@larsoner
Copy link
Copy Markdown
Contributor

but I can't access the log file.

Scroll up in the output -- we treat warnings as errors, and there are a lot of warnings:

/home/circleci/project/doc/auto_examples/index.rst:297: WARNING: toctree contains reference to nonexisting document 'auto_examples/no_output/index'
looking for now-outdated files... none found
pickling environment... done
checking consistency... /home/circleci/project/doc/auto_examples/local_module.rst: WARNING: document isn't included in any toctree
/home/circleci/project/doc/auto_examples/no_output/just_code.rst: WARNING: document isn't included in any toctree
/home/circleci/project/doc/auto_examples/no_output/plot_raise.rst: WARNING: document isn't included in any toctree
/home/circleci/project/doc/auto_examples/no_output/plot_strings.rst: WARNING: document isn't included in any toctree
/home/circleci/project/doc/auto_examples/no_output/plot_syntaxerror.rst: WARNING: document isn't included in any toctree
/home/circleci/project/doc/auto_examples/plot_0_sin.rst: WARNING: document isn't included in any toctree
/home/circleci/project/doc/auto_examples/plot_1_exp.rst: WARNING: document isn't included in any toctree
/home/circleci/project/doc/auto_examples/plot_2_seaborn.rst: WARNING: document isn't included in any toctree
/home/circleci/project/doc/auto_examples/plot_3_capture_repr.rst: WARNING: document isn't included in any toctree
/home/circleci/project/doc/auto_examples/plot_4_choose_thumbnail.rst: WARNING: document isn't included in any toctree
/home/circleci/project/doc/auto_examples/plot_4b_provide_thumbnail.rst: WARNING: document isn't included in any toctree
/home/circleci/project/doc/auto_examples/plot_5_unicode_everywhere.rst: WARNING: document isn't included in any toctree
/home/circleci/project/doc/auto_examples/plot_6_function_identifier.rst: WARNING: document isn't included in any toctree
/home/circleci/project/doc/auto_examples/plot_7_sys_argv.rst: WARNING: document isn't included in any toctree
/home/circleci/project/doc/auto_examples/plot_8_animations.rst: WARNING: document isn't included in any toctree
/home/circleci/project/doc/auto_examples/plot_9_plotly.rst: WARNING: document isn't included in any toctree
/home/circleci/project/doc/auto_mayavi_examples/plot_3d.rst: WARNING: document isn't included in any toctree
/home/circleci/project/doc/auto_pyvista_examples/plot_collisions.rst: WARNING: document isn't included in any toctree
/home/circleci/project/doc/auto_pyvista_examples/plot_glyphs.rst: WARNING: document isn't included in any toctree
/home/circleci/project/doc/auto_pyvista_examples/plot_lighting.rst: WARNING: document isn't included in any toctree
/home/circleci/project/doc/auto_pyvista_examples/plot_ray_trace.rst: WARNING: document isn't included in any toctree
/home/circleci/project/doc/tutorials/plot_parse.rst: WARNING: document isn't included in any toctree

You should be able to replicate locally in the doc folder by doing make html I think

@alexisthual
Copy link
Copy Markdown
Contributor Author

I think this PR is ready for review now. Note that I tried the implementation from this PR with subsubsections, but the tests won't pass easily, so I didn't investigate any further.

@alexisthual alexisthual changed the title [WIP] handle nested structures [WIP] Handle nested structures Jan 19, 2022
@larsoner
Copy link
Copy Markdown
Contributor

@lucyleeow can you look?

@lucyleeow
Copy link
Copy Markdown
Contributor

I think I'll have time in a week or two (sorry very busy atm)

@larsoner
Copy link
Copy Markdown
Contributor

larsoner commented Jan 24, 2022

Looks good but I don't see any added test that makes sure things actually work, just modified existing tests to make sure nothing breaks.

The easiest way to add a test is to modify sphinx_gallery/tests/tinybuild to have EDIT: make use of the new feature, then assert that it is built correctly (in some reasonable way). tinybuild is basically a tiny demo site that gets built (and then rebuilt) from scratch. You can iteratively test to see it if works correctly visually by running make in the tinybuild directory.

@alexisthual
Copy link
Copy Markdown
Contributor Author

Sorry it took me some time to get back to this :)

I gave it some thoughts these last few days, and I couldn't think of a nice test to add (we would need new tests if we handled sub-subsections, but this PR doesn't).

Besides, tinybuild already triggers this new feature: it has a subsection future, for which the current code tests that a new index.rst file is created.
I checked that the built documentation makes sense ; the navigation was affected in a way that is not right, but I reckon the way it is built is out of my jurisdiction (and not very standard 😄 - it seems to list headers and add their toctree items below).

I would advocate that changing the way the navigation is built would be the way to go (other people think it should reflect the site's structure and they convinced me too haha).

See before / after here:

IMHO, this PR doesn't need more tests ; let me know what you think!
Just to let you know, I tested this code on nilearn's documentation, and the generated toctree structure throughout the doc is what we want (in particular, see the navigation in the screenshot below).
Screenshot from 2022-02-08 18-50-51

@larsoner
Copy link
Copy Markdown
Contributor

Besides, tinybuild already triggers this new feature: it has a subsection future, for which the current code tests that a new index.rst file is created.

Okay great, that's enough for me then!

However, I do get new warnings on this PR when building the MNE-Python docs:

/home/larsoner/python/mne-python/doc/auto_examples/connectivity/index.rst:28: WARNING: toctree contains reference to nonexisting document ''
/home/larsoner/python/mne-python/doc/auto_examples/index.rst:35: WARNING: toctree contains reference to nonexisting document ''
/home/larsoner/python/mne-python/doc/auto_tutorials/index.rst:33: WARNING: toctree contains reference to nonexisting document ''
/home/larsoner/python/mne-python/doc/auto_tutorials/io/index.rst:8: WARNING: duplicate label tut-data-formats, other instance in /home/larsoner/python/mne-python/doc/auto_tutorials/index.rst

From the first three warnings, it looks like some documents are created that maybe shouldn't be (?), and then with the last warning, there is a problem where tutorials/io/README.txt:.. _tut-data-formats: ends up being duplicated and emits a warning.

I see something similar building MNE-NIRS:

/home/larsoner/python/mne-nirs/doc/auto_examples/index.rst:33: WARNING: toctree contains reference to nonexisting document ''
/home/larsoner/python/mne-nirs/doc/auto_examples/index.rst:41: WARNING: duplicate label general_examples, other instance in /home/larsoner/python/mne-nirs/doc/auto_examples/general/index.rst
/home/larsoner/python/mne-nirs/doc/auto_examples/migration/index.rst:8: WARNING: duplicate label migration, other instance in /home/larsoner/python/mne-nirs/doc/auto_examples/index.rst

Thoughts on those?

@alexisthual
Copy link
Copy Markdown
Contributor Author

alexisthual commented Mar 2, 2022

Thank you for testing! Sorry it took me so long to get back to you, I was busy lately 🙂

Here are some thoughts:

  • I think the toctree contains reference to nonexisting document '' warning is raised when an empty toctree is generated, which happens whenever no tutorial can be found in the same folder as a readme file ; I just pushed a fix for this, tell me if it works!
  • it is not clear to me why the second warning is raised (I couldn't find another tut-data-formats tag in mne-python). Could you please show me the content of auto_tutorials/io/index.rst so that I can have a better idea of where the problem could come from?

@alexisthual
Copy link
Copy Markdown
Contributor Author

alexisthual commented Mar 2, 2022

It seems that tests are failing because matplotlib changed their intersphinx references' names, so :mod:matplotlib.colors no longer yields the expected value. Eventually, a link to the matplotlib.colors module is missing.

sphinx_gallery\tests\test_backreferences.py .......                      [  3%]
sphinx_gallery\tests\test_binder.py ..                                   [  4%]
sphinx_gallery\tests\test_docs_resolv.py ...                             [  6%]
sphinx_gallery\tests\test_full.py .........F.......................      [ 22%]
sphinx_gallery\tests\test_full_noexec.py .                               [ 23%]
sphinx_gallery\tests\test_gen_gallery.py ............................... [ 39%]
.                                                                        [ 39%]
sphinx_gallery\tests\test_gen_rst.py ................................... [ 57%]
..................................                                       [ 74%]
sphinx_gallery\tests\test_load_style.py .                                [ 75%]
sphinx_gallery\tests\test_notebook.py ..............                     [ 82%]
sphinx_gallery\tests\test_py_source_parser.py .................          [ 90%]
sphinx_gallery\tests\test_scrapers.py ...s........                       [ 96%]
sphinx_gallery\tests\test_sorting.py .                                   [ 97%]
sphinx_gallery\tests\test_sphinx_compatibility.py ...                    [ 98%]
sphinx_gallery\tests\test_utils.py ..                                    [100%]

================================== FAILURES ===================================
_________________________ test_embed_links_and_styles _________________________
sphinx_gallery\tests\test_full.py:278: in test_embed_links_and_styles
    assert '#module-matplotlib.colors' in lines
E   assert '#module-matplotlib.colors' in '\r\n<!DOCTYPE html>\r\n\r\n<html>\r\n  <head>\r\n
<meta charset="utf-8" />\r\n    <meta name="viewport" content="w...lib.rst.txt"\r\n
rel="nofollow">Page source</a>\r\n    </div>\r\n\r\n    \r\n\r\n    \r\n  </body>\r\n</html>'

--------------- generated xml file: D:\a\1\s\junit-results.xml ----------------


============================= slowest 5 durations =============================

@larsoner
Copy link
Copy Markdown
Contributor

larsoner commented Mar 2, 2022

Feel free to update those here or in another PR if you want. If it's not clear what needs to be done, I could make a quick PR

@alexisthual
Copy link
Copy Markdown
Contributor Author

I gave it a quick look today but couldn't fix it right away, so I'll gladly accept your offer to make a quick PR 😄
My humble guess is that correcting line 7 from sphinx_gallery/tests/tinybuild/examples/plot_numpy_matplotlib.py should do the trick.

@larsoner
Copy link
Copy Markdown
Contributor

larsoner commented Mar 2, 2022

it is not clear to me why the second warning is raised (I couldn't find another tut-data-formats tag in mne-python). Could you please show me the content of auto_tutorials/io/index.rst so that I can have a better idea of where the problem could come from?

I pulled your latest changes, and now I get:

$ make html_dev-noplot
...
/home/larsoner/python/mne-python/doc/auto_examples/index.rst:35: WARNING: toctree contains reference to nonexisting document ''
/home/larsoner/python/mne-python/doc/auto_tutorials/index.rst:33: WARNING: toctree contains reference to nonexisting document ''
/home/larsoner/python/mne-python/doc/auto_tutorials/io/index.rst:8: WARNING: duplicate label tut-data-formats, other instance in /home/larsoner/python/mne-python/doc/auto_tutorials/index.rst

The contents of those two files are:

Built from tutorials/README.rst:

auto_tutorials/index.rst
:orphan:

Tutorials
=========

These tutorials provide narrative explanations, sample code, and expected
output for the most common MNE-Python analysis tasks. The emphasis here is on
thorough explanations that get you up to speed quickly, at the expense of
covering only a limited number of topics. The sections and tutorials are
arranged in a fixed order, so in theory a new user should be able to progress
through in order without encountering any cases where background
knowledge is assumed and unexplained. More experienced users (i.e., those with
significant experience analyzing EEG/MEG signals with different software) can
probably skip around to just the topics they need without too much trouble.

.. note::
    If tutorial-scripts contain plots and are run locally, using the
    interactive interactive flag with ``python -i tutorial_script.py``
    keeps them open.



.. raw:: html

    <div class="sphx-glr-thumbnails">


.. raw:: html

    </div>


.. toctree::
   :hidden:

   /

Introductory tutorials
-----------------------

These tutorials cover the basic EEG/MEG pipeline for event-related analysis,
introduce the :class:`mne.Info`, :term:`events`, and :class:`mne.Annotations`
data structures, discuss how sensor locations are handled, and introduce some
of the configuration options available.



.. raw:: html

    <div class="sphx-glr-thumbnails">


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="This tutorial covers the basic EEG/MEG pipeline for event-related analysis: loading data, epoch...">

.. only:: html

  .. image:: /auto_tutorials/intro/images/thumb/sphx_glr_10_overview_thumb.png
    :alt: Overview of MEG/EEG analysis with MNE-Python

  :ref:`sphx_glr_auto_tutorials_intro_10_overview.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Overview of MEG/EEG analysis with MNE-Python</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="Many of MNE-Python&#x27;s data objects (`~mne.io.Raw`, ~mne.Epochs, ~mne.Evoked, etc) have methods t...">

.. only:: html

  .. image:: /auto_tutorials/intro/images/thumb/sphx_glr_15_inplace_thumb.png
    :alt: Modifying data in-place

  :ref:`sphx_glr_auto_tutorials_intro_15_inplace.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Modifying data in-place</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="This tutorial describes how to read experimental events from raw recordings, and how to convert...">

.. only:: html

  .. image:: /auto_tutorials/intro/images/thumb/sphx_glr_20_events_from_raw_thumb.png
    :alt: Parsing events from raw data

  :ref:`sphx_glr_auto_tutorials_intro_20_events_from_raw.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Parsing events from raw data</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="This tutorial describes the mne.Info data structure, which keeps track of various recording det...">

.. only:: html

  .. image:: /auto_tutorials/intro/images/thumb/sphx_glr_30_info_thumb.png
    :alt: The Info data structure

  :ref:`sphx_glr_auto_tutorials_intro_30_info.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">The Info data structure</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="This tutorial describes how to read and plot sensor locations, and how MNE-Python handles physi...">

.. only:: html

  .. image:: /auto_tutorials/intro/images/thumb/sphx_glr_40_sensor_locations_thumb.png
    :alt: Working with sensor locations

  :ref:`sphx_glr_auto_tutorials_intro_40_sensor_locations.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Working with sensor locations</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="This tutorial covers how to configure MNE-Python to suit your local system and your analysis pr...">

.. only:: html

  .. image:: /auto_tutorials/intro/images/thumb/sphx_glr_50_configure_mne_thumb.png
    :alt: Configuring MNE-Python

  :ref:`sphx_glr_auto_tutorials_intro_50_configure_mne.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Configuring MNE-Python</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="mne.Report is a way to create interactive HTML summaries of your data. These reports can show m...">

.. only:: html

  .. image:: /auto_tutorials/intro/images/thumb/sphx_glr_70_report_thumb.png
    :alt: Getting started with mne.Report

  :ref:`sphx_glr_auto_tutorials_intro_70_report.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Getting started with mne.Report</div>
    </div>


.. raw:: html

    </div>

.. _tut-data-formats:

Reading data for different recording systems
--------------------------------------------

These tutorials cover the basics of loading EEG/MEG data into MNE-Python
for various recording devices.



.. raw:: html

    <div class="sphx-glr-thumbnails">


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="This section describes how to read data for various MEG manufacturers.">

.. only:: html

  .. image:: /auto_tutorials/io/images/thumb/sphx_glr_10_reading_meg_data_thumb.png
    :alt: Importing data from MEG devices

  :ref:`sphx_glr_auto_tutorials_io_10_reading_meg_data.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Importing data from MEG devices</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="MNE includes various functions and utilities for reading EEG data and electrode locations.">

.. only:: html

  .. image:: /auto_tutorials/io/images/thumb/sphx_glr_20_reading_eeg_data_thumb.png
    :alt: Importing data from EEG devices

  :ref:`sphx_glr_auto_tutorials_io_20_reading_eeg_data.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Importing data from EEG devices</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="fNIRS devices consist of two kinds of optodes: light sources (AKA &quot;emitters&quot; or &quot;transmitters&quot;)...">

.. only:: html

  .. image:: /auto_tutorials/io/images/thumb/sphx_glr_30_reading_fnirs_data_thumb.png
    :alt: Importing data from fNIRS devices

  :ref:`sphx_glr_auto_tutorials_io_30_reading_fnirs_data.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Importing data from fNIRS devices</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="Here we compute the evoked from raw for the auditory Brainstorm tutorial dataset. For compariso...">

.. only:: html

  .. image:: /auto_tutorials/io/images/thumb/sphx_glr_60_ctf_bst_auditory_thumb.png
    :alt: Working with CTF data: the Brainstorm auditory dataset

  :ref:`sphx_glr_auto_tutorials_io_60_ctf_bst_auditory.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Working with CTF data: the Brainstorm auditory dataset</div>
    </div>


.. raw:: html

    </div>

Working with continuous data
----------------------------

These tutorials cover the basics of loading EEG/MEG data into MNE-Python, and
how to query, manipulate, annotate, plot, and export continuous data in the
:class:`~mne.io.Raw` format.



.. raw:: html

    <div class="sphx-glr-thumbnails">


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="This tutorial covers the basics of working with raw EEG/MEG data in Python. It introduces the R...">

.. only:: html

  .. image:: /auto_tutorials/raw/images/thumb/sphx_glr_10_raw_overview_thumb.png
    :alt: The Raw data structure: continuous data

  :ref:`sphx_glr_auto_tutorials_raw_10_raw_overview.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">The Raw data structure: continuous data</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="This tutorial describes event representation and how event arrays are used to subselect data.">

.. only:: html

  .. image:: /auto_tutorials/raw/images/thumb/sphx_glr_20_event_arrays_thumb.png
    :alt: Working with events

  :ref:`sphx_glr_auto_tutorials_raw_20_event_arrays.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Working with events</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="This tutorial describes adding annotations to a ~mne.io.Raw object, and how annotations are use...">

.. only:: html

  .. image:: /auto_tutorials/raw/images/thumb/sphx_glr_30_annotate_raw_thumb.png
    :alt: Annotating continuous data

  :ref:`sphx_glr_auto_tutorials_raw_30_annotate_raw.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Annotating continuous data</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="This tutorial shows how to plot continuous data as a time series, how to plot the spectral dens...">

.. only:: html

  .. image:: /auto_tutorials/raw/images/thumb/sphx_glr_40_visualize_raw_thumb.png
    :alt: Built-in plotting methods for Raw objects

  :ref:`sphx_glr_auto_tutorials_raw_40_visualize_raw.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Built-in plotting methods for Raw objects</div>
    </div>


.. raw:: html

    </div>

Preprocessing
-------------

These tutorials cover various preprocessing techniques for continuous
data, as well as some diagnostic plotting methods.



.. raw:: html

    <div class="sphx-glr-thumbnails">


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="This tutorial covers the basics of artifact detection, and introduces the artifact detection to...">

.. only:: html

  .. image:: /auto_tutorials/preprocessing/images/thumb/sphx_glr_10_preprocessing_overview_thumb.png
    :alt: Overview of artifact detection

  :ref:`sphx_glr_auto_tutorials_preprocessing_10_preprocessing_overview.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Overview of artifact detection</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="This tutorial covers manual marking of bad channels and reconstructing bad channels based on go...">

.. only:: html

  .. image:: /auto_tutorials/preprocessing/images/thumb/sphx_glr_15_handling_bad_channels_thumb.png
    :alt: Handling bad channels

  :ref:`sphx_glr_auto_tutorials_preprocessing_15_handling_bad_channels.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Handling bad channels</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="This tutorial covers:">

.. only:: html

  .. image:: /auto_tutorials/preprocessing/images/thumb/sphx_glr_20_rejecting_bad_data_thumb.png
    :alt: Rejecting bad data spans and breaks

  :ref:`sphx_glr_auto_tutorials_preprocessing_20_rejecting_bad_data.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Rejecting bad data spans and breaks</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="Here we give some background information on filtering in general, and how it is done in MNE-Pyt...">

.. only:: html

  .. image:: /auto_tutorials/preprocessing/images/thumb/sphx_glr_25_background_filtering_thumb.png
    :alt: Background information on filtering

  :ref:`sphx_glr_auto_tutorials_preprocessing_25_background_filtering.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Background information on filtering</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="This tutorial covers filtering and resampling, and gives examples of how filtering can be used ...">

.. only:: html

  .. image:: /auto_tutorials/preprocessing/images/thumb/sphx_glr_30_filtering_resampling_thumb.png
    :alt: Filtering and resampling data

  :ref:`sphx_glr_auto_tutorials_preprocessing_30_filtering_resampling.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Filtering and resampling data</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="This tutorial covers removal of artifacts using regression as in Gratton et al. (1983) GrattonE...">

.. only:: html

  .. image:: /auto_tutorials/preprocessing/images/thumb/sphx_glr_35_artifact_correction_regression_thumb.png
    :alt: Repairing artifacts with regression

  :ref:`sphx_glr_auto_tutorials_preprocessing_35_artifact_correction_regression.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Repairing artifacts with regression</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="This tutorial covers the basics of independent components analysis (ICA) and shows how ICA can ...">

.. only:: html

  .. image:: /auto_tutorials/preprocessing/images/thumb/sphx_glr_40_artifact_correction_ica_thumb.png
    :alt: Repairing artifacts with ICA

  :ref:`sphx_glr_auto_tutorials_preprocessing_40_artifact_correction_ica.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Repairing artifacts with ICA</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="This tutorial provides background information on projectors and Signal Space Projection (SSP), ...">

.. only:: html

  .. image:: /auto_tutorials/preprocessing/images/thumb/sphx_glr_45_projectors_background_thumb.png
    :alt: Background on projectors and projections

  :ref:`sphx_glr_auto_tutorials_preprocessing_45_projectors_background.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Background on projectors and projections</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="This tutorial covers the basics of signal-space projection (SSP) and shows how SSP can be used ...">

.. only:: html

  .. image:: /auto_tutorials/preprocessing/images/thumb/sphx_glr_50_artifact_correction_ssp_thumb.png
    :alt: Repairing artifacts with SSP

  :ref:`sphx_glr_auto_tutorials_preprocessing_50_artifact_correction_ssp.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Repairing artifacts with SSP</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="This tutorial describes how to set or change the EEG reference in MNE-Python.">

.. only:: html

  .. image:: /auto_tutorials/preprocessing/images/thumb/sphx_glr_55_setting_eeg_reference_thumb.png
    :alt: Setting the EEG reference

  :ref:`sphx_glr_auto_tutorials_preprocessing_55_setting_eeg_reference.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Setting the EEG reference</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="Continuous head movement can be encoded during MEG recordings by use of HPI coils that continuo...">

.. only:: html

  .. image:: /auto_tutorials/preprocessing/images/thumb/sphx_glr_59_head_positions_thumb.png
    :alt: Extracting and visualizing subject head movement

  :ref:`sphx_glr_auto_tutorials_preprocessing_59_head_positions.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Extracting and visualizing subject head movement</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="This tutorial covers reducing environmental noise and compensating for head movement with SSS a...">

.. only:: html

  .. image:: /auto_tutorials/preprocessing/images/thumb/sphx_glr_60_maxwell_filtering_sss_thumb.png
    :alt: Signal-space separation (SSS) and Maxwell filtering

  :ref:`sphx_glr_auto_tutorials_preprocessing_60_maxwell_filtering_sss.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Signal-space separation (SSS) and Maxwell filtering</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="This tutorial covers how to convert functional near-infrared spectroscopy (fNIRS) data from raw...">

.. only:: html

  .. image:: /auto_tutorials/preprocessing/images/thumb/sphx_glr_70_fnirs_processing_thumb.png
    :alt: Preprocessing functional near-infrared spectroscopy (fNIRS) data

  :ref:`sphx_glr_auto_tutorials_preprocessing_70_fnirs_processing.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Preprocessing functional near-infrared spectroscopy (fNIRS) data</div>
    </div>


.. raw:: html

    </div>

Segmenting continuous data into epochs
--------------------------------------

These tutorials cover epoched data, and how it differs from working with
continuous data.



.. raw:: html

    <div class="sphx-glr-thumbnails">


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="This tutorial covers the basics of creating and working with epochs data. It introduces the Epo...">

.. only:: html

  .. image:: /auto_tutorials/epochs/images/thumb/sphx_glr_10_epochs_overview_thumb.png
    :alt: The Epochs data structure: discontinuous data

  :ref:`sphx_glr_auto_tutorials_epochs_10_epochs_overview.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">The Epochs data structure: discontinuous data</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="This tutorial shows how to plot epoched data as time series, how to plot the spectral density o...">

.. only:: html

  .. image:: /auto_tutorials/epochs/images/thumb/sphx_glr_20_visualize_epochs_thumb.png
    :alt: Visualizing epoched data

  :ref:`sphx_glr_auto_tutorials_epochs_20_visualize_epochs.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Visualizing epoched data</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="This tutorial shows how to add metadata to ~mne.Epochs objects, and how to use Pandas query str...">

.. only:: html

  .. image:: /auto_tutorials/epochs/images/thumb/sphx_glr_30_epochs_metadata_thumb.png
    :alt: Working with Epoch metadata

  :ref:`sphx_glr_auto_tutorials_epochs_30_epochs_metadata.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Working with Epoch metadata</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="This tutorial shows how to auto-generate metadata for ~mne.Epochs, based on events via mne.epoc...">

.. only:: html

  .. image:: /auto_tutorials/epochs/images/thumb/sphx_glr_40_autogenerate_metadata_thumb.png
    :alt: Auto-generating Epochs metadata

  :ref:`sphx_glr_auto_tutorials_epochs_40_autogenerate_metadata.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Auto-generating Epochs metadata</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="This tutorial shows how to export the data in Epochs objects to a Pandas DataFrame &lt;pandas.Data...">

.. only:: html

  .. image:: /auto_tutorials/epochs/images/thumb/sphx_glr_50_epochs_to_data_frame_thumb.png
    :alt: Exporting Epochs to Pandas DataFrames

  :ref:`sphx_glr_auto_tutorials_epochs_50_epochs_to_data_frame.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Exporting Epochs to Pandas DataFrames</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="This tutorial shows how to segment continuous data into a set of epochs spaced equidistantly in...">

.. only:: html

  .. image:: /auto_tutorials/epochs/images/thumb/sphx_glr_60_make_fixed_length_epochs_thumb.png
    :alt: Divide continuous data into equally-spaced epochs

  :ref:`sphx_glr_auto_tutorials_epochs_60_make_fixed_length_epochs.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Divide continuous data into equally-spaced epochs</div>
    </div>


.. raw:: html

    </div>

Estimating evoked responses
---------------------------

These tutorials cover estimates of evoked responses (i.e., averages across
several repetitions of an experimental condition).



.. raw:: html

    <div class="sphx-glr-thumbnails">


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="This tutorial covers the basics of creating and working with evoked data. It introduces the Evo...">

.. only:: html

  .. image:: /auto_tutorials/evoked/images/thumb/sphx_glr_10_evoked_overview_thumb.png
    :alt: The Evoked data structure: evoked/averaged data

  :ref:`sphx_glr_auto_tutorials_evoked_10_evoked_overview.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">The Evoked data structure: evoked/averaged data</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="This tutorial shows the different visualization methods for ~mne.Evoked objects.">

.. only:: html

  .. image:: /auto_tutorials/evoked/images/thumb/sphx_glr_20_visualize_evoked_thumb.png
    :alt: Visualizing Evoked data

  :ref:`sphx_glr_auto_tutorials_evoked_20_visualize_evoked.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Visualizing Evoked data</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="This tutorial shows how to perform standard ERP analyses in MNE-Python. Most of the material he...">

.. only:: html

  .. image:: /auto_tutorials/evoked/images/thumb/sphx_glr_30_eeg_erp_thumb.png
    :alt: EEG processing and Event Related Potentials (ERPs)

  :ref:`sphx_glr_auto_tutorials_evoked_30_eeg_erp.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">EEG processing and Event Related Potentials (ERPs)</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="This tutorial demonstrates how to plot whitening evoked data.">

.. only:: html

  .. image:: /auto_tutorials/evoked/images/thumb/sphx_glr_40_whitened_thumb.png
    :alt: Plotting whitened data

  :ref:`sphx_glr_auto_tutorials_evoked_40_whitened.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Plotting whitened data</div>
    </div>


.. raw:: html

    </div>

Time-frequency analysis
-----------------------

These tutorials cover frequency and time-frequency analysis of neural
signals.



.. raw:: html

    <div class="sphx-glr-thumbnails">


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="The objective is to show you how to explore the spectral content of your data (frequency and ti...">

.. only:: html

  .. image:: /auto_tutorials/time-freq/images/thumb/sphx_glr_20_sensors_time_frequency_thumb.png
    :alt: Frequency and time-frequency sensor analysis

  :ref:`sphx_glr_auto_tutorials_time-freq_20_sensors_time_frequency.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Frequency and time-frequency sensor analysis</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="In this tutorial we compute the frequency spectrum and quantify signal-to-noise ratio (SNR) at ...">

.. only:: html

  .. image:: /auto_tutorials/time-freq/images/thumb/sphx_glr_50_ssvep_thumb.png
    :alt: Frequency-tagging: Basic analysis of an SSVEP/vSSR dataset

  :ref:`sphx_glr_auto_tutorials_time-freq_50_ssvep.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Frequency-tagging: Basic analysis of an SSVEP/vSSR dataset</div>
    </div>


.. raw:: html

    </div>

Forward models and source spaces
--------------------------------

These tutorials cover how the cortical source locations (source spaces) and
forward models (AKA leadfield matrices) are defined.



.. raw:: html

    <div class="sphx-glr-thumbnails">


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="This tutorial covers how to use FreeSurfer alongside MNE-Python, to handle the structural MRI d...">

.. only:: html

  .. image:: /auto_tutorials/forward/images/thumb/sphx_glr_10_background_freesurfer_thumb.png
    :alt: FreeSurfer MRI reconstruction

  :ref:`sphx_glr_auto_tutorials_forward_10_background_freesurfer.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">FreeSurfer MRI reconstruction</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="This tutorial shows how to visually assess the spatial alignment of MEG sensor locations, digit...">

.. only:: html

  .. image:: /auto_tutorials/forward/images/thumb/sphx_glr_20_source_alignment_thumb.png
    :alt: Source alignment and coordinate frames

  :ref:`sphx_glr_auto_tutorials_forward_20_source_alignment.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Source alignment and coordinate frames</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="This example shows how to use the coregistration functions to perform an automated MEG-MRI core...">

.. only:: html

  .. image:: /auto_tutorials/forward/images/thumb/sphx_glr_25_automated_coreg_thumb.png
    :alt: Using an automated approach to coregistration

  :ref:`sphx_glr_auto_tutorials_forward_25_automated_coreg.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Using an automated approach to coregistration</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="The aim of this tutorial is to be a getting started for forward computation.">

.. only:: html

  .. image:: /auto_tutorials/forward/images/thumb/sphx_glr_30_forward_thumb.png
    :alt: Head model and forward computation

  :ref:`sphx_glr_auto_tutorials_forward_30_forward.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Head model and forward computation</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="This tutorial explains how to compute the forward operator from EEG data using the standard tem...">

.. only:: html

  .. image:: /auto_tutorials/forward/images/thumb/sphx_glr_35_eeg_no_mri_thumb.png
    :alt: EEG forward operator with a template MRI

  :ref:`sphx_glr_auto_tutorials_forward_35_eeg_no_mri.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">EEG forward operator with a template MRI</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="This tutorial explains how MRI coordinate frames are handled in MNE-Python, and how MNE-Python ...">

.. only:: html

  .. image:: /auto_tutorials/forward/images/thumb/sphx_glr_50_background_freesurfer_mne_thumb.png
    :alt: How MNE uses FreeSurfer's outputs

  :ref:`sphx_glr_auto_tutorials_forward_50_background_freesurfer_mne.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">How MNE uses FreeSurfer's outputs</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="Sometimes when creating a BEM model the surfaces need manual correction because of a series of ...">

.. only:: html

  .. image:: /auto_tutorials/forward/images/thumb/sphx_glr_80_fix_bem_in_blender_thumb.jpg
    :alt: Editing BEM surfaces in Blender

  :ref:`sphx_glr_auto_tutorials_forward_80_fix_bem_in_blender.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Editing BEM surfaces in Blender</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="Many methods in MNE, including source estimation and some classification algorithms, require co...">

.. only:: html

  .. image:: /auto_tutorials/forward/images/thumb/sphx_glr_90_compute_covariance_thumb.png
    :alt: Computing a covariance matrix

  :ref:`sphx_glr_auto_tutorials_forward_90_compute_covariance.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Computing a covariance matrix</div>
    </div>


.. raw:: html

    </div>

Source localization and inverses
--------------------------------

These tutorials cover estimation of cortical activity from sensor recordings.



.. raw:: html

    <div class="sphx-glr-thumbnails">


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="Source estimates, commonly referred to as STC (Source Time Courses) &lt;stc&gt;, are obtained from so...">

.. only:: html

  .. image:: /auto_tutorials/inverse/images/thumb/sphx_glr_10_stc_class_thumb.png
    :alt: The SourceEstimate data structure

  :ref:`sphx_glr_auto_tutorials_inverse_10_stc_class.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">The SourceEstimate data structure</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="This shows how to fit a dipole Sarvas1987 using mne-python.">

.. only:: html

  .. image:: /auto_tutorials/inverse/images/thumb/sphx_glr_20_dipole_fit_thumb.png
    :alt: Source localization with equivalent current dipole (ECD) fit

  :ref:`sphx_glr_auto_tutorials_inverse_20_dipole_fit.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Source localization with equivalent current dipole (ECD) fit</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="The aim of this tutorial is to teach you how to compute and apply a linear minimum-norm inverse...">

.. only:: html

  .. image:: /auto_tutorials/inverse/images/thumb/sphx_glr_30_mne_dspm_loreta_thumb.png
    :alt: Source localization with MNE, dSPM, sLORETA, and eLORETA

  :ref:`sphx_glr_auto_tutorials_inverse_30_mne_dspm_loreta.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Source localization with MNE, dSPM, sLORETA, and eLORETA</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="When performing source localization in a distributed manner (i.e., using MNE/dSPM/sLORETA/eLORE...">

.. only:: html

  .. image:: /auto_tutorials/inverse/images/thumb/sphx_glr_35_dipole_orientations_thumb.png
    :alt: The role of dipole orientations in distributed source localization

  :ref:`sphx_glr_auto_tutorials_inverse_35_dipole_orientations.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">The role of dipole orientations in distributed source localization</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="This example shows example fixed- and free-orientation source localizations produced by the min...">

.. only:: html

  .. image:: /auto_tutorials/inverse/images/thumb/sphx_glr_40_mne_fixed_free_thumb.png
    :alt: Computing various MNE solutions

  :ref:`sphx_glr_auto_tutorials_inverse_40_mne_fixed_free.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Computing various MNE solutions</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="This tutorial gives an overview of the beamformer method and shows how to reconstruct source ac...">

.. only:: html

  .. image:: /auto_tutorials/inverse/images/thumb/sphx_glr_50_beamformer_lcmv_thumb.png
    :alt: Source reconstruction using an LCMV beamformer

  :ref:`sphx_glr_auto_tutorials_inverse_50_beamformer_lcmv.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Source reconstruction using an LCMV beamformer</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="This tutorial focuses on visualization of source estimates &lt;STC&gt;.">

.. only:: html

  .. image:: /auto_tutorials/inverse/images/thumb/sphx_glr_60_visualize_stc_thumb.png
    :alt: Visualize source time courses (stcs)

  :ref:`sphx_glr_auto_tutorials_inverse_60_visualize_stc.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Visualize source time courses (stcs)</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="This tutorial explains how to compute the forward operator from EEG data when the electrodes ar...">

.. only:: html

  .. image:: /auto_tutorials/inverse/images/thumb/sphx_glr_70_eeg_mri_coords_thumb.png
    :alt: EEG source localization given electrode locations on an MRI

  :ref:`sphx_glr_auto_tutorials_inverse_70_eeg_mri_coords.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">EEG source localization given electrode locations on an MRI</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="Here we compute the evoked from raw for the Brainstorm Elekta phantom tutorial dataset. For com...">

.. only:: html

  .. image:: /auto_tutorials/inverse/images/thumb/sphx_glr_80_brainstorm_phantom_elekta_thumb.png
    :alt: Brainstorm Elekta phantom dataset tutorial

  :ref:`sphx_glr_auto_tutorials_inverse_80_brainstorm_phantom_elekta.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Brainstorm Elekta phantom dataset tutorial</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="Here we compute the evoked from raw for the Brainstorm CTF phantom tutorial dataset. For compar...">

.. only:: html

  .. image:: /auto_tutorials/inverse/images/thumb/sphx_glr_85_brainstorm_phantom_ctf_thumb.png
    :alt: Brainstorm CTF phantom dataset tutorial

  :ref:`sphx_glr_auto_tutorials_inverse_85_brainstorm_phantom_ctf.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Brainstorm CTF phantom dataset tutorial</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="Here we read 4DBTi epochs data obtained with a spherical phantom using four different dipole lo...">

.. only:: html

  .. image:: /auto_tutorials/inverse/images/thumb/sphx_glr_90_phantom_4DBTi_thumb.png
    :alt: 4D Neuroimaging/BTi phantom dataset tutorial

  :ref:`sphx_glr_auto_tutorials_inverse_90_phantom_4DBTi.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">4D Neuroimaging/BTi phantom dataset tutorial</div>
    </div>


.. raw:: html

    </div>

Statistical analysis of sensor data
-----------------------------------

These tutorials describe some approaches to statistical analysis of
sensor-level data.



.. raw:: html

    <div class="sphx-glr-thumbnails">


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="Here we will briefly cover multiple concepts of inferential statistics in an introductory manne...">

.. only:: html

  .. image:: /auto_tutorials/stats-sensor-space/images/thumb/sphx_glr_10_background_stats_thumb.png
    :alt: Statistical inference

  :ref:`sphx_glr_auto_tutorials_stats-sensor-space_10_background_stats.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Statistical inference</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="MNE-Python provides a range of tools for statistical hypothesis testing and the visualisation o...">

.. only:: html

  .. image:: /auto_tutorials/stats-sensor-space/images/thumb/sphx_glr_20_erp_stats_thumb.png
    :alt: Visualising statistical significance thresholds on EEG data

  :ref:`sphx_glr_auto_tutorials_stats-sensor-space_20_erp_stats.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Visualising statistical significance thresholds on EEG data</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="This script shows how to estimate significant clusters in time-frequency power estimates. It us...">

.. only:: html

  .. image:: /auto_tutorials/stats-sensor-space/images/thumb/sphx_glr_40_cluster_1samp_time_freq_thumb.png
    :alt: Non-parametric 1 sample cluster statistic on single trial power

  :ref:`sphx_glr_auto_tutorials_stats-sensor-space_40_cluster_1samp_time_freq.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Non-parametric 1 sample cluster statistic on single trial power</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="This script shows how to compare clusters in time-frequency power estimates between conditions....">

.. only:: html

  .. image:: /auto_tutorials/stats-sensor-space/images/thumb/sphx_glr_50_cluster_between_time_freq_thumb.png
    :alt: Non-parametric between conditions cluster statistic on single trial power

  :ref:`sphx_glr_auto_tutorials_stats-sensor-space_50_cluster_between_time_freq.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Non-parametric between conditions cluster statistic on single trial power</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="Tests for differential evoked responses in at least one condition using a permutation clusterin...">

.. only:: html

  .. image:: /auto_tutorials/stats-sensor-space/images/thumb/sphx_glr_75_cluster_ftest_spatiotemporal_thumb.png
    :alt: Spatiotemporal permutation F-test on full sensor data

  :ref:`sphx_glr_auto_tutorials_stats-sensor-space_75_cluster_ftest_spatiotemporal.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Spatiotemporal permutation F-test on full sensor data</div>
    </div>


.. raw:: html

    </div>

Statistical analysis of source estimates
----------------------------------------

These tutorials cover within-subject statistical analysis of source
estimates.



.. raw:: html

    <div class="sphx-glr-thumbnails">


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="This example tests if the evoked response is significantly different between two conditions acr...">

.. only:: html

  .. image:: /auto_tutorials/stats-source-space/images/thumb/sphx_glr_20_cluster_1samp_spatiotemporal_thumb.png
    :alt: Permutation t-test on source data with spatio-temporal clustering

  :ref:`sphx_glr_auto_tutorials_stats-source-space_20_cluster_1samp_spatiotemporal.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Permutation t-test on source data with spatio-temporal clustering</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="Tests if the source space data are significantly different between 2 groups of subjects (simula...">

.. only:: html

  .. image:: /auto_tutorials/stats-source-space/images/thumb/sphx_glr_30_cluster_ftest_spatiotemporal_thumb.png
    :alt: 2 samples permutation test on source data with spatio-temporal clustering

  :ref:`sphx_glr_auto_tutorials_stats-source-space_30_cluster_ftest_spatiotemporal.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">2 samples permutation test on source data with spatio-temporal clustering</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="This example illustrates how to make use of the clustering functions for arbitrary, self-define...">

.. only:: html

  .. image:: /auto_tutorials/stats-source-space/images/thumb/sphx_glr_60_cluster_rmANOVA_spatiotemporal_thumb.png
    :alt: Repeated measures ANOVA on source data with spatio-temporal clustering

  :ref:`sphx_glr_auto_tutorials_stats-source-space_60_cluster_rmANOVA_spatiotemporal.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Repeated measures ANOVA on source data with spatio-temporal clustering</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="This script shows how to conduct a mass-univariate repeated measures ANOVA. As the model to be ...">

.. only:: html

  .. image:: /auto_tutorials/stats-source-space/images/thumb/sphx_glr_70_cluster_rmANOVA_time_freq_thumb.png
    :alt: Mass-univariate twoway repeated measures ANOVA on single trial power

  :ref:`sphx_glr_auto_tutorials_stats-source-space_70_cluster_rmANOVA_time_freq.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Mass-univariate twoway repeated measures ANOVA on single trial power</div>
    </div>


.. raw:: html

    </div>

Machine learning models of neural activity
------------------------------------------

These tutorials cover some of the machine learning methods available in
MNE-Python.



.. raw:: html

    <div class="sphx-glr-thumbnails">


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="This demonstrates how an encoding model can be fit with multiple continuous inputs. In this cas...">

.. only:: html

  .. image:: /auto_tutorials/machine-learning/images/thumb/sphx_glr_30_strf_thumb.png
    :alt: Spectro-temporal receptive field (STRF) estimation on continuous data

  :ref:`sphx_glr_auto_tutorials_machine-learning_30_strf.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Spectro-temporal receptive field (STRF) estimation on continuous data</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="Design philosophy ================= Decoding (a.k.a. MVPA) in MNE largely follows the machine l...">

.. only:: html

  .. image:: /auto_tutorials/machine-learning/images/thumb/sphx_glr_50_decoding_thumb.png
    :alt: Decoding (MVPA)

  :ref:`sphx_glr_auto_tutorials_machine-learning_50_decoding.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Decoding (MVPA)</div>
    </div>


.. raw:: html

    </div>

Clinical applications
---------------------

These tutorials illustrate clinical uses of MNE-Python.



.. raw:: html

    <div class="sphx-glr-thumbnails">


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="Analysis of intracranial electrophysiology recordings typically involves finding the position o...">

.. only:: html

  .. image:: /auto_tutorials/clinical/images/thumb/sphx_glr_10_ieeg_localize_thumb.png
    :alt: Locating intracranial electrode contacts

  :ref:`sphx_glr_auto_tutorials_clinical_10_ieeg_localize.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Locating intracranial electrode contacts</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="MNE-Python supports working with more than just MEG and EEG data. Here we show some of the func...">

.. only:: html

  .. image:: /auto_tutorials/clinical/images/thumb/sphx_glr_20_seeg_thumb.png
    :alt: Working with sEEG data

  :ref:`sphx_glr_auto_tutorials_clinical_20_seeg.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Working with sEEG data</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="MNE supports working with more than just MEG and EEG data. Here we show some of the functions t...">

.. only:: html

  .. image:: /auto_tutorials/clinical/images/thumb/sphx_glr_30_ecog_thumb.png
    :alt: Working with ECoG data

  :ref:`sphx_glr_auto_tutorials_clinical_30_ecog.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Working with ECoG data</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="This tutorial explains how to perform a toy polysomnography analysis that answers the following...">

.. only:: html

  .. image:: /auto_tutorials/clinical/images/thumb/sphx_glr_60_sleep_thumb.png
    :alt: Sleep stage classification from polysomnography (PSG) data

  :ref:`sphx_glr_auto_tutorials_clinical_60_sleep.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Sleep stage classification from polysomnography (PSG) data</div>
    </div>


.. raw:: html

    </div>

Simulation
----------

These tutorials describe how to populate MNE-Python data structures with
arbitrary data, using the array-based constructors and the simulation
submodule.



.. raw:: html

    <div class="sphx-glr-thumbnails">


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="This tutorial shows how to create MNE-Python&#x27;s core data structures using an existing NumPy arr...">

.. only:: html

  .. image:: /auto_tutorials/simulation/images/thumb/sphx_glr_10_array_objs_thumb.png
    :alt: Creating MNE-Python data structures from scratch

  :ref:`sphx_glr_auto_tutorials_simulation_10_array_objs.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Creating MNE-Python data structures from scratch</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="The aim of this tutorial is to demonstrate how to put a known signal at a desired location(s) i...">

.. only:: html

  .. image:: /auto_tutorials/simulation/images/thumb/sphx_glr_70_point_spread_thumb.png
    :alt: Corrupt known signal with point spread

  :ref:`sphx_glr_auto_tutorials_simulation_70_point_spread.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Corrupt known signal with point spread</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="In this tutorial, we&#x27;ll simulate two signals originating from two locations on the cortex. Thes...">

.. only:: html

  .. image:: /auto_tutorials/simulation/images/thumb/sphx_glr_80_dics_thumb.png
    :alt: DICS for power mapping

  :ref:`sphx_glr_auto_tutorials_simulation_80_dics.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">DICS for power mapping</div>
    </div>


.. raw:: html

    </div>


.. toctree::
   :hidden:
   :includehidden:

   /auto_tutorials/intro/index.rst
   /auto_tutorials/io/index.rst
   /auto_tutorials/raw/index.rst
   /auto_tutorials/preprocessing/index.rst
   /auto_tutorials/epochs/index.rst
   /auto_tutorials/evoked/index.rst
   /auto_tutorials/time-freq/index.rst
   /auto_tutorials/forward/index.rst
   /auto_tutorials/inverse/index.rst
   /auto_tutorials/stats-sensor-space/index.rst
   /auto_tutorials/stats-source-space/index.rst
   /auto_tutorials/machine-learning/index.rst
   /auto_tutorials/clinical/index.rst
   /auto_tutorials/simulation/index.rst



.. only :: html

 .. container:: sphx-glr-footer
    :class: sphx-glr-footer-gallery


  .. container:: sphx-glr-download sphx-glr-download-python

    :download:`Download all examples in Python source code: auto_tutorials_python.zip </auto_tutorials/auto_tutorials_python.zip>`



  .. container:: sphx-glr-download sphx-glr-download-jupyter

    :download:`Download all examples in Jupyter notebooks: auto_tutorials_jupyter.zip </auto_tutorials/auto_tutorials_jupyter.zip>`


.. only:: html

 .. rst-class:: sphx-glr-signature

    `Gallery generated by Sphinx-Gallery <https://sphinx-gallery.github.io>`_

Built from tutorials/io/README.rst:

auto_tutorials/io/index.rst


.. _sphx_glr_auto_tutorials_io:

.. _tut-data-formats:

Reading data for different recording systems
--------------------------------------------

These tutorials cover the basics of loading EEG/MEG data into MNE-Python
for various recording devices.



.. raw:: html

    <div class="sphx-glr-thumbnails">


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="This section describes how to read data for various MEG manufacturers.">

.. only:: html

  .. image:: /auto_tutorials/io/images/thumb/sphx_glr_10_reading_meg_data_thumb.png
    :alt: Importing data from MEG devices

  :ref:`sphx_glr_auto_tutorials_io_10_reading_meg_data.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Importing data from MEG devices</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="MNE includes various functions and utilities for reading EEG data and electrode locations.">

.. only:: html

  .. image:: /auto_tutorials/io/images/thumb/sphx_glr_20_reading_eeg_data_thumb.png
    :alt: Importing data from EEG devices

  :ref:`sphx_glr_auto_tutorials_io_20_reading_eeg_data.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Importing data from EEG devices</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="fNIRS devices consist of two kinds of optodes: light sources (AKA &quot;emitters&quot; or &quot;transmitters&quot;)...">

.. only:: html

  .. image:: /auto_tutorials/io/images/thumb/sphx_glr_30_reading_fnirs_data_thumb.png
    :alt: Importing data from fNIRS devices

  :ref:`sphx_glr_auto_tutorials_io_30_reading_fnirs_data.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Importing data from fNIRS devices</div>
    </div>


.. raw:: html

    <div class="sphx-glr-thumbcontainer" tooltip="Here we compute the evoked from raw for the auditory Brainstorm tutorial dataset. For compariso...">

.. only:: html

  .. image:: /auto_tutorials/io/images/thumb/sphx_glr_60_ctf_bst_auditory_thumb.png
    :alt: Working with CTF data: the Brainstorm auditory dataset

  :ref:`sphx_glr_auto_tutorials_io_60_ctf_bst_auditory.py`

.. raw:: html

      <div class="sphx-glr-thumbnail-title">Working with CTF data: the Brainstorm auditory dataset</div>
    </div>


.. raw:: html

    </div>


.. toctree::
   :hidden:

   /auto_tutorials/io/10_reading_meg_data
   /auto_tutorials/io/20_reading_eeg_data
   /auto_tutorials/io/30_reading_fnirs_data
   /auto_tutorials/io/60_ctf_bst_auditory


@alexisthual
Copy link
Copy Markdown
Contributor Author

Oops, I pushed a little bit too quickly, it should be better now and solve the warnings you mentioned previously.
(Besides, I integrated both of your suggested changes, and still can't figure out what's wrong with the tests on windows haha)

@larsoner
Copy link
Copy Markdown
Contributor

larsoner commented Mar 8, 2022

Ready for review/merge from your end @alexisthual ?

@larsoner
Copy link
Copy Markdown
Contributor

larsoner commented Mar 8, 2022

MNE-Python docs:

Screenshot from 2022-03-08 11-46-56

Look so much better on this PR:

Screenshot from 2022-03-08 11-47-27

I'm going to assume this is good to go despite the WIP title. Thanks for this great enhancement @alexisthual !

@larsoner larsoner merged commit d8ad979 into sphinx-gallery:master Mar 8, 2022
@larsoner
Copy link
Copy Markdown
Contributor

larsoner commented Mar 8, 2022

@alexisthual any idea why this is happening on a Sphinx windows build?

D:\a\1\s\doc\auto_examples\index.rst:2019: WARNING: toctree contains reference to nonexisting document 'auto_examples\\io\\index'

target_dir = os.path.join(gallery_dir_abs_path, subsection)
subsection_index_files.append(
os.path.join(
'/', gallery_dir, subsection, 'index.rst'
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this have been os.path.sep rather than / (to make things work properly on Windows)? Or something else?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You think you are right!

Copy link
Copy Markdown
Contributor

@larsoner larsoner Mar 8, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Either that or '/'.join(..) would make sense. Do you have a Windows machine or VM where you could test and try to replicate the failure? If not, I can do it

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry I don't have a Windows machine!
I had a quick look at the rest of the PR to see if other os.path.sep were missing, but I guess this would pop out on the build you are currently trying to run.

@alexisthual
Copy link
Copy Markdown
Contributor Author

Yaay, I am very happy this was merged!

I expect some backfire after this is released, but it's hard to assess how many projects will want to stick to a flat structure instead of a nested one.
I would advocate that a nested structure is better (it is easier to flatten than to nest), but if pressure grows in the issues to keep the flat structure, I am willing to add a config parameter to choose between the two behaviours 😊

@GaelVaroquaux
Copy link
Copy Markdown
Contributor

GaelVaroquaux commented Mar 8, 2022

This change should be documented in the changelog in a didactic way (I must admit that with a quick look, I myself do not fully understand what it does).

Also, I hope that it is not a too drastic change. Elsewhere, as an upstream of many major project, it is irresponsible. We cannot decide by ourselves a major change that will affect all the downstream projects (so pages seen by millions of users, and curated by hundreds of developers), without leaving an option to roll back. It would be terrible behavior from an ecosystem point of view.

@GaelVaroquaux
Copy link
Copy Markdown
Contributor

Ha, I understand: this only affect the sidebar, right?

If so, it's not a major breaking change, and my comment above does not hold.

@larsoner
Copy link
Copy Markdown
Contributor

larsoner commented Mar 8, 2022

Yes, that's my understanding! @alexisthual do you want to try crafting a changelog entry for this? I agree it would be useful

@alexisthual
Copy link
Copy Markdown
Contributor Author

Ha, I understand: this only affect the sidebar, right?

That is the goal 🙂 In short, it modifies generated toctrees in a way that mimics the real structure of sphinx galleries.

If so, it's not a major breaking change, and my comment above does not hold.

Eventually, it only adds some new .rst files and intermediate toctrees, which I think are fair, non-breaking changes.

Maybe some projects built their own custom sidebar by tweaking the generated toctree (precisely to display subsections), which I think is the only case that would require some refactoring.
Although I don't know of any such project, they would most likely only need to let sphinx-gallery handle the toctree.

However, big documentations will benefit from this PR, as it will enable them to organise their sidebar items in categories (which is greatly needed for matplotlib in my opinion, which features 20 categories, most of which have at least 10 examples).

Screenshot from 2022-03-08 21-58-51

@alexisthual
Copy link
Copy Markdown
Contributor Author

Yes, that's my understanding! @alexisthual do you want to try crafting a changelog entry for this? I agree it would be useful

Yes, should I open a new PR to commit the changelog @larsoner ?

@larsoner
Copy link
Copy Markdown
Contributor

larsoner commented Mar 8, 2022

Yep, that would be great

@GaelVaroquaux
Copy link
Copy Markdown
Contributor

GaelVaroquaux commented Mar 8, 2022 via email

@alexisthual alexisthual changed the title [WIP] Handle nested structures [MRG] Handle nested structures Mar 9, 2022
@tacaswell
Copy link
Copy Markdown
Contributor

🎉 I'm super excited this is in!

@tacaswell
Copy link
Copy Markdown
Contributor

It builds out of the box and looks alot better!

image

@alexisthual
Copy link
Copy Markdown
Contributor Author

It builds out of the box and looks alot better!

Haha, super happy to hear that, that's a relief 😅

@NicolasGensollen
Copy link
Copy Markdown

This is a great achievement! 🎉
Thanks a lot @alexisthual ! 👍

clrpackages referenced this pull request in clearlinux-pkgs/pypi-sphinx_gallery Jul 27, 2022
… to version 0.11.0

v0.11.0
-------

In this version, the "Out:" prefix applied to code outputs is now created from
CSS pseudo-elements instead of additional real text. For more details, see
`#896 <https://github.com/sphinx-gallery/sphinx-gallery/pull/896>`__.

**Implemented enhancements:**

Nesting gallery sections (i.e. gallery subfolders) was implemented in`#904 <https://github.com/sphinx-gallery/sphinx-gallery/pull/904>`__. This feature can be disabled (see config option ``nested_sections`` in the documentation) if the previous behaviour is prefered (`alexisthual <https://github.com/alexisthual>`__)

Tooltips now overlay gallery items `commit 36166cd <https://github.com/sphinx-gallery/sphinx-gallery/pull/944/commits/36166cd2fc2b43ecbd585654cfe8745f3a1b3f64>`__. Custom CSS might need to be adapted (`alexisthual <https://github.com/alexisthual>`__).

-  Problem in section and example title level in subgalleries `#935 <https://github.com/sphinx-gallery/sphinx-gallery/issues/935>`__
-  Add ability to write nested ``index.rst`` `#855 <https://github.com/sphinx-gallery/sphinx-gallery/issues/855>`__

(NEWS truncated at 15 lines)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add ability to write nested index.rst

6 participants