Skip to content

The current design of mne.bem.make_watershed_bem is incompatible with ipython #7080

@christian-oreilly

Description

@christian-oreilly

Describe the bug

Runinng mne.bem.make_watershed_bem(fs_subject) within ipython (actually, this was tried it in Jupyter notebook more specifically) crashes with:

---------------------------------------------------------------------------
UnsupportedOperation                      Traceback (most recent call last)
<ipython-input-8-a3517d055ccb> in <module>
----> 1 mne.bem.make_watershed_bem(fs_subject, overwrite=True)

</home/christian/mne-python/mne/externals/decorator.py:decorator-gen-50> in make_watershed_bem(subject, subjects_dir, overwrite, volume, atlas, gcaatlas, preflood, show, copy, T1, brainmask, verbose)

~/mne-python/mne/utils/_logging.py in wrapper(*args, **kwargs)
     88             with use_log_level(verbose_level):
     89                 return function(*args, **kwargs)
---> 90         return function(*args, **kwargs)
     91     return FunctionMaker.create(
     92         function, 'return decfunc(%(signature)s)',

~/mne-python/mne/bem.py in make_watershed_bem(subject, subjects_dir, overwrite, volume, atlas, gcaatlas, preflood, show, copy, T1, brainmask, verbose)
   1148                 % (ws_dir, ' '.join(cmd)))
   1149     os.makedirs(op.join(ws_dir))
-> 1150     run_subprocess_env(cmd)
   1151     if op.isfile(T1_mgz):
   1152         new_info = _extract_volume_info(T1_mgz)

</home/christian/mne-python/mne/externals/decorator.py:decorator-gen-1> in run_subprocess(command, return_code, verbose, *args, **kwargs)

~/mne-python/mne/utils/_logging.py in wrapper(*args, **kwargs)
     88             with use_log_level(verbose_level):
     89                 return function(*args, **kwargs)
---> 90         return function(*args, **kwargs)
     91     return FunctionMaker.create(
     92         function, 'return decfunc(%(signature)s)',

~/mne-python/mne/utils/misc.py in run_subprocess(command, return_code, verbose, *args, **kwargs)
    113         The return code, only returned if ``return_code == True``.
    114     """
--> 115     with running_subprocess(command, *args, **kwargs) as p:
    116         stdout_, stderr = p.communicate()
    117     stdout_ = u'' if stdout_ is None else stdout_.decode('utf-8')

~/anaconda3/lib/python3.7/contextlib.py in __enter__(self)
    110         del self.args, self.kwds, self.func
    111         try:
--> 112             return next(self.gen)
    113         except StopIteration:
    114             raise RuntimeError("generator didn't yield") from None

~/mne-python/mne/utils/misc.py in running_subprocess(command, after, verbose, *args, **kwargs)
    187     try:
    188         print(args, kwargs)
--> 189         p = subprocess.Popen(command, *args, **kwargs)
    190     except Exception:
    191         if isinstance(command, str):

~/anaconda3/lib/python3.7/subprocess.py in __init__(self, args, bufsize, executable, stdin, stdout, stderr, preexec_fn, close_fds, shell, cwd, env, universal_newlines, startupinfo, creationflags, restore_signals, start_new_session, pass_fds, encoding, errors, text)
    726         (p2cread, p2cwrite,
    727          c2pread, c2pwrite,
--> 728          errread, errwrite) = self._get_handles(stdin, stdout, stderr)
    729 
    730         # We wrap OS handles *before* launching the child, otherwise a

~/anaconda3/lib/python3.7/subprocess.py in _get_handles(self, stdin, stdout, stderr)
   1357             else:
   1358                 # Assuming file-like object
-> 1359                 c2pwrite = stdout.fileno()
   1360 
   1361             if stderr is None:

UnsupportedOperation: fileno

My undestanding of this issue is that the problems comes from sys.stdout being passed within the kwargs argument to subprocess.Popen() by mne.utils.misc.running_subprocess. Thus, sys.stdout gets passed as sysout to Popen() and the object contained in the variable sys.stdout does not have a fileno method: Within a Jupyter notebook:

import sys
print("#1:", sys.stdout)
print("#2:", sys.__stdout__)
print("#3:", sys.__stdout__.fileno())
print("#4:", sys.stdout.fileno())

gives the following output:

#1: <ipykernel.iostream.OutStream object at 0x7fc4e055fdd0>
#2: <_io.TextIOWrapper name='<stdout>' mode='w' encoding='UTF-8'>
#3: 1

---------------------------------------------------------------------------
UnsupportedOperation                      Traceback (most recent call last)
<ipython-input-19-ac77b7d7dd6f> in <module>
      3 print("#2:", sys.__stdout__)
      4 print("#3:", sys.__stdout__.fileno())
----> 5 print("#4:", sys.stdout.fileno())

UnsupportedOperation: fileno

Any mne functionnality relying on mne.utils.misc.running_subprocess is therefore expected to fail in a similar fashion.

Actually, I am sure you will be familiar with the issue since I found ipython/ipykernel#272 related to this issue by @larsoner . Not sure if the fact that this causes bugs in MNE was spotted since I did not find corresponding issues reported for MNE.

Additional information

Platform: Linux-5.0.0-32-generic-x86_64-with-debian-buster-sid
Python: 3.7.4 (default, Aug 13 2019, 20:35:49) [GCC 7.3.0]
Executable: /home/christian/anaconda3/bin/python
CPU: x86_64: 4 cores
Memory: 7.5 GB
mne: 0.20.dev0
numpy: 1.17.4 {blas=openblas, lapack=openblas}
scipy: 1.3.1
matplotlib: 3.1.1 {backend=module://ipykernel.pylab.backend_inline}
sklearn: 0.21.3
numba: Not found
nibabel: 2.5.1
cupy: Not found
pandas: 0.25.1
dipy: Not found
mayavi: 4.7.1 {qt_api=pyqt5, PyQt5=5.9.2}
pyvista: Not found
vtk: 8.1.2

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions