Now there is a warning on Azure for notebook testing:

FutureWarning: Method cleanup(connection_file=True) is deprecated, use cleanup_resources(restart=False).

This is probably because of the new version of jupyter_client:

Bumping nbconvert solved the warning but now pytest-notebook is not ready for the change:

AttributeError: 'ExecuteCoveragePreprocessor' object has no attribute 'setup_preprocessor'

I think it's fine since nbconvert 6.0.0 is not officially released yet. We could probably ignore for now and report the change on pytest-notebook ?

This is trickier than it seems. There is an API change in nbconvert 6.0.0 leading to some part of the code to be split in nbclient.

As we do not gather coverage informations from this notebook testing, I could consider migrating to nbclient instead.

I reported the FutureWarning in jupyter/nbclient#89.

For now, we can just pin jupyter_client<6.1.5.

What do you think @agramfort, @larsoner ?

I am confused. Why removing pytest-notebook?

We don't use the coverage information and it's not ready yet for nbconvert 6.0.0.

What do you mean we don't use it? It should be reported like all other runs (other than style)

It should be reported like all other runs

I was not aware of that

nbclient executes the notebook test.ipynb in the test_notebook_3d_backend() function of the test_notebook.py file.

I'm now completely lost for coverage.

nbclient executes the notebook test.ipynb in the test_notebook_3d_backend() function of the test_notebook.py file.

I'm now completely lost for coverage.

As long as it does this in the same Python process, I think coverage should work. And based on the coverage numbers for the notebook backend, it should be working.

https://codecov.io/gh/mne-tools/mne-python/src/master/mne/viz/backends/_notebook.py

Closing in favor of #7943