Thanks for doing this @null-a !

I'm not sure why the build is failing, but my guess is that it's a dependency issue. You might try running make test-tutorials locally in a fresh conda env where Pyro is installed using only pip install -e .[test] and ensuring that enough dependencies are added to the test section of setup.py.

@fritzo I don't think it's a dependency thing. The cells that fail are the ones that have output checked in to the repo. (Such output is part of the tutorial content.) I'll try to dig a bit deeper.

Hmm I wonder if it's a headless graphics backend thing like matplotlib.use('Agg') or something with torchvision? We're already trying to ignore output via the --nbval-lax flag 😕

Hmm I wonder if it's a headless graphics backend thing

@fritzo Not sure, but I don't think so, since cell 6 is failing, and that only has textual output.

I've upgraded nbval locally (0.7 -> 0.9) and I'm now seeing the failure locally too.

If I "run all cells" and then save the notebook, then the smoke tests pass. The error message I'm seeing suggests this is something to do with the execution count been None. I think these were stripped out by nbstripout at some point, but I don't recall the details.

I've added the execution counts to the repo by running the notebook locally, and then running nbstripout --keep-count. Let's see if the tests pass with this.

Well I'm not opposed to pinning to an old "working" version of nbval==0.7 if that's the easiest solution 😄

Actually I recall being unable to load a tutorial notebook due to a cell_count=None error at some point, so --keep-count sounds like a good idea.

@null-a Do you want me to merge this as-is, or do you want to add --keep-count to the scrub command in Makefile?

@fritzo I don't have a preference, happy to go with whatever seems best to you. If you'd like the Makefile updating, I can do it as part of this.

We can do it in a late PR. merging. Thanks for adding this!

@fritzo: no worries, thanks for your help.