Snakemake version
$ snakemake --version
5.15.0
Describe the bug
This could be viewed as either a bug or a feature request depending on how the jupyter notebook rule was intended to work.
This relates to the usage of the Jupyter notebook integration rules. Currently, if an exception occurs in one of the cells of the template notebook that is being executed and a log: of the notebook is requested, when the rule fails all output is removed, but an empty notebook file is left behind. This makes it difficult to diagnose the problem in the notebook. It would be good to have a copy of the executed notebook left behind with the exception output for diagnosing the problem. This could be achieved if the --allow-errors flag is passed to the call to jupyter-nbconvert at
|
cmd_tmp = "jupyter-nbconvert --execute {output_parameter} --to notebook --ExecutePreprocessor.timeout=-1 {{fname:q}}".format( |
However, adding this flag will also mean that the rule will not fail if an exception is raised in the notebook as long as all outputs are generated. Therefore, it may be desirable to have this flag be added only when specified by the rule declaration.
Logs
n/a
Minimal example
Snakefile:
rule test_notebook_error:
input:
output:
"output"
log:
notebook = "generated.ipynb"
notebook:
"template.ipynb"
template.ipynb:
{
"nbformat": 4,
"nbformat_minor": 2,
"metadata": {
"language_info": {
"name": "python",
"codemirror_mode": {
"name": "ipython",
"version": 3
}
},
"orig_nbformat": 2,
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"npconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": 3
},
"cells": [
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"raise Exception('something went wrong')\n",
"with open(snakemake.output[0], 'w') as output:\n",
" output.write('completed')"
]
}
]
}
Running snakemake --cores 1 in a directory with the two files above demonstrates that the rule fails but an empty generated.ipynb is left behind.
Additional context
I'd be happy to submit a pull request to address this issue with some guidance on the preferred approach to allowing errors to occur in the notebook (by default vs explicit request in rule).
Snakemake version
Describe the bug
This could be viewed as either a bug or a feature request depending on how the jupyter notebook rule was intended to work.
This relates to the usage of the Jupyter notebook integration rules. Currently, if an exception occurs in one of the cells of the template notebook that is being executed and a
log:of the notebook is requested, when the rule fails all output is removed, but an empty notebook file is left behind. This makes it difficult to diagnose the problem in the notebook. It would be good to have a copy of the executed notebook left behind with the exception output for diagnosing the problem. This could be achieved if the--allow-errorsflag is passed to the call tojupyter-nbconvertatsnakemake/snakemake/notebook.py
Line 27 in 6d3aa42
However, adding this flag will also mean that the rule will not fail if an exception is raised in the notebook as long as all outputs are generated. Therefore, it may be desirable to have this flag be added only when specified by the rule declaration.
Logs
n/a
Minimal example
Snakefile:
template.ipynb:
Running
snakemake --cores 1in a directory with the two files above demonstrates that the rule fails but an empty generated.ipynb is left behind.Additional context
I'd be happy to submit a pull request to address this issue with some guidance on the preferred approach to allowing errors to occur in the notebook (by default vs explicit request in rule).