--show-failed-logs if binary file used as log file #2826
Description
Snakemake version
main banch
Describe the bug
If a binary file is set as log output and --show-failed-logs (or other options triggering logs to be shown) is activated, if a job crashes the whole workflow execution crashes as snakemake tries to print the binary log file as text.
Logs
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/project/home/vizano/mambaforge/envs/snakemake8/lib/python3.12/site-packages/snakemake/executors/local.py", line 259, in _callback
raise ex
File "/project/home/vizano/mambaforge/envs/snakemake8/lib/python3.12/concurrent/futures/thread.py", line 58, in run
result = self.fn(*self.args, **self.kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/project/home/vizano/mambaforge/envs/snakemake8/lib/python3.12/site-packages/snakemake/executors/local.py", line 245, in cached_or_run
run_func(*args)
File "/project/home/vizano/mambaforge/envs/snakemake8/lib/python3.12/site-packages/snakemake/executors/local.py", line 232, in spawn_job
raise SpawnedJobError()
snakemake.exceptions.SpawnedJobError
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/project/home/vizano/mambaforge/envs/snakemake8/lib/python3.12/concurrent/futures/_base.py", line 340, in _invoke_callbacks
callback(self)
File "/project/home/vizano/mambaforge/envs/snakemake8/lib/python3.12/site-packages/snakemake/executors/local.py", line 266, in _callback
self.report_job_error(job_info)
File "/project/home/vizano/mambaforge/envs/snakemake8/lib/python3.12/site-packages/snakemake_interface_executor_plugins/executors/base.py", line 92, in report_job_error
self.print_job_error(job_info, msg, **kwargs)
File "/project/home/vizano/mambaforge/envs/snakemake8/lib/python3.12/site-packages/snakemake_interface_executor_plugins/executors/base.py", line 113, in print_job_error
job_info.job.log_error(msg, **kwargs)
File "/project/home/vizano/mambaforge/envs/snakemake8/lib/python3.12/site-packages/snakemake/jobs.py", line 1023, in log_error
logger.job_error(**self.get_log_error_info(msg, indent, aux_logs, **kwargs))
File "/project/home/vizano/mambaforge/envs/snakemake8/lib/python3.12/site-packages/snakemake/logging.py", line 418, in job_error
self.handler(msg)
File "/project/home/vizano/mambaforge/envs/snakemake8/lib/python3.12/site-packages/snakemake/logging.py", line 359, in handler
handler(msg)
File "/project/home/vizano/mambaforge/envs/snakemake8/lib/python3.12/site-packages/snakemake/logging.py", line 589, in text_handler
self.logger.error("\n".join(map(indent, job_error())))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/project/home/vizano/mambaforge/envs/snakemake8/lib/python3.12/site-packages/snakemake/logging.py", line 584, in job_error
yield from show_logs(msg["log"])
File "/project/home/vizano/mambaforge/envs/snakemake8/lib/python3.12/site-packages/snakemake/logging.py", line 505, in show_logs
content = open(f, "r").read()
^^^^^^^^^^^^^^^^^^^
File "<frozen codecs>", line 322, in decode
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x80 in position 0: invalid start byte
Minimal example
Snakefile:
rule broken_rule:
log:
'log.bin'
run:
import pickle
with open(log[0], 'wb') as f:
pickle.dump(log, f)
asd
Run with:
snakemake --jobs 1 --show-failed-logs
Additional context
I encountered this issue when running a workflow that uses the log to store R environment when the rule is run (I guess to facilitate debugging).
For me this triggered a whole Slurm execution to fail and it took me quite a while to figure out what was going on.