Suggestion: Prettier terminal outputs #3976
Description
A few ideas for making the terminal output from Nextflow a bit prettier / easier to use (taken from a discussion on Slack with @bentsherman):
- More use of colour
- For example: Cached = grey, failed but ignored = orange, etc etc. Even small things like the
[characters in dim / grey. Nextflow version number in grey, etc.
- For example: Cached = grey, failed but ignored = orange, etc etc. Even small things like the
- Use of spinners to make it clear that Nextflow hasn't crashed, so that there's always movement even when waiting for stuff.
- Bonus points: combine with a progress bar when we know the number of expected tasks for a process.
- Ambitious: ability to show stdout from running commands, but in a truncated rolling tail of 3-4 lines.
- Example:
docker compose, which shows the build log output as it goes along, but without taking up the whole terminal.
- Example:
Just to get the conversation moving, I threw a Nextflow log into a Python script and coloured it using Rich. Gives an idea of the kind of thing I'm thinking of for the colours (I stole the header style from Astro).
demo.py
from time import sleep
from random import randint
from rich.console import Console
from rich.progress import Progress, TextColumn, SpinnerColumn, BarColumn, ProgressColumn
console = Console()
log_msg = """
🚀 [black on green] N E X T F L O W [/] [dim]~ [/][green]version 23.04.1[/]
┃ Launching [magenta]`../main.nf`[/] [dim][[/][bold cyan]boring_montalcini[/][dim]][/] [dim]DSL2 - [/][dim magenta]revision: [/][magenta]d420d96c87[/]
[dim]executor > local (3)[/]
[dim][[/][blue]89/615a07[/][dim]][/] [dim]process >[/] [grey]NFCORE_METHYLSEQ:METHYLSEQ:PREPARE_GENOME:BISMARK_GENOMEPREPARATION[/] [dim]([/][yellow]BismarkIndex/genome.fa[/][dim])[/] [dim][[/][green]100%[/][dim]][/] 1 of 1 [green]✔[/]
[dim][[/][blue]2a/a06065[/][dim]][/] [dim]process >[/] [grey]NFCORE_METHYLSEQ:METHYLSEQ:INPUT_CHECK:SAMPLESHEET_CHECK[/][dim]([/][yellow]samplesheet_test.csv[/][dim])[/] [dim][[/][green]100%[/][dim]][/] 1 of 1, [dim]cached: 1[/] [green]✔[/]
[dim]executor > local (3)[/]
[dim][[/][blue]89/615a07[/][dim]][/] [dim]process >[/] [grey]NFCORE_METHYLSEQ:METHYLSEQ:PREPARE_GENOME:BISMARK_GENOMEPREPARATION[/] [dim]([/][yellow]BismarkIndex/genome.fa[/][dim])[/] [dim][[/][green]100%[/][dim]][/] 1 of 1 [green]✔[/]
[dim][[/][blue]2a/a06065[/][dim]][/] [dim]process >[/] [grey]NFCORE_METHYLSEQ:METHYLSEQ:INPUT_CHECK:SAMPLESHEET_CHECK[/] [dim]([/][yellow]samplesheet_test.csv[/][dim])[/] [dim][[/][green]100%[/][dim]][/] 1 of 1, [dim]cached: 1[/] [green]✔[/]
[dim][- ][/] [dim]process >[/] [grey]NFCORE_METHYLSEQ:METHYLSEQ:CAT_FASTQ[/] [dim][[/][blue] 0%[/][dim]][/] 0 of 1
[dim][[/][blue]45/35fba0[/][dim]][/] [dim]process >[/] [grey]NFCORE_METHYLSEQ:METHYLSEQ:FASTQC[/] [dim]([/][yellow]SRR389222_sub1[/][dim])[/] [dim][[/][blue] 0%[/][dim]][/] 0 of 1
[dim][[/][blue]88/b2a6c9[/][dim]][/] [dim]process >[/] [grey]NFCORE_METHYLSEQ:METHYLSEQ:TRIMGALORE[/] [dim]([/][yellow]SRR389222_sub1[/][dim])[/] [dim][[/][blue] 0%[/][dim]][/] 0 of 1
[dim][- ][/] [dim]process >[/] [grey]NFCORE_METHYLSEQ:METHYLSEQ:BISMARK:BISMARK_ALIGN[/] -
[dim][- ][/] [dim]process >[/] [grey]NFCORE_METHYLSEQ:METHYLSEQ:BISMARK:SAMTOOLS_SORT_ALIGNED[/] -
[dim][- ][/] [dim]process >[/] [grey]NFCORE_METHYLSEQ:METHYLSEQ:BISMARK:BISMARK_DEDUPLICATE[/] -
[dim][- ][/] [dim]process >[/] [grey]NFCORE_METHYLSEQ:METHYLSEQ:BISMARK:BISMARK_METHYLATIONEXTRACTOR[/] -
[dim][- ][/] [dim]process >[/] [grey]NFCORE_METHYLSEQ:METHYLSEQ:BISMARK:BISMARK_REPORT[/] -
[dim][- ][/] [dim]process >[/] [grey]NFCORE_METHYLSEQ:METHYLSEQ:BISMARK:BISMARK_SUMMARY[/] -
[dim][- ][/] [dim]process >[/] [grey]NFCORE_METHYLSEQ:METHYLSEQ:BISMARK:SAMTOOLS_SORT_DEDUPLICATED[/] -
[dim][- ][/] [dim]process >[/] [grey]NFCORE_METHYLSEQ:METHYLSEQ:QUALIMAP_BAMQC[/] -
[dim][- ][/] [dim]process >[/] [grey]NFCORE_METHYLSEQ:METHYLSEQ:PRESEQ_LCEXTRAP[/] -
[dim][- ][/] [dim]process >[/] [grey]NFCORE_METHYLSEQ:METHYLSEQ:CUSTOM_DUMPSOFTWAREVERSIONS[/] -
[dim][- ][/] [dim]process >[/] [grey]NFCORE_METHYLSEQ:METHYLSEQ:MULTIQC[/] -
"""
console.print(log_msg, highlight=False)
with Progress(
TextColumn("{task.description}"),
SpinnerColumn(),
TextColumn(
"[dim][[/][blue]{task.percentage:>3.0f}%[/][dim]][/] {task.completed:>2.0f} of {task.total:>2.0f} "
),
BarColumn(bar_width=10),
) as progress:
task1 = progress.add_task(
"[dim][[/][blue]2a/a06065[/][dim]][/] [dim]process >[/] [grey]NFCORE_METHYLSEQ:METHYLSEQ:INPUT_CHECK:SAMPLESHEET_CHECK[/] [dim]([/][yellow]samplesheet_test.csv[/][dim])[/] ",
total=4,
)
task2 = progress.add_task(
"[dim][[/][blue]45/35fba0[/][dim]][/] [dim]process >[/] [grey]NFCORE_METHYLSEQ:METHYLSEQ:FASTQC[/] [dim]([/][yellow]SRR389222_sub1[/][dim])[/]",
total=18,
)
for i in range(18):
sleep(randint(2, 4) / 10.0)
if i < 4:
progress.update(task1, advance=1)
progress.update(task2, advance=1)
Arguably overkill with the colours, but hopefully you get my point - it's easier to scan for key information with this.
..ok and updated with an animated spinner / tiny progress bar to give the impression of animation:
CleanShot.2023-05-29.at.11.58.18.mp4
Metadata
Metadata
Assignees
Labels
No labels