Skip to content

Canu 1.7 Crashed #831

@romeA4

Description

@romeA4

My high coverage, about 500x, crashes when running on the grid.
Seems that batch job submission failed due to some parameter being too long.
Restarting or manually trying to run batch script does not seem to work.
Please advice.

Runing the following on the grid:

#!/bin/bash
#SBATCH --job-name=strain4-canu-maxcov  		
#SBATCH --partition=hugemem				
#SBATCH -o output_%j.txt				
#SBATCH -e errors_%j.txt				


cp Strain04-from1804.fastq.gz $SCRATCH

## Move to work directory
cd $SCRATCH

## Loading and running canu module
module load canu
canu -p strain4-canu-maxcov -d strain4-canu-maxcov genomeSize=4.8m corOutCoverage=999 -nanopore-raw Strain04-from1804.fastq.gz

**Error output**

-- CONFIGURE CANU
--
-- Detected Java(TM) Runtime Environment '1.8.0_65' (from '/local/genome/packages/java/jdk1.8.0_65/bin/java').
-- Detected gnuplot version '5.0 patchlevel 6' (from 'gnuplot') and image format 'svg'.
-- Detected 80 CPUs and 1010 gigabytes of memory.
-- Detected Slurm with 'sinfo' binary in /local/genome/packages/slurm/15.08.5/bin/sinfo.
-- Detected Slurm with 'MaxArraySize' limited to 99999 jobs.
-- 
-- Found   1 host  with  12 cores and   31 GB memory under Slurm control.
-- Found   2 hosts with  80 cores and 1009 GB memory under Slurm control.
-- Found  18 hosts with   8 cores and   15 GB memory under Slurm control.
-- Found   7 hosts with  32 cores and  189 GB memory under Slurm control.
-- Found   3 hosts with   4 cores and    3 GB memory under Slurm control.
-- Found   1 host  with   6 cores and   14 GB memory under Slurm control.
-- Found   1 host  with  48 cores and  189 GB memory under Slurm control.
-- Found   2 hosts with  12 cores and   62 GB memory under Slurm control.
--
--                     (tag)Threads
--            (tag)Memory         |
--        (tag)         |         |  algorithm
--        -------  ------  --------  -----------------------------
-- Grid:  meryl      8 GB    4 CPUs  (k-mer counting)
-- Grid:  cormhap    6 GB    4 CPUs  (overlap detection with mhap)
-- Grid:  obtovl     4 GB    4 CPUs  (overlap detection)
-- Grid:  utgovl     4 GB    4 CPUs  (overlap detection)
-- Grid:  ovb        2 GB    1 CPU   (overlap store bucketizer)
-- Grid:  ovs        8 GB    1 CPU   (overlap store sorting)
-- Grid:  red        3 GB    4 CPUs  (read error detection)
-- Grid:  oea        4 GB    1 CPU   (overlap error adjustment)
-- Grid:  bat       16 GB    4 CPUs  (contig construction)
-- Grid:  gfa        8 GB    4 CPUs  (GFA alignment and processing)
--
-- In 'strain4-canu-maxcov.gkpStore', found Nanopore reads:
--   Raw:        332249
--   Corrected:  216721
--   Trimmed:    196452
--
-- Generating assembly 'strain4-canu-maxcov' in '/mnt/SCRATCH/rmeisal/strain4-canu-maxcov'
--
-- Parameters:
--
--  genomeSize        4800000
--
--  Overlap Generation Limits:
--    corOvlErrorRate 0.3200 ( 32.00%)
--    obtOvlErrorRate 0.1440 ( 14.40%)
--    utgOvlErrorRate 0.1440 ( 14.40%)
--
--  Overlap Processing Limits:
--    corErrorRate    0.5000 ( 50.00%)
--    obtErrorRate    0.1440 ( 14.40%)
--    utgErrorRate    0.1440 ( 14.40%)
--    cnsErrorRate    0.1920 ( 19.20%)
--
--
-- BEGIN ASSEMBLY
--
--
-- Running jobs.  First attempt out of 2.
--

CRASH:
CRASH: Canu 1.7
CRASH: Please panic, this is abnormal.
ABORT:
CRASH:   Failed to submit batch jobs.
CRASH:
CRASH: Failed at /mnt/various/local/genome/packages/canu/1.7/Linux-amd64/bin/../lib/site_perl/canu/Execution.pm line 1101.
CRASH: 	canu::Execution::submitOrRunParallelJob("strain4-canu-maxcov", "red", "unitigging/3-overlapErrorAdjustment", "red", 1, 2, 3, 4, ...) called at /mnt/various/local/genome/packages/canu/1.7/Linux-amd64/bin/../lib/site_perl/canu/OverlapErrorAdjustment.pm line 351
CRASH: 	canu::OverlapErrorAdjustment::readErrorDetectionCheck("strain4-canu-maxcov") called at /local/genome/packages/canu/1.7/bin/canu line 685
CRASH: 
CRASH: Last 50 lines of the relevant log file (unitigging/3-overlapErrorAdjustment/red.jobSubmit-01.out):
CRASH:
CRASH:

Output from red.jobsubmit-01.out

sbatch: error: Batch job submission failed: Pathname of a file, directory or other parameter too long

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions