My high coverage, about 500x, crashes when running on the grid.
Seems that batch job submission failed due to some parameter being too long.
Restarting or manually trying to run batch script does not seem to work.
Please advice.
Runing the following on the grid:
#!/bin/bash
#SBATCH --job-name=strain4-canu-maxcov
#SBATCH --partition=hugemem
#SBATCH -o output_%j.txt
#SBATCH -e errors_%j.txt
cp Strain04-from1804.fastq.gz $SCRATCH
## Move to work directory
cd $SCRATCH
## Loading and running canu module
module load canu
canu -p strain4-canu-maxcov -d strain4-canu-maxcov genomeSize=4.8m corOutCoverage=999 -nanopore-raw Strain04-from1804.fastq.gz
**Error output**
-- CONFIGURE CANU
--
-- Detected Java(TM) Runtime Environment '1.8.0_65' (from '/local/genome/packages/java/jdk1.8.0_65/bin/java').
-- Detected gnuplot version '5.0 patchlevel 6' (from 'gnuplot') and image format 'svg'.
-- Detected 80 CPUs and 1010 gigabytes of memory.
-- Detected Slurm with 'sinfo' binary in /local/genome/packages/slurm/15.08.5/bin/sinfo.
-- Detected Slurm with 'MaxArraySize' limited to 99999 jobs.
--
-- Found 1 host with 12 cores and 31 GB memory under Slurm control.
-- Found 2 hosts with 80 cores and 1009 GB memory under Slurm control.
-- Found 18 hosts with 8 cores and 15 GB memory under Slurm control.
-- Found 7 hosts with 32 cores and 189 GB memory under Slurm control.
-- Found 3 hosts with 4 cores and 3 GB memory under Slurm control.
-- Found 1 host with 6 cores and 14 GB memory under Slurm control.
-- Found 1 host with 48 cores and 189 GB memory under Slurm control.
-- Found 2 hosts with 12 cores and 62 GB memory under Slurm control.
--
-- (tag)Threads
-- (tag)Memory |
-- (tag) | | algorithm
-- ------- ------ -------- -----------------------------
-- Grid: meryl 8 GB 4 CPUs (k-mer counting)
-- Grid: cormhap 6 GB 4 CPUs (overlap detection with mhap)
-- Grid: obtovl 4 GB 4 CPUs (overlap detection)
-- Grid: utgovl 4 GB 4 CPUs (overlap detection)
-- Grid: ovb 2 GB 1 CPU (overlap store bucketizer)
-- Grid: ovs 8 GB 1 CPU (overlap store sorting)
-- Grid: red 3 GB 4 CPUs (read error detection)
-- Grid: oea 4 GB 1 CPU (overlap error adjustment)
-- Grid: bat 16 GB 4 CPUs (contig construction)
-- Grid: gfa 8 GB 4 CPUs (GFA alignment and processing)
--
-- In 'strain4-canu-maxcov.gkpStore', found Nanopore reads:
-- Raw: 332249
-- Corrected: 216721
-- Trimmed: 196452
--
-- Generating assembly 'strain4-canu-maxcov' in '/mnt/SCRATCH/rmeisal/strain4-canu-maxcov'
--
-- Parameters:
--
-- genomeSize 4800000
--
-- Overlap Generation Limits:
-- corOvlErrorRate 0.3200 ( 32.00%)
-- obtOvlErrorRate 0.1440 ( 14.40%)
-- utgOvlErrorRate 0.1440 ( 14.40%)
--
-- Overlap Processing Limits:
-- corErrorRate 0.5000 ( 50.00%)
-- obtErrorRate 0.1440 ( 14.40%)
-- utgErrorRate 0.1440 ( 14.40%)
-- cnsErrorRate 0.1920 ( 19.20%)
--
--
-- BEGIN ASSEMBLY
--
--
-- Running jobs. First attempt out of 2.
--
CRASH:
CRASH: Canu 1.7
CRASH: Please panic, this is abnormal.
ABORT:
CRASH: Failed to submit batch jobs.
CRASH:
CRASH: Failed at /mnt/various/local/genome/packages/canu/1.7/Linux-amd64/bin/../lib/site_perl/canu/Execution.pm line 1101.
CRASH: canu::Execution::submitOrRunParallelJob("strain4-canu-maxcov", "red", "unitigging/3-overlapErrorAdjustment", "red", 1, 2, 3, 4, ...) called at /mnt/various/local/genome/packages/canu/1.7/Linux-amd64/bin/../lib/site_perl/canu/OverlapErrorAdjustment.pm line 351
CRASH: canu::OverlapErrorAdjustment::readErrorDetectionCheck("strain4-canu-maxcov") called at /local/genome/packages/canu/1.7/bin/canu line 685
CRASH:
CRASH: Last 50 lines of the relevant log file (unitigging/3-overlapErrorAdjustment/red.jobSubmit-01.out):
CRASH:
CRASH:
Output from red.jobsubmit-01.out
sbatch: error: Batch job submission failed: Pathname of a file, directory or other parameter too long
My high coverage, about 500x, crashes when running on the grid.
Seems that batch job submission failed due to some parameter being too long.
Restarting or manually trying to run batch script does not seem to work.
Please advice.
Runing the following on the grid:
Output from red.jobsubmit-01.out