Skip to content

failed to add error rates to overlap store #758

@dibyendukumar

Description

@dibyendukumar

Hi,
For several days, I am stuck at unitigging/3-overlapErrorAdjustment/ stage. Our IT expert suggested to increase memory as CANU is running out of memory. I have Increased memory to all steps that seems relevant, still getting the same error. Please suggest !!!
Thanks,

CANU command

/ingens/apps/canu/Linux-amd64/bin/canu -assemble -p SeqTrim -d SeqRStrim gridOptionsoea="-l mem=30gb" gridOptions="-c enabled" gridOptionsGFA="-l mem
=400gb" gridOptionsovb="-l mem=150gb" gridOptionsovs="-l mem=150gb" gridOptionscorovl="-l mem=150gb" gridOptionscor="-l mem=120gb" corPartitions=1000
 minOverlapLength=1200 minReadLength=3000 ovsMemory=150g-300g genomeSize=2.3g -pacbio-corrected /***/***/***/***/canu/SeqRStrim15/SeqTrim.trimmedReads.fasta.gz'

I am using Canu snapshot v1.6 +120 changes (r8532 6ed280b)
unitigging/3-overlapErrorAdjustment/oea.apply.err file is empty. Upon restart, I haven't deleted 3-overlapErrorAdjustment folder.

CONFIGURE CANU
--
-- Detected Java(TM) Runtime Environment '1.8.0_141' (from 'java').
-- Detected gnuplot version '4.2 patchlevel 6 ' (from 'gnuplot') and image format 'png'.
-- Detected 24 CPUs and 47 gigabytes of memory.
-- Detected PBS/Torque '' with 'pbsnodes' binary in /usr/bin/pbsnodes.
-- Detecting PBS/Torque resources.
-- 
-- Found   2 hosts with  24 cores and   47 GB memory under PBS/Torque control.
-- Found   3 hosts with  24 cores and  126 GB memory under PBS/Torque control.
-- Found   4 hosts with  64 cores and  504 GB memory under PBS/Torque control.
--
--                     (tag)Threads
--            (tag)Memory         |
--        (tag)         |         |  algorithm
--        -------  ------  --------  -----------------------------
-- Grid:  meryl    126 GB   21 CPUs  (k-mer counting)
-- Grid:  cormhap   23 GB   12 CPUs  (overlap detection with mhap)
-- Grid:  obtovl    15 GB    8 CPUs  (overlap detection)
-- Grid:  utgovl    15 GB    8 CPUs  (overlap detection)
-- Grid:  ovb        2 GB    1 CPU   (overlap store bucketizer)
-- Grid:  ovs      300 GB    1 CPU   (overlap store sorting)
-- Grid:  red       15 GB    8 CPUs  (read error detection)
-- Grid:  oea        4 GB    1 CPU   (overlap error adjustment)
-- Grid:  bat      252 GB   32 CPUs  (contig construction)
-- Grid:  cns       72 GB    8 CPUs  (consensus)
-- Grid:  gfa       23 GB   12 CPUs  (GFA alignment and processing)
--
-- Found PacBio corrected reads in 'unitigging/SeqTrim.gkpStore'.
--
-- Generating assembly 'SeqTrim' in '/***/***/***/***/canu/SeqRStrim'
--
-- Parameters:
--
--  genomeSize        2300000000
--
--  Overlap Generation Limits:
--    corOvlErrorRate 0.2400 ( 24.00%)
--    obtOvlErrorRate 0.0450 (  4.50%)
--    utgOvlErrorRate 0.0450 (  4.50%)
--
--  Overlap Processing Limits:
--    corErrorRate    0.3000 ( 30.00%)
--    obtErrorRate    0.0450 (  4.50%)
--    utgErrorRate    0.0450 (  4.50%)
--    cnsErrorRate    0.0750 (  7.50%)
--
--
-- BEGIN ASSEMBLY
--
----------------------------------------
-- Starting command on Thu Jan 18 12:34:55 2018 with 149640.256 GB free disk space

    cd unitigging/3-overlapErrorAdjustment
    /ingens/apps/canu-master/Linux-amd64/bin/ovStoreBuild \
      -G ../SeqTrim.gkpStore \
      -O ../SeqTrim.ovlStore \
      -evalues \
      -L ./oea.files \
    > ./oea.apply.err 2>&1
sh: line 5: 18243 Killed                  /ingens/apps/canu-master/Linux-amd64/bin/ovStoreBuild -G ../SeqTrim.gkpStore -O ../SeqTrim.ovlStore -evalue
s -L ./oea.files > ./oea.apply.err 2>&1

-- Finished on Thu Jan 18 12:35:34 2018 (39 seconds) with 149640.102 GB free disk space
----------------------------------------
ERROR:
ERROR:  Failed with exit code 137.  (rc=35072)
ERROR:

ABORT:
ABORT: Canu snapshot v1.6 +120 changes (r8532 6ed280b7565194370584994956987ea4dae10a05)
ABORT: Don't panic, but a mostly harmless error occurred and Canu stopped.
ABORT: Try restarting.  If that doesn't work, ask for help.
ABORT:
ABORT:   failed to add error rates to overlap store.
ABORT:
ABORT: Disk space available:  149640.102 GB
ABORT:
ABORT: Last 50 lines of the relevant log file (unitigging/3-overlapErrorAdjustment/oea.apply.err):
ABORT:
ABORT:

I have also tried latest CANU (Canu snapshot v1.6 +165 changes (r8577 c7394fc)) that failed with following message. our IT group sugested some syntax error in the code (more on this some another time).

Can't "last" outside a loop block at /ingens/apps/canu-latest/Linux-amd64/bin/../lib/site_perl/canu/Gatekeeper.pm line 558.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions