I have a similar issue like #546 and #718
I installed canu 1.7 on an ubuntu (16.4) workstation. I have a 1G assembly length but with a very low nanopore reads coverage (5X).
This is the command I used : canu -p X -d X_canu genomeSize=1g -nanopore-raw X.fastq corMinCoverage=0 useGrid=false maxMemory=64g maxThreads=6
This is the unitigger.err log
==> PARAMETERS.
Resources:
Memory 64 GB
Compute Threads 6 (command line)
Lengths:
Minimum read 0 bases
Minimum overlap 500 bases
Overlap Error Rates:
Graph 0.144 (14.400%)
Max 0.144 (14.400%)
Deviations:
Graph 6.000
Bubble 6.000
Repeat 3.000
Edge Confusion:
Absolute 2100
Percent 200.0000
Unitig Construction:
Minimum intersection 500 bases
Maxiumum placements 2 positions
Debugging Enabled:
(none)
==> LOADING AND FILTERING OVERLAPS.
ReadInfo()-- Using 541708 reads, no minimum read length used.
OverlapCache()-- limited to 65536MB memory (user supplied).
OverlapCache()-- 4MB for read data.
OverlapCache()-- 20MB for best edges.
OverlapCache()-- 53MB for tigs.
OverlapCache()-- 14MB for tigs - read layouts.
OverlapCache()-- 20MB for tigs - error profiles.
OverlapCache()-- 16384MB for tigs - error profile overlaps.
OverlapCache()-- 0MB for other processes.
OverlapCache()-- ---------
OverlapCache()-- 16507MB for data structures (sum of above).
OverlapCache()-- ---------
OverlapCache()-- 10MB for overlap store structure.
OverlapCache()-- 49017MB for overlap data.
OverlapCache()-- ---------
OverlapCache()-- 65536MB allowed.
OverlapCache()--
OverlapCache()-- Retain at least 3 overlaps/read, based on 1.64x coverage.
OverlapCache()-- Initial guess at 5930 overlaps/read.
OverlapCache()--
OverlapCache()-- Adjusting for sparse overlaps.
OverlapCache()--
OverlapCache()-- reads loading olaps olaps memory
OverlapCache()-- olaps/read all some loaded free
OverlapCache()-- ---------- ------- ------- ----------- ------- --------
OverlapCache()-- 5930 541301 407 151305132 99.95% 46708 MB
OverlapCache()-- 7527103 541708 0 151386714 100.00% 46707 MB
OverlapCache()--
OverlapCache()-- Loading overlaps.
OverlapCache()--
OverlapCache()-- read from store saved in cache
OverlapCache()-- ------------ --------- ------------ ---------
OverlapCache()-- 45163057 (029.83%) 45052037 (029.76%)
OverlapCache()-- 89462599 (059.10%) 89239694 (058.95%)
OverlapCache()-- 128408747 (084.82%) 128092895 (084.61%)
OverlapCache()-- ------------ --------- ------------ ---------
OverlapCache()-- 151386714 (100.00%) 151011835 (099.75%)
OverlapCache()--
OverlapCache()-- Ignored 171306 duplicate overlaps.
OverlapCache()--
OverlapCache()-- Symmetrizing overlaps.
OverlapCache()-- Finding missing twins.
OverlapCache()-- Found 47871 missing twins in 151011835 overlaps, 31 are strong.
OverlapCache()-- Dropping weak non-twin overlaps; allocated 0 MB scratch space.
OverlapCache()-- Dropped 574 overlaps; scratch space released.
OverlapCache()-- Adding 47297 missing twin overlaps.
OverlapCache()-- Finished.
BestOverlapGraph()-- allocating best edges (16MB)
BestOverlapGraph()-- finding initial best edges.
BestOverlapGraph()-- filtering suspicious reads.
BestOverlapGraph()-- marked 186977 reads as suspicious.
BestOverlapGraph()-- filtering high error edges.
BestOverlapGraph()-- filtering reads with lopsided best edges.
BestOverlapGraph()-- filtering spur reads.
BestOverlapGraph()-- detected 44569 spur reads and 210080 singleton reads.
BestOverlapGraph()-- removing best edges for contained reads.
==> BUILDING GREEDY TIGS.
breakSingletonTigs()-- Removed 61406 singleton tigs; reads are now unplaced.
optimizePositions()-- Optimizing read positions for 541709 reads in 67790 tigs, with 6 threads.
optimizePositions()-- Allocating scratch space for 541709 reads (33856 KB).
optimizePositions()-- Initializing positions with 6 threads.
optimizePositions()-- Recomputing positions, iteration 1, with 6 threads.
optimizePositions()-- Reset zero.
optimizePositions()-- Checking convergence.
optimizePositions()-- converged: 354702 reads
optimizePositions()-- changed: 187007 reads
optimizePositions()-- Recomputing positions, iteration 2, with 6 threads.
optimizePositions()-- Reset zero.
optimizePositions()-- Checking convergence.
optimizePositions()-- converged: 356948 reads
optimizePositions()-- changed: 184761 reads
optimizePositions()-- Recomputing positions, iteration 3, with 6 threads.
optimizePositions()-- Reset zero.
optimizePositions()-- Checking convergence.
optimizePositions()-- converged: 358789 reads
optimizePositions()-- changed: 182920 reads
optimizePositions()-- Recomputing positions, iteration 4, with 6 threads.
optimizePositions()-- Reset zero.
optimizePositions()-- Checking convergence.
optimizePositions()-- converged: 359789 reads
optimizePositions()-- changed: 181920 reads
optimizePositions()-- Recomputing positions, iteration 5, with 6 threads.
optimizePositions()-- Reset zero.
optimizePositions()-- Checking convergence.
optimizePositions()-- converged: 360202 reads
optimizePositions()-- changed: 181507 reads
optimizePositions()-- Expanding short reads with 6 threads.
optimizePositions()-- Updating positions.
optimizePositions()-- Finished.
==> PLACE CONTAINED READS.
computeErrorProfiles()-- Computing error profiles for 67790 tigs, with 6 threads.
computeErrorProfiles()-- Finished.
placeContains()-- placing 261725 contained and 261328 unplaced reads, with 6 threads.
placeContains()-- Placed 117228 contained reads and 436 unplaced reads.
placeContains()-- Failed to place 144497 contained reads (too high error suspected) and 260892 unplaced reads (lack of overlaps suspected).
optimizePositions()-- Optimizing read positions for 541709 reads in 67790 tigs, with 6 threads.
optimizePositions()-- Allocating scratch space for 541709 reads (33856 KB).
optimizePositions()-- Initializing positions with 6 threads.
optimizePositions()-- Recomputing positions, iteration 1, with 6 threads.
bogart: bogart/AS_BAT_OptimizePositions.C:210: void Unitig::optimize_recompute(uint32, optPos*, optPos*, bool): Assertion `cnt > 0' failed.
Failed with 'Aborted'; backtrace (libbacktrace):
AS_UTL/AS_UTL_stackTrace.C::97 in _Z17AS_UTL_catchCrashiP9siginfo_tPv()
(null)::0 in (null)()
(null)::0 in (null)()
(null)::0 in (null)()
(null)::0 in (null)()
(null)::0 in (null)()
bogart/AS_BAT_OptimizePositions.C::210 in _ZN6Unitig18optimize_recomputeEjP6optPosS1_b()
bogart/AS_BAT_OptimizePositions.C::414 in _ZN9TigVector17optimizePositionsEPKcS1_._omp_fn.1()
(null)::0 in (null)()
(null)::0 in (null)()
(null)::0 in (null)()
(null)::0 in (null)()
Aborted (core dumped)
and the log messages
-- Canu 1.7
--
-- CITATIONS
--
-- Koren S, Walenz BP, Berlin K, Miller JR, Phillippy AM.
-- Canu: scalable and accurate long-read assembly via adaptive k-mer weighting and repeat separation.
-- Genome Res. 2017 May;27(5):722-736.
-- http://doi.org/10.1101/gr.215087.116
--
-- Read and contig alignments during correction, consensus and GFA building use:
-- Šošic M, Šikic M.
-- Edlib: a C/C ++ library for fast, exact sequence alignment using edit distance.
-- Bioinformatics. 2017 May 1;33(9):1394-1395.
-- http://doi.org/10.1093/bioinformatics/btw753
--
-- Overlaps are generated using:
-- Berlin K, et al.
-- Assembling large genomes with single-molecule sequencing and locality-sensitive hashing.
-- Nat Biotechnol. 2015 Jun;33(6):623-30.
-- http://doi.org/10.1038/nbt.3238
--
-- Myers EW, et al.
-- A Whole-Genome Assembly of Drosophila.
-- Science. 2000 Mar 24;287(5461):2196-204.
-- http://doi.org/10.1126/science.287.5461.2196
--
-- Li H.
-- Minimap and miniasm: fast mapping and de novo assembly for noisy long sequences.
-- Bioinformatics. 2016 Jul 15;32(14):2103-10.
-- http://doi.org/10.1093/bioinformatics/btw152
--
-- Corrected read consensus sequences are generated using an algorithm derived from FALCON-sense:
-- Chin CS, et al.
-- Phased diploid genome assembly with single-molecule real-time sequencing.
-- Nat Methods. 2016 Dec;13(12):1050-1054.
-- http://doi.org/10.1038/nmeth.4035
--
-- Contig consensus sequences are generated using an algorithm derived from pbdagcon:
-- Chin CS, et al.
-- Nonhybrid, finished microbial genome assemblies from long-read SMRT sequencing data.
-- Nat Methods. 2013 Jun;10(6):563-9
-- http://doi.org/10.1038/nmeth.2474
--
-- CONFIGURE CANU
--
-- Detected Java(TM) Runtime Environment '1.8.0_161' (from '/usr/lib/jvm/java-8-oracle/bin/java').
-- Detected gnuplot version '5.0 patchlevel 3' (from 'gnuplot') and image format 'png'.
-- Detected 12 CPUs and 126 gigabytes of memory.
-- Limited to 64g gigabytes from maxMemory option.
-- Limited to 6 CPUs from maxThreads option.
-- No grid engine detected, grid disabled.
--
-- (tag)Concurrency
-- (tag)Threads |
-- (tag)Memory | |
-- (tag) | | | total usage algorithm
-- ------- ------ -------- -------- ----------------- -----------------------------
-- Local: meryl 64 GB 6 CPUs x 1 job 64 GB 6 CPUs (k-mer counting)
-- Local: cormhap 32 GB 6 CPUs x 1 job 32 GB 6 CPUs (overlap detection with mhap)
-- Local: obtovl 16 GB 6 CPUs x 1 job 16 GB 6 CPUs (overlap detection)
-- Local: utgovl 16 GB 6 CPUs x 1 job 16 GB 6 CPUs (overlap detection)
-- Local: ovb 4 GB 1 CPU x 6 jobs 24 GB 6 CPUs (overlap store bucketizer)
-- Local: ovs 32 GB 1 CPU x 2 jobs 64 GB 2 CPUs (overlap store sorting)
-- Local: red 8 GB 3 CPUs x 2 jobs 16 GB 6 CPUs (read error detection)
-- Local: oea 4 GB 1 CPU x 6 jobs 24 GB 6 CPUs (overlap error adjustment)
-- Local: bat 64 GB 6 CPUs x 1 job 64 GB 6 CPUs (contig construction)
-- Local: gfa 16 GB 6 CPUs x 1 job 16 GB 6 CPUs (GFA alignment and processing)
--
-- Found Nanopore uncorrected reads in the input files.
--
-- Generating assembly 'X' in '/media/hal9000/2e96b5f4-d012-482e-a4b5-17a3f95c75061/minion/X/fast5/basecall/assembly/X_canu'
--
-- Parameters:
--
-- genomeSize 1000000000
--
-- Overlap Generation Limits:
-- corOvlErrorRate 0.3200 ( 32.00%)
-- obtOvlErrorRate 0.1440 ( 14.40%)
-- utgOvlErrorRate 0.1440 ( 14.40%)
--
-- Overlap Processing Limits:
-- corErrorRate 0.5000 ( 50.00%)
-- obtErrorRate 0.1440 ( 14.40%)
-- utgErrorRate 0.1440 ( 14.40%)
-- cnsErrorRate 0.1920 ( 19.20%)
--
--
-- BEGIN CORRECTION
--
----------------------------------------
-- Starting command on Thu Apr 12 10:58:38 2018 with 321.307 GB free disk space
cd .
/home/hal9000/soft/canu-1.7/Linux-amd64/bin/gatekeeperCreate \
-minlength 1000 \
-o ./X.gkpStore.BUILDING \
./X.gkpStore.gkp \
> ./X.gkpStore.BUILDING.err 2>&1
-- Finished on Thu Apr 12 11:00:08 2018 (90 seconds) with 319.886 GB free disk space
----------------------------------------
--
-- In gatekeeper store './X.gkpStore':
-- Found 541708 reads.
-- Found 5250323567 bases (5.25 times coverage).
--
-- Read length histogram (one '*' equals 2814.97 reads):
-- 0 4999 197048 **********************************************************************
-- 5000 9999 142994 **************************************************
-- 10000 14999 87150 ******************************
-- 15000 19999 52797 ******************
-- 20000 24999 29757 **********
-- 25000 29999 16401 *****
-- 30000 34999 8357 **
-- 35000 39999 3884 *
-- 40000 44999 1788
-- 45000 49999 803
-- 50000 54999 352
-- 55000 59999 188
-- 60000 64999 90
-- 65000 69999 49
-- 70000 74999 23
-- 75000 79999 10
-- 80000 84999 6
-- 85000 89999 3
-- 90000 94999 5
-- 95000 99999 1
-- 100000 104999 0
-- 105000 109999 0
-- 110000 114999 1
-- 115000 119999 0
-- 120000 124999 0
-- 125000 129999 1
--
-- Running jobs. First attempt out of 2.
----------------------------------------
-- Starting 'meryl' concurrent execution on Thu Apr 12 11:00:13 2018 with 319.883 GB free disk space (1 processes; 1 concurrently)
cd correction/0-mercounts
./meryl.sh 1 > ./meryl.000001.out 2>&1
-- Finished on Thu Apr 12 11:09:23 2018 (550 seconds) with 314.588 GB free disk space
----------------------------------------
-- Meryl finished successfully.
--
-- 16-mers Fraction
-- Occurrences NumMers Unique Total
-- 1- 1 536105395 *******************************************************************--> 0.4161 0.1023
-- 2- 2 281216840 ********************************************************************** 0.6343 0.2096
-- 3- 4 242121204 ************************************************************ 0.7530 0.2970
-- 5- 7 118460611 ***************************** 0.8656 0.4183
-- 8- 11 51998858 ************ 0.9287 0.5235
-- 12- 16 24489358 ****** 0.9599 0.6018
-- 17- 22 12603178 *** 0.9758 0.6591
-- 23- 29 7014713 * 0.9844 0.7016
-- 30- 37 4196580 * 0.9893 0.7339
-- 38- 46 2667898 0.9923 0.7592
-- 47- 56 1781927 0.9943 0.7797
-- 57- 67 1242749 0.9956 0.7965
-- 68- 79 896394 0.9965 0.8108
-- 80- 92 665571 0.9972 0.8231
-- 93- 106 506252 0.9977 0.8338
-- 107- 121 390698 0.9981 0.8432
-- 122- 137 311157 0.9984 0.8516
-- 138- 154 252195 0.9986 0.8592
-- 155- 172 206141 0.9988 0.8661
-- 173- 191 170026 0.9990 0.8725
-- 192- 211 141294 0.9991 0.8783
-- 212- 232 119059 0.9992 0.8837
-- 233- 254 100803 0.9993 0.8887
-- 255- 277 86327 0.9994 0.8934
-- 278- 301 74059 0.9995 0.8977
-- 302- 326 64061 0.9995 0.9018
-- 327- 352 55905 0.9996 0.9056
-- 353- 379 48573 0.9996 0.9092
-- 380- 407 42779 0.9996 0.9126
-- 408- 436 38239 0.9997 0.9158
-- 437- 466 34093 0.9997 0.9188
-- 467- 497 30154 0.9997 0.9218
-- 498- 529 26859 0.9998 0.9245
-- 530- 562 24407 0.9998 0.9272
-- 563- 596 21863 0.9998 0.9297
-- 597- 631 19442 0.9998 0.9321
-- 632- 667 17681 0.9998 0.9344
-- 668- 704 15825 0.9998 0.9366
-- 705- 742 14614 0.9999 0.9386
-- 743- 781 13184 0.9999 0.9406
-- 782- 821 11680 0.9999 0.9425
--
-- 316624 (max occurrences)
-- 4706092552 (total mers, non-unique)
-- 752347427 (distinct mers, non-unique)
-- 536105395 (unique mers)
-- For mhap overlapping, set repeat k-mer threshold to 52421.
--
-- Found 5242197947 16-mers; 1288452822 distinct and 536105395 unique. Largest count 316624.
--
-- OVERLAPPER (mhap) (correction)
--
-- Set corMhapSensitivity=high based on read coverage of 5.
--
-- PARAMETERS: hashes=768, minMatches=2, threshold=0.78
--
-- Given 32 GB, can fit 48000 reads per block.
-- For 13 blocks, set stride to 2 blocks.
-- Logging partitioning to 'correction/1-overlapper/partitioning.log'.
-- Configured 12 mhap precompute jobs.
-- Configured 37 mhap overlap jobs.
--
-- Running jobs. First attempt out of 2.
----------------------------------------
-- Starting 'cormhap' concurrent execution on Thu Apr 12 11:09:52 2018 with 319.882 GB free disk space (12 processes; 1 concurrently)
cd correction/1-overlapper
./precompute.sh 1 > ./precompute.000001.out 2>&1
./precompute.sh 2 > ./precompute.000002.out 2>&1
./precompute.sh 3 > ./precompute.000003.out 2>&1
./precompute.sh 4 > ./precompute.000004.out 2>&1
./precompute.sh 5 > ./precompute.000005.out 2>&1
./precompute.sh 6 > ./precompute.000006.out 2>&1
./precompute.sh 7 > ./precompute.000007.out 2>&1
./precompute.sh 8 > ./precompute.000008.out 2>&1
./precompute.sh 9 > ./precompute.000009.out 2>&1
./precompute.sh 10 > ./precompute.000010.out 2>&1
./precompute.sh 11 > ./precompute.000011.out 2>&1
./precompute.sh 12 > ./precompute.000012.out 2>&1
-- Finished on Thu Apr 12 18:19:51 2018 (25799 seconds) with 304.495 GB free disk space
----------------------------------------
-- All 12 mhap precompute jobs finished successfully.
--
-- Running jobs. First attempt out of 2.
----------------------------------------
-- Starting 'cormhap' concurrent execution on Thu Apr 12 18:19:51 2018 with 304.495 GB free disk space (37 processes; 1 concurrently)
cd correction/1-overlapper
./mhap.sh 1 > ./mhap.000001.out 2>&1
./mhap.sh 2 > ./mhap.000002.out 2>&1
./mhap.sh 3 > ./mhap.000003.out 2>&1
./mhap.sh 4 > ./mhap.000004.out 2>&1
./mhap.sh 5 > ./mhap.000005.out 2>&1
./mhap.sh 6 > ./mhap.000006.out 2>&1
./mhap.sh 7 > ./mhap.000007.out 2>&1
./mhap.sh 8 > ./mhap.000008.out 2>&1
./mhap.sh 9 > ./mhap.000009.out 2>&1
./mhap.sh 10 > ./mhap.000010.out 2>&1
./mhap.sh 11 > ./mhap.000011.out 2>&1
./mhap.sh 12 > ./mhap.000012.out 2>&1
./mhap.sh 13 > ./mhap.000013.out 2>&1
./mhap.sh 14 > ./mhap.000014.out 2>&1
./mhap.sh 15 > ./mhap.000015.out 2>&1
./mhap.sh 16 > ./mhap.000016.out 2>&1
./mhap.sh 17 > ./mhap.000017.out 2>&1
./mhap.sh 18 > ./mhap.000018.out 2>&1
./mhap.sh 19 > ./mhap.000019.out 2>&1
./mhap.sh 20 > ./mhap.000020.out 2>&1
./mhap.sh 21 > ./mhap.000021.out 2>&1
./mhap.sh 22 > ./mhap.000022.out 2>&1
./mhap.sh 23 > ./mhap.000023.out 2>&1
./mhap.sh 24 > ./mhap.000024.out 2>&1
./mhap.sh 25 > ./mhap.000025.out 2>&1
./mhap.sh 26 > ./mhap.000026.out 2>&1
./mhap.sh 27 > ./mhap.000027.out 2>&1
./mhap.sh 28 > ./mhap.000028.out 2>&1
./mhap.sh 29 > ./mhap.000029.out 2>&1
./mhap.sh 30 > ./mhap.000030.out 2>&1
./mhap.sh 31 > ./mhap.000031.out 2>&1
./mhap.sh 32 > ./mhap.000032.out 2>&1
./mhap.sh 33 > ./mhap.000033.out 2>&1
./mhap.sh 34 > ./mhap.000034.out 2>&1
./mhap.sh 35 > ./mhap.000035.out 2>&1
./mhap.sh 36 > ./mhap.000036.out 2>&1
./mhap.sh 37 > ./mhap.000037.out 2>&1
-- Finished on Fri Apr 13 11:52:12 2018 (63141 seconds) with 263.324 GB free disk space
----------------------------------------
-- Found 37 mhap overlap output files.
----------------------------------------
-- Starting command on Fri Apr 13 11:52:12 2018 with 263.324 GB free disk space
cd correction
/home/hal9000/soft/canu-1.7/Linux-amd64/bin/ovStoreBuild \
-O ./X.ovlStore.BUILDING \
-G ./X.gkpStore \
-M 4-32 \
-L ./1-overlapper/ovljob.files \
> ./X.ovlStore.err 2>&1
-- Finished on Fri Apr 13 12:50:37 2018 (3505 seconds) with 170.359 GB free disk space
----------------------------------------
-- Checking store.
----------------------------------------
-- Starting command on Fri Apr 13 12:50:37 2018 with 170.359 GB free disk space
cd correction
/home/hal9000/soft/canu-1.7/Linux-amd64/bin/ovStoreDump \
-G ./X.gkpStore \
-O ./X.ovlStore \
-d -counts \
> ./X.ovlStore/counts.dat 2> ./X.ovlStore/counts.err
-- Finished on Fri Apr 13 12:50:37 2018 (lickety-split) with 170.353 GB free disk space
----------------------------------------
--
-- Overlap store 'correction/X.ovlStore' successfully constructed.
-- Found 4988296752 overlaps for 541682 reads; 26 reads have no overlaps.
--
--
-- Purged 135.787 GB in 152 overlap output files.
-- Global filter scores will be estimated.
-- Computing correction layouts.
----------------------------------------
-- Starting command on Fri Apr 13 12:50:40 2018 with 226.917 GB free disk space
cd correction
/home/hal9000/soft/canu-1.7/Linux-amd64/bin/generateCorrectionLayouts \
-G ./X.gkpStore \
-O ./X.ovlStore \
-C ./X.corStore.WORKING \
-ec 0 \
-eC 80 \
> ./X.corStore.err 2>&1
-- Finished on Fri Apr 13 13:00:01 2018 (561 seconds) with 226.008 GB free disk space
----------------------------------------
-- Computing correction layouts.
----------------------------------------
-- Starting command on Fri Apr 13 13:00:01 2018 with 226.008 GB free disk space
cd correction/2-correction
/home/hal9000/soft/canu-1.7/Linux-amd64/bin/filterCorrectionLayouts \
-G ../X.gkpStore \
-C ../X.corStore \
-R ./X.readsToCorrect.WORKING \
-cc 0 \
-cl 1000 \
-g 1000000000 \
-c 40 \
> ./X.readsToCorrect.err 2>&1
-- Finished on Fri Apr 13 13:00:06 2018 (5 seconds) with 225.968 GB free disk space
----------------------------------------
-- original original
-- raw reads raw reads
-- category w/overlaps w/o/overlaps
-- -------------------- ------------- -------------
-- Number of Reads 535055 6653
-- Number of Bases 5225314893 24579045
-- Coverage 5.225 0.025
-- Median 7362 2442
-- Mean 9765 3694
-- N50 14876 5312
-- Minimum 1000 0
-- Maximum 126774 54531
--
-- --------corrected--------- ----------rescued----------
-- evidence expected expected
-- category reads raw corrected raw corrected
-- -------------------- ------------- ------------- ------------- ------------- -------------
-- Number of Reads 541682 535055 535055 0 0
-- Number of Bases 5249893938 5225314893 4955911548 0 0
-- Coverage 5.250 5.225 4.956 0.000 0.000
-- Median 7269 7362 6925 0 0
-- Mean 9691 9765 9262 0 0
-- N50 14824 14876 14737 0 0
-- Minimum 1000 1000 2 0 0
-- Maximum 126774 126774 120177 0 0
--
-- --------uncorrected--------
-- expected
-- category raw corrected
-- -------------------- ------------- -------------
-- Number of Reads 6653 6653
-- Number of Bases 24579045 0
-- Coverage 0.025 0.000
-- Median 2442 0
-- Mean 3694 0
-- N50 5312 0
-- Minimum 0 0
-- Maximum 54531 0
--
-- Maximum Memory 1669058660
--
-- (tag)Concurrency
-- (tag)Threads |
-- (tag)Memory | |
-- (tag) | | | total usage algorithm
-- ------- ------ -------- -------- ----------------- -----------------------------
-- Local: cor 6 GB 4 CPUs x 1 job 6 GB 4 CPUs (read correction)
--
-- Using overlaps no worse than 0.5 fraction error for correcting reads (from corErrorRate parameter).
--
-- Running jobs. First attempt out of 2.
----------------------------------------
-- Starting 'cor' concurrent execution on Fri Apr 13 13:00:06 2018 with 225.968 GB free disk space (37 processes; 1 concurrently)
cd correction/2-correction
./correctReads.sh 1 > ./correctReads.000001.out 2>&1
./correctReads.sh 2 > ./correctReads.000002.out 2>&1
./correctReads.sh 3 > ./correctReads.000003.out 2>&1
./correctReads.sh 4 > ./correctReads.000004.out 2>&1
./correctReads.sh 5 > ./correctReads.000005.out 2>&1
./correctReads.sh 6 > ./correctReads.000006.out 2>&1
./correctReads.sh 7 > ./correctReads.000007.out 2>&1
./correctReads.sh 8 > ./correctReads.000008.out 2>&1
./correctReads.sh 9 > ./correctReads.000009.out 2>&1
./correctReads.sh 10 > ./correctReads.000010.out 2>&1
./correctReads.sh 11 > ./correctReads.000011.out 2>&1
./correctReads.sh 12 > ./correctReads.000012.out 2>&1
./correctReads.sh 13 > ./correctReads.000013.out 2>&1
./correctReads.sh 14 > ./correctReads.000014.out 2>&1
./correctReads.sh 15 > ./correctReads.000015.out 2>&1
./correctReads.sh 16 > ./correctReads.000016.out 2>&1
./correctReads.sh 17 > ./correctReads.000017.out 2>&1
./correctReads.sh 18 > ./correctReads.000018.out 2>&1
./correctReads.sh 19 > ./correctReads.000019.out 2>&1
./correctReads.sh 20 > ./correctReads.000020.out 2>&1
./correctReads.sh 21 > ./correctReads.000021.out 2>&1
./correctReads.sh 22 > ./correctReads.000022.out 2>&1
./correctReads.sh 23 > ./correctReads.000023.out 2>&1
./correctReads.sh 24 > ./correctReads.000024.out 2>&1
./correctReads.sh 25 > ./correctReads.000025.out 2>&1
./correctReads.sh 26 > ./correctReads.000026.out 2>&1
./correctReads.sh 27 > ./correctReads.000027.out 2>&1
./correctReads.sh 28 > ./correctReads.000028.out 2>&1
./correctReads.sh 29 > ./correctReads.000029.out 2>&1
./correctReads.sh 30 > ./correctReads.000030.out 2>&1
./correctReads.sh 31 > ./correctReads.000031.out 2>&1
./correctReads.sh 32 > ./correctReads.000032.out 2>&1
./correctReads.sh 33 > ./correctReads.000033.out 2>&1
./correctReads.sh 34 > ./correctReads.000034.out 2>&1
./correctReads.sh 35 > ./correctReads.000035.out 2>&1
./correctReads.sh 36 > ./correctReads.000036.out 2>&1
./correctReads.sh 37 > ./correctReads.000037.out 2>&1
-- Finished on Sat Apr 14 07:18:50 2018 (65924 seconds) with 211.161 GB free disk space
----------------------------------------
-- Found 37 read correction output files.
-- Found 37 read correction output files.
--
-- Loading corrected reads into corStore and gkpStore.
----------------------------------------
-- Starting command on Sat Apr 14 07:18:50 2018 with 211.161 GB free disk space
cd correction
/home/hal9000/soft/canu-1.7/Linux-amd64/bin/loadCorrectedReads \
-G ./X.gkpStore \
-C ./X.corStore \
-L ./2-correction/corjob.files \
> ./2-correction/corjob.files.load.err 2>&1
-- Finished on Sat Apr 14 07:19:19 2018 (29 seconds) with 199.647 GB free disk space
----------------------------------------
--
-- In gatekeeper store './X.gkpStore':
-- Found 535018 reads.
-- Found 4559772904 bases (4.55 times coverage).
--
-- Read length histogram (one '*' equals 3355.2 reads):
-- 0 4999 234864 **********************************************************************
-- 5000 9999 136492 ****************************************
-- 10000 14999 73343 *********************
-- 15000 19999 41969 ************
-- 20000 24999 23530 *******
-- 25000 29999 12822 ***
-- 30000 34999 6533 *
-- 35000 39999 2914
-- 40000 44999 1365
-- 45000 49999 609
-- 50000 54999 285
-- 55000 59999 139
-- 60000 64999 74
-- 65000 69999 36
-- 70000 74999 18
-- 75000 79999 9
-- 80000 84999 6
-- 85000 89999 2
-- 90000 94999 5
-- 95000 99999 1
-- 100000 104999 0
-- 105000 109999 0
-- 110000 114999 1
-- 115000 119999 0
-- 120000 124999 0
-- 125000 129999 1
--
-- Purging correctReads output after loading into stores.
-- Purged 37 .cns outputs.
-- Purged 74 .out job log outputs.
--
-- Purging overlaps used for correction.
----------------------------------------
-- Starting command on Sat Apr 14 07:19:29 2018 with 307.44 GB free disk space
cd .
/home/hal9000/soft/canu-1.7/Linux-amd64/bin/gatekeeperDumpFASTQ \
-corrected \
-G ./X.gkpStore \
-o ./X.correctedReads.gz \
-fasta \
-nolibname \
> X.correctedReads.fasta.err 2>&1
-- Finished on Sat Apr 14 07:20:33 2018 (64 seconds) with 306.012 GB free disk space
----------------------------------------
--
-- Corrected reads saved in 'X.correctedReads.fasta.gz'.
--
--
-- BEGIN TRIMMING
--
--
-- Running jobs. First attempt out of 2.
----------------------------------------
-- Starting 'meryl' concurrent execution on Sat Apr 14 07:20:33 2018 with 306.012 GB free disk space (1 processes; 1 concurrently)
cd trimming/0-mercounts
./meryl.sh 1 > ./meryl.000001.out 2>&1
-- Finished on Sat Apr 14 07:29:44 2018 (551 seconds) with 290.572 GB free disk space
----------------------------------------
-- Meryl finished successfully.
--
-- 22-mers Fraction
-- Occurrences NumMers Unique Total
-- 1- 1 2582982711 *******************************************************************--> 0.8856 0.5679
-- 2- 2 167080701 ********************************************************************** 0.9429 0.6413
-- 3- 4 99522783 ***************************************** 0.9659 0.6856
-- 5- 7 35286752 ************** 0.9831 0.7336
-- 8- 11 13201314 ***** 0.9907 0.7666
-- 12- 16 6776641 ** 0.9942 0.7897
-- 17- 22 3910935 * 0.9962 0.8085
-- 23- 29 2329621 0.9974 0.8239
-- 30- 37 1425759 0.9982 0.8364
-- 38- 46 914097 0.9986 0.8463
-- 47- 56 624079 0.9989 0.8544
-- 57- 67 457920 0.9991 0.8612
-- 68- 79 347886 0.9993 0.8673
-- 80- 92 270279 0.9994 0.8729
-- 93- 106 212779 0.9995 0.8779
-- 107- 121 176969 0.9995 0.8825
-- 122- 137 146670 0.9996 0.8869
-- 138- 154 124392 0.9996 0.8910
-- 155- 172 105280 0.9997 0.8950
-- 173- 191 91464 0.9997 0.8987
-- 192- 211 77325 0.9998 0.9023
-- 212- 232 66610 0.9998 0.9057
-- 233- 254 58141 0.9998 0.9090
-- 255- 277 50711 0.9998 0.9121
-- 278- 301 45460 0.9998 0.9150
-- 302- 326 39630 0.9999 0.9179
-- 327- 352 35278 0.9999 0.9206
-- 353- 379 31436 0.9999 0.9232
-- 380- 407 27950 0.9999 0.9258
-- 408- 436 24953 0.9999 0.9282
-- 437- 466 22278 0.9999 0.9305
-- 467- 497 19286 0.9999 0.9327
-- 498- 529 17334 0.9999 0.9347
-- 530- 562 15607 0.9999 0.9367
-- 563- 596 14064 0.9999 0.9385
-- 597- 631 13320 0.9999 0.9403
-- 632- 667 11878 0.9999 0.9421
-- 668- 704 11401 0.9999 0.9438
-- 705- 742 10340 1.0000 0.9455
-- 743- 781 8713 1.0000 0.9472
-- 782- 821 8034 1.0000 0.9486
--
-- 174679 (max occurrences)
-- 1965554828 (total mers, non-unique)
-- 333728948 (distinct mers, non-unique)
-- 2582982711 (unique mers)
----------------------------------------
-- Starting command on Sat Apr 14 07:29:44 2018 with 290.572 GB free disk space
cd trimming/0-mercounts
/home/hal9000/soft/canu-1.7/Linux-amd64/bin/meryl \
-Dt \
-n 1045 \
-s ./X.ms22 \
> ./X.ms22.frequentMers.fasta \
2> ./X.ms22.frequentMers.fasta.err
-- Finished on Sat Apr 14 07:30:21 2018 (37 seconds) with 290.57 GB free disk space
----------------------------------------
-- Reset obtOvlMerThreshold from auto to 1045.
--
-- Found 4548537539 22-mers; 2916711659 distinct and 2582982711 unique. Largest count 174679.
--
-- OVERLAPPER (normal) (trimming) erate=0.144
--
----------------------------------------
-- Starting command on Sat Apr 14 07:30:22 2018 with 306.009 GB free disk space
cd trimming/1-overlapper
/home/hal9000/soft/canu-1.7/Linux-amd64/bin/overlapInCorePartition \
-G ../X.gkpStore \
-hl 341248000 \
-rl 15000000000 \
-ol 500 \
-o ./X.partition \
> ./X.partition.err 2>&1
-- Finished on Sat Apr 14 07:30:22 2018 (lickety-split) with 306.009 GB free disk space
----------------------------------------
--
-- Configured 14 overlapInCore jobs.
--
-- Running jobs. First attempt out of 2.
----------------------------------------
-- Starting 'obtovl' concurrent execution on Sat Apr 14 07:30:22 2018 with 306.009 GB free disk space (14 processes; 1 concurrently)
cd trimming/1-overlapper
./overlap.sh 1 > ./overlap.000001.out 2>&1
./overlap.sh 2 > ./overlap.000002.out 2>&1
./overlap.sh 3 > ./overlap.000003.out 2>&1
./overlap.sh 4 > ./overlap.000004.out 2>&1
./overlap.sh 5 > ./overlap.000005.out 2>&1
./overlap.sh 6 > ./overlap.000006.out 2>&1
./overlap.sh 7 > ./overlap.000007.out 2>&1
./overlap.sh 8 > ./overlap.000008.out 2>&1
./overlap.sh 9 > ./overlap.000009.out 2>&1
./overlap.sh 10 > ./overlap.000010.out 2>&1
./overlap.sh 11 > ./overlap.000011.out 2>&1
./overlap.sh 12 > ./overlap.000012.out 2>&1
./overlap.sh 13 > ./overlap.000013.out 2>&1
./overlap.sh 14 > ./overlap.000014.out 2>&1
-- Finished on Mon Apr 16 10:09:20 2018 (182338 seconds) with 300.914 GB free disk space
----------------------------------------
-- Found 14 overlapInCore output files.
--
-- overlapInCore compute 'trimming/1-overlapper':
-- kmer hits
-- with no overlap 3851224378 455.571429 +- 160476820.9
-- with an overlap 254047958 82.7142857 +- 10075741.865
--
-- overlaps 254039585 84.6428571 +- 10075394.61
-- contained 0 0 +- 0
-- dovetail 0 0 +- 0
--
-- overlaps rejected
-- multiple per pair 0 0 +- 0
-- bad short window 0 0 +- 0
-- bad long window 0 0 +- 0
----------------------------------------
-- Starting command on Mon Apr 16 10:09:20 2018 with 300.914 GB free disk space
cd trimming
/home/hal9000/soft/canu-1.7/Linux-amd64/bin/ovStoreBuild \
-O ./X.ovlStore.BUILDING \
-G ./X.gkpStore \
-M 4-32 \
-L ./1-overlapper/ovljob.files \
> ./X.ovlStore.err 2>&1
-- Finished on Mon Apr 16 10:14:23 2018 (303 seconds) with 291.402 GB free disk space
----------------------------------------
-- Checking store.
----------------------------------------
-- Starting command on Mon Apr 16 10:14:23 2018 with 291.402 GB free disk space
cd trimming
/home/hal9000/soft/canu-1.7/Linux-amd64/bin/ovStoreDump \
-G ./X.gkpStore \
-O ./X.ovlStore \
-d -counts \
> ./X.ovlStore/counts.dat 2> ./X.ovlStore/counts.err
-- Finished on Mon Apr 16 10:14:23 2018 (lickety-split) with 291.397 GB free disk space
----------------------------------------
--
-- Overlap store 'trimming/X.ovlStore' successfully constructed.
-- Found 508095916 overlaps for 444950 reads; 96758 reads have no overlaps.
--
--
-- Purged 5.094 GB in 42 overlap output files.
----------------------------------------
-- Starting command on Mon Apr 16 10:14:24 2018 with 296.497 GB free disk space
cd trimming/3-overlapbasedtrimming
/home/hal9000/soft/canu-1.7/Linux-amd64/bin/trimReads \
-G ../X.gkpStore \
-O ../X.ovlStore \
-Co ./X.1.trimReads.clear \
-e 0.144 \
-minlength 1000 \
-ol 1 \
-oc 1 \
-o ./X.1.trimReads \
> ./X.1.trimReads.err 2>&1
-- Finished on Mon Apr 16 10:15:53 2018 (89 seconds) with 296.456 GB free disk space
----------------------------------------
-- PARAMETERS:
-- ----------
-- 1000 (reads trimmed below this many bases are deleted)
-- 0.1440 (use overlaps at or below this fraction error)
-- 1 (break region if overlap is less than this long, for 'largest covered' algorithm)
-- 1 (break region if overlap coverage is less than this many read, for 'largest covered' algorithm)
--
-- INPUT READS:
-- -----------
-- 541708 reads 4559772904 bases (reads processed)
-- 0 reads 0 bases (reads not processed, previously deleted)
-- 0 reads 0 bases (reads not processed, in a library where trimming isn't allowed)
--
-- OUTPUT READS:
-- ------------
-- 344659 reads 1561521903 bases (trimmed reads output)
-- 23075 reads 80539642 bases (reads with no change, kept as is)
-- 96758 reads 346770244 bases (reads with no overlaps, deleted)
-- 77216 reads 394900211 bases (reads with short trimmed length, deleted)
--
-- TRIMMING DETAILS:
-- ----------------
-- 306653 reads 1036737604 bases (bases trimmed from the 5' end of a read)
-- 332221 reads 1139303300 bases (bases trimmed from the 3' end of a read)
----------------------------------------
-- Starting command on Mon Apr 16 10:15:53 2018 with 296.456 GB free disk space
cd trimming/3-overlapbasedtrimming
/home/hal9000/soft/canu-1.7/Linux-amd64/bin/splitReads \
-G ../X.gkpStore \
-O ../X.ovlStore \
-Ci ./X.1.trimReads.clear \
-Co ./X.2.splitReads.clear \
-e 0.144 \
-minlength 1000 \
-o ./X.2.splitReads \
> ./X.2.splitReads.err 2>&1
-- Finished on Mon Apr 16 10:17:45 2018 (112 seconds) with 296.452 GB free disk space
----------------------------------------
-- PARAMETERS:
-- ----------
-- 1000 (reads trimmed below this many bases are deleted)
-- 0.1440 (use overlaps at or below this fraction error)
-- INPUT READS:
-- -----------
-- 367734 reads 3818102449 bases (reads processed)
-- 173974 reads 741670455 bases (reads not processed, previously deleted)
-- 0 reads 0 bases (reads not processed, in a library where trimming isn't allowed)
--
-- PROCESSED:
-- --------
-- 0 reads 0 bases (no overlaps)
-- 8559 reads 65241370 bases (no coverage after adjusting for trimming done already)
-- 0 reads 0 bases (processed for chimera)
-- 0 reads 0 bases (processed for spur)
-- 359175 reads 3752861079 bases (processed for subreads)
--
-- READS WITH SIGNALS:
-- ------------------
-- 0 reads 0 signals (number of 5' spur signal)
-- 0 reads 0 signals (number of 3' spur signal)
-- 0 reads 0 signals (number of chimera signal)
-- 1162 reads 1188 signals (number of subread signal)
--
-- SIGNALS:
-- -------
-- 0 reads 0 bases (size of 5' spur signal)
-- 0 reads 0 bases (size of 3' spur signal)
-- 0 reads 0 bases (size of chimera signal)
-- 1188 reads 525123 bases (size of subread signal)
--
-- TRIMMING:
-- --------
-- 557 reads 1222432 bases (trimmed from the 5' end of the read)
-- 613 reads 1417511 bases (trimmed from the 3' end of the read)
----------------------------------------
-- Starting command on Mon Apr 16 10:17:45 2018 with 296.452 GB free disk space
cd trimming/3-overlapbasedtrimming
/home/hal9000/soft/canu-1.7/Linux-amd64/bin/loadTrimmedReads \
-G ../X.gkpStore \
-c ./X.2.splitReads.clear \
> ./X.loadtrimmedReads.err 2>&1
-- Finished on Mon Apr 16 10:17:45 2018 (lickety-split) with 296.452 GB free disk space
----------------------------------------
--
-- In gatekeeper store './X.gkpStore':
-- Found 367639 reads.
-- Found 1639346059 bases (1.63 times coverage).
--
-- Read length histogram (one '*' equals 1606.47 reads):
-- 0 999 0
-- 1000 1999 112453 **********************************************************************
-- 2000 2999 70814 ********************************************
-- 3000 3999 48740 ******************************
-- 4000 4999 32532 ********************
-- 5000 5999 22956 **************
-- 6000 6999 16387 **********
-- 7000 7999 12466 *******
-- 8000 8999 9487 *****
-- 9000 9999 7578 ****
-- 10000 10999 6371 ***
-- 11000 11999 5073 ***
-- 12000 12999 4110 **
-- 13000 13999 3318 **
-- 14000 14999 2857 *
-- 15000 15999 2257 *
-- 16000 16999 1852 *
-- 17000 17999 1575
-- 18000 18999 1220
-- 19000 19999 1014
-- 20000 20999 870
-- 21000 21999 691
-- 22000 22999 563
-- 23000 23999 431
-- 24000 24999 373
-- 25000 25999 329
-- 26000 26999 243
-- 27000 27999 205
-- 28000 28999 188
-- 29000 29999 123
-- 30000 30999 112
-- 31000 31999 90
-- 32000 32999 75
-- 33000 33999 54
-- 34000 34999 37
-- 35000 35999 35
-- 36000 36999 27
-- 37000 37999 28
-- 38000 38999 29
-- 39000 39999 14
-- 40000 40999 9
-- 41000 41999 11
-- 42000 42999 11
-- 43000 43999 4
-- 44000 44999 6
-- 45000 45999 3
-- 46000 46999 7
-- 47000 47999 2
-- 48000 48999 2
-- 49000 49999 2
-- 50000 50999 1
-- 51000 51999 1
-- 52000 52999 0
-- 53000 53999 0
-- 54000 54999 0
-- 55000 55999 0
-- 56000 56999 2
-- 57000 57999 0
-- 58000 58999 0
-- 59000 59999 1
--
-- Overlaps used for trimming saved.
----------------------------------------
-- Starting command on Mon Apr 16 10:17:49 2018 with 296.45 GB free disk space
cd .
/home/hal9000/soft/canu-1.7/Linux-amd64/bin/gatekeeperDumpFASTQ \
-trimmed \
-G ./X.gkpStore \
-o ./X.trimmedReads.gz \
-fasta \
-nolibname \
> ./X.trimmedReads.fasta.err 2>&1
-- Finished on Mon Apr 16 10:18:12 2018 (23 seconds) with 295.927 GB free disk space
----------------------------------------
--
-- Trimmed reads saved in 'X.trimmedReads.fasta.gz'.
--
--
-- BEGIN ASSEMBLY
--
--
-- Running jobs. First attempt out of 2.
----------------------------------------
-- Starting 'meryl' concurrent execution on Mon Apr 16 10:18:12 2018 with 295.927 GB free disk space (1 processes; 1 concurrently)
cd unitigging/0-mercounts
./meryl.sh 1 > ./meryl.000001.out 2>&1
-- Finished on Mon Apr 16 10:21:51 2018 (219 seconds) with 291.889 GB free disk space
----------------------------------------
-- Meryl finished successfully.
--
-- 22-mers Fraction
-- Occurrences NumMers Unique Total
-- 1- 1 579416226 *******************************************************************--> 0.7664 0.3551
-- 2- 2 81460796 ********************************************************************** 0.8741 0.4550
-- 3- 4 57861153 ************************************************* 0.9258 0.5268
-- 5- 7 19940760 ***************** 0.9641 0.6040
-- 8- 11 7312439 ****** 0.9805 0.6552
-- 12- 16 3787728 *** 0.9880 0.6910
-- 17- 22 2124025 * 0.9923 0.7202
-- 23- 29 1203745 * 0.9948 0.7434
-- 30- 37 726497 0.9962 0.7612
-- 38- 46 470397 0.9971 0.7753
-- 47- 56 321515 0.9977 0.7869
-- 57- 67 238769 0.9981 0.7968
-- 68- 79 184370 0.9984 0.8056
-- 80- 92 145426 0.9987 0.8138
-- 93- 106 119817 0.9989 0.8213
-- 107- 121 101882 0.9990 0.8286
-- 122- 137 85737 0.9992 0.8356
-- 138- 154 74241 0.9993 0.8424
-- 155- 172 63258 0.9994 0.8490
-- 173- 191 51200 0.9994 0.8553
-- 192- 211 42861 0.9995 0.8609
-- 212- 232 37288 0.9996 0.8662
-- 233- 254 32703 0.9996 0.8712
-- 255- 277 28710 0.9997 0.8761
-- 278- 301 24508 0.9997 0.8807
-- 302- 326 20097 0.9997 0.8850
-- 327- 352 17902 0.9998 0.8889
-- 353- 379 16227 0.9998 0.8926
-- 380- 407 14122 0.9998 0.8962
-- 408- 436 12337 0.9998 0.8996
-- 437- 466 10744 0.9998 0.9028
-- 467- 497 9433 0.9998 0.9057
-- 498- 529 8197 0.9999 0.9085
-- 530- 562 7574 0.9999 0.9111
-- 563- 596 6316 0.9999 0.9136
-- 597- 631 5443 0.9999 0.9158
-- 632- 667 4675 0.9999 0.9179
-- 668- 704 4138 0.9999 0.9197
-- 705- 742 3931 0.9999 0.9215
-- 743- 781 3826 0.9999 0.9232
-- 782- 821 3508 0.9999 0.9250
--
-- 72567 (max occurrences)
-- 1052209414 (total mers, non-unique)
-- 176646505 (distinct mers, non-unique)
-- 579416226 (unique mers)
----------------------------------------
-- Starting command on Mon Apr 16 10:21:51 2018 with 291.889 GB free disk space
cd unitigging/0-mercounts
/home/hal9000/soft/canu-1.7/Linux-amd64/bin/meryl \
-Dt \
-n 1055 \
-s ./X.ms22 \
> ./X.ms22.frequentMers.fasta \
2> ./X.ms22.frequentMers.fasta.err
-- Finished on Mon Apr 16 10:22:03 2018 (12 seconds) with 291.888 GB free disk space
----------------------------------------
-- Reset utgOvlMerThreshold from auto to 1055.
--
-- Found 1631625640 22-mers; 756062731 distinct and 579416226 unique. Largest count 72567.
--
-- OVERLAPPER (normal) (assembly) erate=0.144
--
----------------------------------------
-- Starting command on Mon Apr 16 10:22:04 2018 with 295.925 GB free disk space
cd unitigging/1-overlapper
/home/hal9000/soft/canu-1.7/Linux-amd64/bin/overlapInCorePartition \
-G ../X.gkpStore \
-hl 341248000 \
-rl 15000000000 \
-ol 500 \
-o ./X.partition \
> ./X.partition.err 2>&1
-- Finished on Mon Apr 16 10:22:04 2018 (lickety-split) with 295.925 GB free disk space
----------------------------------------
--
-- Configured 5 overlapInCore jobs.
--
-- Running jobs. First attempt out of 2.
----------------------------------------
-- Starting 'utgovl' concurrent execution on Mon Apr 16 10:22:04 2018 with 295.925 GB free disk space (5 processes; 1 concurrently)
cd unitigging/1-overlapper
./overlap.sh 1 > ./overlap.000001.out 2>&1
./overlap.sh 2 > ./overlap.000002.out 2>&1
./overlap.sh 3 > ./overlap.000003.out 2>&1
./overlap.sh 4 > ./overlap.000004.out 2>&1
./overlap.sh 5 > ./overlap.000005.out 2>&1
-- Finished on Wed Apr 18 07:20:24 2018 (161900 seconds) with 294.66 GB free disk space
----------------------------------------
-- Found 5 overlapInCore output files.
--
-- overlapInCore compute 'unitigging/1-overlapper':
-- kmer hits
-- with no overlap 789660588 57932117.6 +- 87655117.084
-- with an overlap 75693357 15138671.4 +- 8118637.791
--
-- overlaps 75693357 15138671.4 +- 8118637.791
-- contained 35464854 7092970.8 +- 3824192.859
-- dovetail 40228503 8045700.6 +- 4295090.15
--
-- overlaps rejected
-- multiple per pair 0 0 +- 0
-- bad short window 0 0 +- 0
-- bad long window 0 0 +- 0
----------------------------------------
-- Starting command on Wed Apr 18 07:20:24 2018 with 294.66 GB free disk space
cd unitigging
/home/hal9000/soft/canu-1.7/Linux-amd64/bin/ovStoreBuild \
-O ./X.ovlStore.BUILDING \
-G ./X.gkpStore \
-M 4-32 \
-L ./1-overlapper/ovljob.files \
> ./X.ovlStore.err 2>&1
-- Finished on Wed Apr 18 07:21:44 2018 (80 seconds) with 291.794 GB free disk space
----------------------------------------
-- Checking store.
----------------------------------------
-- Starting command on Wed Apr 18 07:21:44 2018 with 291.794 GB free disk space
cd unitigging
/home/hal9000/soft/canu-1.7/Linux-amd64/bin/ovStoreDump \
-G ./X.gkpStore \
-O ./X.ovlStore \
-d -counts \
> ./X.ovlStore/counts.dat 2> ./X.ovlStore/counts.err
-- Finished on Wed Apr 18 07:21:44 2018 (lickety-split) with 291.789 GB free disk space
----------------------------------------
--
-- Overlap store 'unitigging/X.ovlStore' successfully constructed.
-- Found 151386714 overlaps for 357130 reads; 184578 reads have no overlaps.
--
--
-- Purged 1.265 GB in 15 overlap output files.
----------------------------------------
-- Starting command on Wed Apr 18 07:21:44 2018 with 293.059 GB free disk space
cd unitigging
/home/hal9000/soft/canu-1.7/Linux-amd64/bin/ovStoreStats \
-G ./X.gkpStore \
-O ./X.ovlStore \
-C 1 \
-o ./X.ovlStore \
> ./X.ovlStore.summary.err 2>&1
-- Finished on Wed Apr 18 07:22:06 2018 (22 seconds) with 293.05 GB free disk space
----------------------------------------
ERROR:
ERROR: Failed with exit code 139. (rc=35584)
ERROR:
--
-- WARNING: failed to generate statistics for the overlap store; no summary will appear in report.
--
----------------------------------------
--
-- Loading read lengths.
-- Loading number of overlaps per read.
--
-- Configure RED for 8gb memory.
-- Batches of at most (unlimited) reads.
-- 500000000 bases.
-- Expecting evidence of at most 336753168 bases per iteration.
--
-- Total Reads Olaps Evidence
-- Job Memory Read Range Reads Bases Memory Olaps Memory Memory (Memory in MB)
-- ---- -------- ------------------- --------- ------------ -------- ------------ -------- --------
-- 1 8192.03 1-138688 96822 438117109 5016.90 42364558 484.82 642.31
-- 2 8192.11 138689-270320 95245 439516189 5032.86 40976622 468.94 642.31
-- 3 8192.01 270321-423126 98936 442165506 5063.30 38308301 438.40 642.31
-- 4 6689.96 423127-541708 76636 319547255 3659.34 29737233 340.32 642.31
-- ---- -------- ------------------- --------- ------------ -------- ------------ -------- --------
-- 1639346059 151386714
--
-- Running jobs. First attempt out of 2.
----------------------------------------
-- Starting 'red' concurrent execution on Wed Apr 18 07:22:10 2018 with 293.05 GB free disk space (4 processes; 2 concurrently)
cd unitigging/3-overlapErrorAdjustment
./red.sh 1 > ./red.000001.out 2>&1
./red.sh 2 > ./red.000002.out 2>&1
./red.sh 3 > ./red.000003.out 2>&1
./red.sh 4 > ./red.000004.out 2>&1
-- Finished on Wed Apr 18 09:13:57 2018 (6707 seconds) with 292.978 GB free disk space
----------------------------------------
-- Found 4 read error detection output files.
--
-- Loading read lengths.
-- Loading number of overlaps per read.
--
-- Configure OEA for 4gb memory.
-- Batches of at most (unlimited) reads.
-- 300000000 bases.
--
-- Total Reads Olaps Adjusts
-- Job Memory Read Range Reads Bases Memory Olaps Memory Memory (Memory in MB)
-- ---- -------- ------------------- --------- ------------ -------- ------------ -------- --------
-- 1 3744.75 1-98278 67049 300005450 296.15 29051658 886.59 514.01
-- 2 3731.63 98279-188103 65125 300001462 296.09 28623556 873.52 514.01
-- 3 3705.53 188104-276885 64712 300003813 296.08 27768665 847.43 514.01
-- 4 3643.20 276886-380247 66512 300002845 296.13 25724694 785.06 514.01
-- 5 3696.35 380248-488962 70811 300002091 296.27 27461820 838.07 514.01
-- 6 3093.20 488963-541708 33430 139330398 141.90 12756321 389.29 514.01
-- ---- -------- ------------------- --------- ------------ -------- ------------ -------- --------
-- 1639346059 151386714
--
-- Running jobs. First attempt out of 2.
----------------------------------------
-- Starting 'oea' concurrent execution on Wed Apr 18 09:14:01 2018 with 292.978 GB free disk space (6 processes; 6 concurrently)
cd unitigging/3-overlapErrorAdjustment
./oea.sh 1 > ./oea.000001.out 2>&1
./oea.sh 2 > ./oea.000002.out 2>&1
./oea.sh 3 > ./oea.000003.out 2>&1
./oea.sh 4 > ./oea.000004.out 2>&1
./oea.sh 5 > ./oea.000005.out 2>&1
./oea.sh 6 > ./oea.000006.out 2>&1
-- Finished on Wed Apr 18 10:57:34 2018 (6213 seconds) with 292.696 GB free disk space
----------------------------------------
-- Found 6 overlap error adjustment output files.
----------------------------------------
-- Starting command on Wed Apr 18 10:57:34 2018 with 292.696 GB free disk space
cd unitigging/3-overlapErrorAdjustment
/home/hal9000/soft/canu-1.7/Linux-amd64/bin/ovStoreBuild \
-G ../X.gkpStore \
-O ../X.ovlStore \
-evalues \
-L ./oea.files \
> ./oea.apply.err 2>&1
-- Finished on Wed Apr 18 10:57:35 2018 (1 second) with 292.414 GB free disk space
----------------------------------------
-- No report available.
--
-- Running jobs. First attempt out of 2.
----------------------------------------
-- Starting 'bat' concurrent execution on Wed Apr 18 10:57:35 2018 with 292.414 GB free disk space (1 processes; 1 concurrently)
cd unitigging/4-unitigger
./unitigger.sh 1 > ./unitigger.000001.out 2>&1
-- Finished on Wed Apr 18 10:58:26 2018 (51 seconds) with 292.385 GB free disk space
----------------------------------------
--
-- Bogart failed, retry
--
--
-- Running jobs. Second attempt out of 2.
----------------------------------------
-- Starting 'bat' concurrent execution on Wed Apr 18 10:58:26 2018 with 292.385 GB free disk space (1 processes; 1 concurrently)
cd unitigging/4-unitigger
./unitigger.sh 1 > ./unitigger.000001.out 2>&1
-- Finished on Wed Apr 18 10:59:16 2018 (50 seconds) with 292.385 GB free disk space
----------------------------------------
--
-- Bogart failed, tried 2 times, giving up.
--
ABORT:
ABORT: Canu 1.7
ABORT: Don't panic, but a mostly harmless error occurred and Canu stopped.
ABORT: Try restarting. If that doesn't work, ask for help.
ABORT:
Can you help me?
thank you
Bessem
I have a similar issue like #546 and #718
I installed canu 1.7 on an ubuntu (16.4) workstation. I have a 1G assembly length but with a very low nanopore reads coverage (5X).
This is the command I used : canu -p X -d X_canu genomeSize=1g -nanopore-raw X.fastq corMinCoverage=0 useGrid=false maxMemory=64g maxThreads=6
This is the unitigger.err log
and the log messages
Can you help me?
thank you
Bessem