What's Changed

Major or API updates:

• Support parsing k-mer abundances from headers by @adamant-pwn in #558
• Optimization: don't serialize the inverse label mapping by @karasikov in #561
  Note: The new version can load both old and new annotations, but annotations generated with the new version require this version or later. (Only backward compatibility is ensured. Make sure to use the latest version of metagraph.)
• Aggregate k-mer counts with --count-kmers in transform_anno by @adamant-pwn in #568
• Support mapping annotation label+coordinates to accessions by @hmusta and @karasikov in #549

Reliability improvements

• Improvements in threading, parallel integration tests by @karasikov in #538
• Optimization of single-threaded RowDiff queries by @karasikov in #551
• Workflow testing on MacOS and full parallelization of non-rectangular loops by @karasikov in #556
• Don't use range-based loops with omp by @karasikov in #557
• Server changes: load all graphs, don't compute num_relations in stats by @karasikov in #563
• a workaround for cases where the outfile is on a different filesystem by @karasikov in #578

Build and CI improvements

• Make jemalloc optional: allow compiling without it by @karasikov in #555
• Support add_subdirectory and ExternalProject_Add in third-party projects, improve CI by @adamant-pwn in #565
• Consolidate workflows with YAML anchors, put examples into repo by @adamant-pwn in #566
• Pass CMAKE_POSITION_INDEPENDENT_CODE to SDSL by @adamant-pwn in #573
• updated sdsl-lite (removed unused variable) by @karasikov in #579

Note: This release changes the label encoder format used in all annotations. Backward compatibility is preserved and this version can load annotations constructed with older versions of metagraph. However, older releases cannot load annotations created with the new format. For this reason, always use the latest version.

Full Changelog: v0.4.5...v0.5.0

What's Changed

Build and updates

• Bump pip from 23.3 to 25.2 in /metagraph/api/python by @dependabot[bot] in #533
• make sure htslib is built for the same arch by @karasikov in #534
• Updated htslib and Simple-Web-Server by @karasikov in #536
• better detection of icu4c by @karasikov in #537
• Bump sshash version by @adamant-pwn in #543
• Replace 'make check' with 'make integration_tests' by @adamant-pwn in #546
• Update sshash by @adamant-pwn in #547
• fixed compilation warnings in sdsl-lite by @karasikov in #554
• Updated Docs by @karasikov in #542
• Updated Docs by @karasikov in #548

Docker

• Updated Dockerfile by @karasikov in #535
• Also include metagraph_DNA5 in the docker image (for search with Ns) by @karasikov in #539
• removed the entrypoint from the docker container by @karasikov in #545

Server and API

• use 'top_labels' instead of 'num_labels' in metagraph-core api by @karasikov in #541
• consistently use discovery_fraction in the api by @karasikov in #544
• multi-graph server: support hosting multiple graphs in one process by @karasikov in #540
• Increased content/response timeouts in server by @karasikov in #553

Optimizations

• Optimized select_support_mcl in sdsl-lite for faster loading with mmap by @karasikov in #532

Full Changelog: v0.4.4...v0.4.5

The pre-compiled binaries (only for Linux) are statically linked and hence don't require additional libraries to be installed.
The *_noAVX binaries are compiled without the modern AVX2 and MSSE4.2 instructions and should be used when the processor does not support these instructions.

What's Changed

• Update docs by @hmusta in #516
• Warning fixes by @adamant-pwn in #514
• Fully multi-threaded to_vector by @hmusta in #517
• process query batches in parallel by @hmusta in #521
• fix timer for batch aggregate timer by @hmusta in #523
• support conversion from RowDiff to RowDiff by @karasikov in #520
• Hide progress bar when building query graph by @karasikov in #524
• store rows in small vectors by @karasikov in #525
• Support reading from zstd, switched to boost iostreams for compressed files by @hmusta in #519
• make zstd optional by @hmusta in #527
• minor compilation fixes when not using zstd by @hmusta in #528
• SSHash query fixes by @hmusta in #529
• handle canonical and primary graphs within DBGSSHash by @hmusta in #530
• Updated folly to fix small vectors, also compilation fixes by @karasikov in #531

Full Changelog: v0.4.3...v0.4.4

Update to correct the version number in package.json

What's Changed

• Update Python in workflows by @hmusta in #515

Full Changelog: v0.4.1...v0.4.2

What's Changed

• Seeding and chaining optimizations by @hmusta in #400
• Row sparse disk by @marekkokot in #420
• sd_vector_disk by @karasikov in #433
• RowFlat, RowDiff, and their in-place constructors by @karasikov in #435
• fixes for compiling on ARM and g++-12, use SIMDe as a wrapper for SIMD instructions by @hmusta in #437
• Cleaner flags for setting query regime by @karasikov in #450
• remove obsolete bin_rel_wt by @karasikov in #464
• deprecate adj-rc index, optimize and fix alignment to CanonicalDBG by @karasikov in #454
• Using SSHash submodule for graph representation by @mmarzett in #481
• user-selectable error k-mer percentile by @hmusta in #506
• Use BOSS indexing in DBGSuccinct + make RowDiff independent by @adamant-pwn in #484

New Contributors

• @marekkokot made their first contribution in #420
• @rob-p made their first contribution in #467
• @mmarzett made their first contribution in #481

Full Changelog: v0.3.6...v0.4.1

This update contains mostly reliability improvements.

Important: Besides that, it implements a new compression for the index of suffix ranges. The de Bruijn graphs constructed with older versions of MetaGraph have to be reindexed with metagraph transform -o graph_updated graph.dbg -v -p <NUM_THREADS>

What's Changed

• detect max number of open files and limit num chunks merged by @karasikov in #411
• Server now reports the max score for a query sequence by @hmusta in #414
• Use 1 file per EliasFano stream for faster construction with fewer open files by @karasikov in #415
• retry on failed write in sdsl::int_vector_buffer by @karasikov in #422
• Retry on failed read by @karasikov in #429
• Discard edges to dummy nodes in GFA assembly by @hmusta in #428
• Workaround for bug in g++-8.2 by @hmusta in #425
• Compress node ranges index (=> index longer suffixes with less memory overhead) by @karasikov in #417
• Various improvements and fixes in annotations by @karasikov in #423

Full Changelog: v0.3.5...v0.3.6

What's Changed

• Alignment to labeled subgraphs and counting DBGs
• Other improvements and fixes in alignment algorithms (chaining, seeding, etc.)
• Reduce memory usage in seed generation, seed extension improvements
• allow renaming labels in arbitrary annotations
• support annotating coordinates with disk swap
• improved usage help description for flags
• updated external libraries to support compiling with AppleClang 13.1.6

Full Changelog: v0.3.4...v0.3.5

What's Changed

• Query expressions (counts) in labels for individual k-mers in query
• Updated the MetaGraph API package
• more docs
• Support printing version of the binary
• Also pre-compile binaries without AVX2 and MSSE4.2

Full Changelog: v0.3.3...v0.3.4

The pre-compiled binaries (only for Linux) are statically linked and hence don't require additional libraries to be installed.
The *_noAVX binaries are compiled without the modern AVX2 and MSSE4.2 instructions and should be used on older processors in case normal binaries cannot be run.

What's Changed

• API support for count/coord indexes, parallel search in multiple indexes
• hide some rarely used flags from default help messages in CLI
• output counts for aggregated columns
• refactoring and cleanups in row diff annotations
• fixed default user permissions in docker containers (now does not require changing permissions or resetting the user)
• more docs

New Contributors

• @thomastzhou made their first contribution in #358

Full Changelog: v0.3.2...v0.3.3