Change log

Features 🏋️

• milestone#38 - Relevance Feedback (docs)
• milestone#44 - API for detailed report on optimization progress and stages (docs)
• milestone#40 - API for aggregated telemetry of the whole cluster (docs)
• milestone#43 - Unlimited update queue to gracefully smooth update spikes (docs)
• #8071 - Add Audit Access Logging (docs)
• #8063 - Add Weighted RRF (docs)
• #7643 - Add config option to control update throughput and prevent unoptimized searches (docs)
• #7929 - Add configurable read fan-out delay for dealing with tail latency in distributed clusters (docs)
• #7963 - For upserts, add update_mode parameter to either upsert, update or insert (docs)
• #7835 - Add secondary API key configuration for zero downtime key rotation in distributed clusters
• #7838 - Add dedicated HTTP port for /metrics endpoint for internal monitoring
• #7615 - Add API to list shard keys (docs)

Improvements 🤸

• #7802 - Improve timeout handling on read operations
• #7750 - Improve timeout handling in update operations, prevent shard failures in case of timed out updates after WAL
• #8025 - Recover snapshot without creating intermediate files, greatly improves recovery time and disk usage
• #8059 - Recover snapshots directly into target file system to avoid expensive file movements
• #7883 - Flush after snapshot unpack with syncfs to persist a large number of files much more efficiently
• #8072 - Don't lock shard holder structure during creation of a snapshot, previously blocking shard level operations
• #8166 - Add timeout to snapshot downloads, abort if connection gets stuck for more than a minute
• #8007, #8056 - Improve segments locking approach to minimize lock contention
• #8105 - Limit number of parallel updates on a shard to 64 to prevent order tracking overhead
• #8169 - Reduce locking in Gridstore to lower search tail latencies
• #8164 - Actively free cache memory for closed WAL segments to reduce memory pressure
• #7952 - Disable in-place payload updates on unindexed fields, improve immutability guarantees of indexed segments improving partial snapshots
• #7887 - Add ability to disable extra HNSW links construction for specific payload indices (docs)
• #7971 - Enable missing option for vector storage to populate single-file mmap
• #7928 - Enable io_uring when reading batch of vectors
• #7919 - Improve error message for datetime parse failures
• #8053 - Allow to configure load concurrency for collections, shards and segments
• #7809 - Add more convenient way to provide API-keys for external inference providers (docs)
• #8093 - Don't lock WAL during serialization of new updates, which was costly for large operations
• #7834 - Extend WAL retention when replicas are dead, prevent full shard transfers in case of peer failures
• #7565 - Disable old shard key format deprecated in 1.15.0
• #8125 - Skip building extra HNSW links for deleted vectors
• #8163 - Improve search result processing to use less CPU with a high search limit
• #8175 - Use less allocations for HNSW plain filtered search

Bug Fixes 🤹

• #7850 - Fix flush ordering to follow segment dependencies, prevents dataloss by CoW on flush interruption
• #8103 - Fix data race in stream records transfer potentially missing ongoing updates
• #7983 - Fix interlocking problem on creation of payload index
• #7999 - Fix interlocking problem on collection-level update operations
• #8131 - Fix deadlock during snapshot with concurrent updates
• #8128 - Fix gRPC/HTTP2 too_many_internal_resets error due to how we internally cancel ongoing requests
• #8019 - Improve handling of HTTP2 channels closing in connection pool
• #8104 - Fix data race in WAL and shard clocks snapshot, ensure they remain consistent
• #7961 - Fix using incorrect versions in partial snapshot manifest construction
• #8095 - Fix incorrect internal protocol usage for shard snapshot transfers
• #7950 - Fix integer overflow in query batch when using high limits
• #7972 - Fix search aggregator panic with limit 0
• #8100 - Fix round floats not used in integer index, JSON doesn't distinguish between integers and floats
• #8097 - Fix score_threshold not being used in score boosting queries
• #7877 - Fix Corrupted ID tracker mapping storage bug when disk is full
• #7944 - Fix gRPC API response status counting in telemetry and metrics
• #7857 - Fix total count in progress tracker for replicate points with filter
• #7856 - Fix creation of payload index in empty collection using user-defined sharding
• #8099 - Fix ignoring CA certs for internal requests if configured
• #8176 - Add missing timeout parameter to some endpoints

Web UI 🍱

• qdrant/qdrant-web-ui#345 - Detailed visualization of optimization progress
• qdrant/qdrant-web-ui#334 - Create collection dialog now previews the exact command
• qdrant/qdrant-web-ui#341 - Buttons for resharding control
• qdrant/qdrant-web-ui#344 - Points filter and search bar restored and improved

Qdrant Edge 🔪

Qdrant Edge is an in-process version of Qdrant, which shares the same internals, storage format, and points API as the server version, but designed to work locally. Qdrant Edge is compatible with server version and it can read shard snapshots created by server version of Qdrant. More documentation available here.

Deprecations ⚠️

• Starting from v1.17.0 Qdrant changes response format for vector fields in gRPC interface. All official Qdrant clients should be already adopted to this change, so please make sure you upgrade your client libraries and check that you are not using deprecated fields. More info: #7183

• Upcoming deprecations:

  • In Qdrant v1.18.x all deprecated search methods will be completely removed and won't be available even from old client libraries.
  • In Qdrant v1.17.x we will completely remove RocksDB support in favor of gridstore, that means that direct upgrade from v1.15.x into v1.17.x won't be possible. Please follow upgrade instructions and upgrade one minor version at a time to avoid unsupported storage errors. Note that Qdrant Cloud infrastructure automatically generates a proper upgrade steps, so you don't have to worry about that.

Change log

Improvements

• #7755, #7588 - Respect search and point retrieve timeout when trying to access segments
• #7685 - Respect telemetry timeout when fetching shard statistics
• #7715 - Log snapshot download duration and speed

Bug fixes

• #7787, #7791 - Fix WAL delta transfer corrupting replica after a previous full transfer was aborted
• #7801, #7805 - Fix flush losing changes on transient disk IO errors, potentially corrupting data
• #7792 - Fix incorrectly aborting shard transfers when dropping unrelated shard
• #7741 - Fix flush error in Gridstore, potentially corrupting data when quickly alternating inserts/deletes
• #7702 - Fix flush data race in Gridstore, potentially corrupting data when storage is cleared in parallel
• #7759, #7782 - Fix handling of collection names with weird characters, breaking snapshot transfers for example
• #7788 - Fix snapshot metrics not always reporting when zero (snapshot_{creation,recovery}_running, snapshot_created_total)
• #7783 - Fix incorrectly reporting optimization errors in telemetry on panic
• #7765 - Fix slow shutdown on SIGINT when optimizations are running
• #7690 - Fix Qdrant not building on Windows ARM64
• #7683 - Keep RocksDB support until 1.18.0 in development builds

Change log

Improvements

• #7607 - Improve request timeout handling for telemetry and metrics
• #7623 - Add user agent to HTTP requests sent by Qdrant server

Bug fixes

• #7674 - Fix critical WAL bug that could break consensus or cause data corruption on restart
• #7684 - Fix consensus crash when applying consensus snapshot with non-replicated collection
• #7620 - Fix panic during search on segments with empty HNSW graph
• #7629, #7640, #7673 - Fix shard resource cleanup when shard is replaced, prevent deadlock on small CPUs
• #7621, #7626 - Fix payload index storage still flushing after removal, fixing data corruption and IO errors on Windows
• #7624, #7627 - Fix Gridstore storage still flushing after wipe, fixing data corruption and IO errors
• #7614, #7618 - Fix Docker/WSL on Windows with bind mount corrupting storage
• #7678 - Fix collections_vector_total metric reporting -0.0 if there are no vectors
• #7649 - Also report collection_indexed_only_excluded_points metric if zero

Change log

Improvements

• #7514, #7572 - Make batch queries up to 3 times faster on full scans by reading each point only once
• #7551 - Actively migrate vector, payload and payload index storage from RocksDB into Gridstore on startup for better and more predictable performance
• #7579 - Add 60s internal timeout for telemetry/metrics endpoints to prevent long hanging tasks
• #7557 - Add validation to restart shard transfer operation
• #7446 - Defer Gridstore flushing to make flushing behavior consistent with all other storage components
• #7580 - Improve consensus WAL compaction logging to aid debugging
• #7598 - Make timeout for inference requests user configurable

Bug fixes

• #7564 - Fix panic at startup on old clusters with user defined sharding, if not updated to Qdrant 1.15.5 first
• #7577 - Fix breaking Raft by killing node at specific time during consensus snapshot, preventing potential crash loop
• #7587 - Fix corrupting WAL with broken flush edge case after WAL is cleared or truncated
• #7570 - Fix incorrect rescoring default on mutable segments when using binary quantization
• #7569, #7575 - Spawn search/update tasks on the correct runtime, significantly reduce number of general/actix threads
• #7558 - Fix data race in shard transfers, wait on transfer to start before initializing shards
• #7556 - Fix incorrect log message when failing to read lock segment for some time

Change log

Features 🌰

• https://github.com/qdrant/qdrant/milestone/33?closed=1 - Inline Storage: Add option to inline vectors in HNSW graph for efficient IO usage (docs)
• https://github.com/qdrant/qdrant/milestone/37?closed=1 - Tenant promotion mechanism for tiered multitenancy: (docs)
  • Add ReplicatePoints action to promote payload based tenant into dedicated shard key (docs)
  • Add fallback shard key for intelligent routing to tenants that are or are not promoted to a dedicated shard (docs)
  • On shard key creation, allow to specify initial state of new replicas
• #7414 - Add ACORN-1 search method, accurate search over many filtered points at the cost of performance (docs)
• #7408 - Add ASCII folding (normalization) to full text indices, fold diacritics into ASCII characters (docs)
• #7006 - Add conditional update functionality, only apply update on points matching filter (docs)
• #7100 - Add text_any full text filter to match any query term (docs)
• #7065 - Add option to customize RRF k parameter (docs)
• #7222 - In strict mode, specify maximum number of payload indices per collection
• #7123 - Add custom key-value metadata to collections (docs)
• #7291 - Add profiler to log slow point update and read requests

Improvements 🏎️

• #7385 - When loading Gridstore, populate tracker data into memory for faster first access
• #7407, #7405 - Spawn updates and flush workers task on updates runtime, use two system threads less for each local shard
• #7413 - Use system thread on demand in WAL, use one less system thread per local shard by default
• #7468 - Use atomic bit flags on HNSW construction to significantly increase indexing performance
• #7052, #7471 - Implement AVX512 SIMD optimizations for binary quantization on modern x86_64 CPUs
• #7433 - Switch new mutable payload indices and storage from RocksDB to Gridstore for better performance
• #7508 - Enable quantization in appendable segments by default, improving search performance
• #7347 - Change default score of query-less prefetch to 1.0 to ease score boosting
• #7369 - Don't explicitly disable strict mode by default
• #7345 - Simplify internal handling of copy-on-write segments, now write incoming updates to dedicated segments directly
• #7293, #7523 - Add warnings field to collection info, report misconfiguration
• #7319, #7401 - Report more helpful error messages on file IO errors
• #7377 - When using inference, propagate rate limit responses
• #7434 - Rate limit slow request warning to prevent spamming logs
• #7373 - Log shard transfers as a result of the consensus recovery procedure
• #7337 - Tone down optimizer logging to make it less verbose
• #7370 - Add TARGET_CPU and JEMALLOC_SYS_WITH_LG_PAGE build parameters to Docker image

Metrics 📈

• #7302, #7441 - In metrics, report point and vector counts per collection and vector name (collection_{points,vectors})
• #7307 - In metrics, report number of points skipped in indexed_only search (collection_indexed_only_excluded_points)
• #7301 - In metrics, report global effective minimum and maximum shard replication count (collection_active_replicas_{min,max})
• #7310, #7516 - In metrics, report total number of non-active replicas (collection_dead_replicas)
• #7316, #7480 - In metrics, expose collection_running_optimizations with number of optimizers running per collection
• #7497 - In metrics, report running, created and recovered number of snapshots (snapshot_{creation,recovery}_running, snapshot_created_total)
• #7484 - In metrics, report active thread count (process_threads)
• #7451 - In metrics, report number of open file descriptors and the limit (process_{open,max}_fds)
• #7487 - In metrics, report open mmaps and system limit (process_open_mmaps, system_max_mmaps)
• #7482 - In metrics, report total number of major and minor page faults (process_{minor,major}_page_faults_total)
• #7438 - In metrics, add configuration option to prefix all metrics with qdrant_ or something else

Bug fixes 🕵️

• #7527 - Fix logger API allowing arbitrary file writes, now this can only be configured through configuration
• #7530 - Abort pending search tasks when search is cancelled, fixing optimizer instability under huge load
• #7533 - Abort other blocking tasks such as retrieve and snapshot prematurely if the caller gets cancelled
• #7531 - Cancel ongoing searches more aggressively if the search is cancelled
• #7517 - Fix resharding down panic if no shard key is provided on collection with custom sharding
• #7469 - Fix panic on certain queries with unknown vector name
• #7375 - Forbid peer to join cluster with URI that is already used, which could break a cluster
• #7400 - Fix corrupt segments on load if segment was partially flushed, prevent payload index corruption
• #7342, #7404, #7427 - Force flush all segments when taking snapshot to prevent data corruption
• #7381 - Fix flush ordering on segments currently being snapshotted, fixing data consistency on crash
• #7388, #7416 - Fix flush ordering with concurrent flushes to ensure data consistency
• #7512, #7521 - Fix dummy shards not allowing snapshot recovery

Web UI 🫂

• https://github.com/qdrant/qdrant-web-ui/releases/tag/v0.2.0 - New web UI design to match Qdrant Cloud

Deprecations 🚧

• #7454 - Remove init_from collection API, deprecated since Qdrant 1.15
• #7449 - Remove lock API, deprecated since Qdrant 1.15
• #7047 - Remove old internal shard key format, deprecated and migrated away from in Qdrant 1.15
• #7450 - Remove payload filter from RBAC/JWT, deprecated since Qdrant 1.15, API keys using it are rejected
• #7183 - Deprecate old variant of vector output

Change log

Improvements

• #7157 - Acknowledge update/delete by filter operations on flush, preventing very slow restart
• #7217, #7218, #7219, #7220, #7221 - Add API validation to min_should, filters, point update batch and others
• #7235 - Add timeout parameter to remove peer operation
• #7320 - Decrease internal update batch sizes to minimize search latency spikes on large user batches
• #7233 - Limit number of segments loaded in parallel, preventing potential OOM on large clusters
• #7222 - Add strict mode configuration to specify max number of payload indices
• #7240 - Improve error reporting on mutable ID tracker load errors
• #7151 - Improve error reporting on flush problems
• #7244 - Remove vector count field from collection info
• #7177 - Do not anonymize peer ID in telemetry data

Bug fixes

• #7263 - Fix not flushing mutable ID tracker files after creation, potentially causing segment corruption
• #7248 - Fix data race at the end of snapshot creation causing missing points
• #7298, #7306 - Fix data race during snapshots corrupting point data if a point is moved
• #7241, #7265, #7267, #7269, #7277 - Fix potential deadlock in REST runtime while streaming shard snapshot
• #7015 - Fix potential deadlock on REST server runtime
• #7172 - Fix potential recursive deadlock when fetching all vectors
• #7203 - Fix potential deadlock in scroll API
• #7303 - Fix incorrectly deleting old point versions from segments
• #7194 - Fix upsert operations with duplicate point IDs not being applied properly
• #7252 - Fix phrase matching ignoring unknown tokens
• #7264 - Fix strict mode validation on nested filters

Change log

Improvements

• #7005 - Reduce Docker image size by 10-40%
• #7073 - Adjust metrics histogram buckets, show empty buckets and remove small ones
• #7111 - Include SBOM in Docker image
• #7119 - Sign Docker images with cosign
• #7110 - Actively migrate away from old shard key format on disk to a more robust format
• #7120 - Measure segment size on disk more reliably for improved available disk space checks

Bug fixes

• #7133 - Fix not loading some full text index types correctly, temporarily resulting in an empty index
• #7109 - Fix incorrectly reusing old deleted points in proxy segments
• #7106 - Preserve segment ID mapping when proxying all segments for a snapshot, potentially mixing up segments

Change log

Improvements

• #7002 - Optimize dot product calculation on AVX systems
• #7049 - In Nix package, use rustup for Rust version management

Bug fixes

• #7056 - Fix local BM25 not working as expected due to default parameter differences

Change log

Improvements

• #6891 - Implement BM25 inference in Qdrant locally
• #6926 - Improve performance of mutable map index, used for full text, integers and more
• #6993 - Make log buffer size adjustable when logging to a file

Bug fixes

• #6954 - Fix consistency problem in null index storage by deferring writes until flush
• #6977 - Fix consistency problem in boolean index storage by deferring writes until flush
• #6994 - Fix consistency problem for deleted vectors by deferring writes until flush
• #6966 - Fix flush logic in memory mapped bit flags structure, improving data consistency
• #6975 - Delete versionless points before WAL replay, fix incorrect use of corrupt points
• #6952 - Fix date time error in custom score boosting formula when having multiple nodes
• #6998 - In score boosting formula validation error, show each index type only once
• #6959 - Fix range bounds not being inclusive for linear decay function
• #6958 - Fix default storage path and missing web UI assets in Debian package

Web UI

• qdrant/qdrant-web-ui#299 - Show shard distribution of collection in cluster as replica/node matrix

Changelog

Improvements

• #6931 - gRPC HealthCheck method now works without authentication, making it consistent with REST methods.
• #6923 - Use populate storage components with MADV_SEQUENTIAL for better IO performance on indexing

Bug Fixes

• #6932 - Fix point shard routing broken in 1.15 (Points created in 1.15.0 in multi-shard collections might not behave properly on updates!)
• #6903 - Fix compile error in MUSL builds
• #6910 - Fix panic in /metrics on empty collection data
• #6916 - Fix UUID index storage, resulted in missing filter results.