Skip to content

Add runtime flag to disable opening multiple mmaps on the same file#7614

Merged
timvisee merged 8 commits intodevfrom
disable-multi-mmap-at-runtime
Dec 1, 2025
Merged

Add runtime flag to disable opening multiple mmaps on the same file#7614
timvisee merged 8 commits intodevfrom
disable-multi-mmap-at-runtime

Conversation

@timvisee
Copy link
Member

@timvisee timvisee commented Nov 26, 2025

Fixes #7610, #6676

Add the QDRANT_NO_MULTI_MMAP environment variable to disable multi-mmap at runtime. It disables opening multiple memory maps to the same piece of data on disk in all of Qdrant.

This pattern of opening multiple memory maps is broken in some environment, corrupting storage on disk. That's why you can now disable it on demand.

An example of such environment is Docker on Windows with a storage mount to the Windows host. This flag effectively brings us better compatibility.

Usage example:

# run without multiple mmaps for better compatibility
QDRANT_NO_MULTI_MMAP=1 ./qdrant

Once memory maps are first used a warning pops up if this feature is disabled:

# create collection with mmaps
$ bfb -n 100

# Qdrant logs now show:
2025-11-26T10:46:22.885080Z  WARN memory::mmap_ops: QDRANT_NO_MULTI_MMAP is set, you may see reduced performance

All Submissions:

  • Contributions should target the dev branch. Did you create your branch from dev?
  • Have you followed the guidelines in our Contributing document?
  • Have you checked to ensure there aren't other open Pull Requests for the same update/change?

Changes to Core Features:

  • Have you added an explanation of what your changes do and why you'd like us to include them?
  • Have you written new tests for your core changes, as applicable?
  • Have you successfully ran tests with your changes locally?

@timvisee timvisee force-pushed the disable-multi-mmap-at-runtime branch from 82b96d3 to ffb6bca Compare November 26, 2025 11:46
@timvisee timvisee marked this pull request as ready for review November 26, 2025 13:36
coderabbitai[bot]

This comment was marked as resolved.

Copy link
Contributor

@coszio coszio left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it possible to add this as part of settings? I figure it can be more cleanly deployed if this setting is part of the config file.

@timvisee
Copy link
Member Author

timvisee commented Dec 1, 2025

Is it possible to add this as part of settings? I figure it can be more cleanly deployed if this setting is part of the config file.

In this PR I didn't because I pitched this as hidden setting.

Though I have no objection in including it in config.yaml, albeit a bit tricky to implement since I'm using a static variable.

Are you okay with changing this in a separate PR?

@timvisee
Copy link
Member Author

timvisee commented Dec 1, 2025

To clarify, I definitely do not recommend to deploy with this setting. That's why I kept it hidden in the first place.

It shows that the IO behavior on Docker/WSL is broken. There may definitely be more dragons we're not aware of yet, breaking production workloads in unexpected ways.

These changes were only intended to not immediately break naive deployments on this environment.

Copy link
Contributor

@coszio coszio left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@timvisee Thanks for elaborating

@timvisee timvisee merged commit 4010e72 into dev Dec 1, 2025
15 checks passed
@timvisee timvisee deleted the disable-multi-mmap-at-runtime branch December 1, 2025 17:34
timvisee added a commit that referenced this pull request Dec 3, 2025
…7614)

* Fix delete_page not dropping sequential mmap

* In Gridstore Page, support using a single mmap

* In Gridstore Page, only open multiple mmaps if supported at runtime

* Support opening single mmap in UniversalMmapChunk

* Rename environment var, log warning when QDRANT_NO_MULTI_MMAP is set

* Support opening single mmap in MmapDenseVectors

* Remove unused result response

* Import LazyLock
@timvisee timvisee mentioned this pull request Dec 3, 2025
1 task
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants