Skip to content

Applying a different madvise after PopulateRead cancels its effect#6714

Closed
agourlay wants to merge 1 commit intodevfrom
fix-mmap-populate-read
Closed

Applying a different madvise after PopulateRead cancels its effect#6714
agourlay wants to merge 1 commit intodevfrom
fix-mmap-populate-read

Conversation

@agourlay
Copy link
Member

@agourlay agourlay commented Jun 17, 2025

Based on my manual testing, setting a different madvise right after setting PopulateRead cancels its effect.
It appears the kernel discards the pages loaded.

When using a vector storage on_disk = false with this patch:

  • The read_bytes from proc/pid/io is much higher on startup than dev
  • No page faults reported during querying using perf stat -e major-faults -p $(pidof qdrant) unlike dev

@agourlay agourlay changed the title Applying a different Advice after PopulateRead cancels its effect Applying a different madvise after PopulateRead cancels its effect Jun 17, 2025
@agourlay agourlay marked this pull request as ready for review June 17, 2025 20:31
@coderabbitai
Copy link
Contributor

coderabbitai bot commented Jun 17, 2025

📝 Walkthrough

Walkthrough

The code was updated to change when the madvise::madvise function is called within the open_read_mmap and open_write_mmap functions. Previously, madvise::madvise was called after checking the populate condition, regardless of its value. With the new logic, if populate is true, only mmap.populate() is executed; if populate is false, then madvise::madvise is called. No changes were made to the declarations of exported or public entities.

✨ Finishing Touches
  • 📝 Generate Docstrings

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Explain this complex logic.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai explain this code block.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and explain its main purpose.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Support

Need help? Create a ticket on our support page for assistance with any issues or questions.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai generate docstrings to generate docstrings for this PR.
  • @coderabbitai generate sequence diagram to generate a sequence diagram of the changes in this PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

♻️ Duplicate comments (1)
lib/common/memory/src/mmap_ops.rs (1)

71-75: Replicate the advice / error-handling fixes for the mutable mapping

Everything said for the read-only path applies to open_write_mmap as well—ignoring the caller’s advice and the populate error may cause inconsistent behaviour between read and write code paths. Consider applying the same refactor here to keep the APIs symmetrical.

🧹 Nitpick comments (1)
lib/common/memory/src/mmap_ops.rs (1)

55-56: mmap.populate() result is ignored

memmap2::Mmap::{populate, populate_mut} return io::Result<()>, yet the code drops the result.
Silent failures (e.g., ENOMEM, EIO) will go unnoticed and may lead to subtle runtime issues.

-    mmap.populate();
+    mmap.populate()?;   // propagate error to the caller

If best-effort population is acceptable, at least log the error via tracing::warn! to aid diagnostics.

Also applies to: 71-72

📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between a87a4e0 and 374c043.

📒 Files selected for processing (1)
  • lib/common/memory/src/mmap_ops.rs (2 hunks)
🧰 Additional context used
🧠 Learnings (1)
lib/common/memory/src/mmap_ops.rs (1)
Learnt from: generall
PR: qdrant/qdrant#6323
File: lib/segment/src/vector_storage/quantized/quantized_mmap_storage.rs:12-16
Timestamp: 2025-04-07T23:31:22.515Z
Learning: The `populate()` method in `QuantizedMmapStorage` correctly returns void instead of a `Result` because it directly implements the `Madviseable` trait which defines `populate(&self)` without a return type. Adding an unnecessary `Ok(())` return would trigger Clippy warnings about unnecessary Result wrapping.
⏰ Context from checks skipped due to timeout of 90000ms (6)
  • GitHub Check: test-low-resources
  • GitHub Check: Basic TLS/HTTPS tests
  • GitHub Check: test-snapshot-operations-s3-minio
  • GitHub Check: test-consensus-compose
  • GitHub Check: integration-tests-consensus
  • GitHub Check: rust-tests (windows-latest)

Comment on lines 55 to 59
if populate {
mmap.populate();
} else {
madvise::madvise(&mmap, advice.resolve())?;
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

❓ Verification inconclusive

Advice hint is silently dropped when populate == true

With the new else‐branch, any non-default AdviceSetting provided by the caller is ignored whenever populate is requested.
Call sites that relied on open_*_mmap(path, AdviceSetting::Sequential /* or Random */, true) will now receive the default kernel advice, potentially hurting read-ahead behaviour and cache locality.

A minimal fix that preserves both semantics and the intended kernel behaviour is to apply the advice before populating:

if populate {
-    mmap.populate();
+    madvise::madvise(&mmap, advice.resolve())?;
+    mmap.populate();           // MADV_WILLNEED comes last, no cancellation
} else {
    madvise::madvise(&mmap, advice.resolve())?;
}

This keeps PopulateRead effective while still honouring the caller’s hint.
Please double-check the micro-benchmarks and update the documentation/comments accordingly.


Advice hint is silently dropped when populate == true

With the new else‐branch, any non-default AdviceSetting provided by the caller is ignored whenever populate is requested.
Call sites that relied on open_*_mmap(path, AdviceSetting::Sequential /* or Random */, true) will now receive the default kernel advice, potentially hurting read-ahead behaviour and cache locality.

A minimal fix that preserves both semantics and the intended kernel behaviour is to apply the advice before populating:

if populate {
-    mmap.populate();
+    madvise::madvise(&mmap, advice.resolve())?;
+    mmap.populate();           // MADV_WILLNEED comes last, no cancellation
} else {
    madvise::madvise(&mmap, advice.resolve())?;
}

This keeps PopulateRead effective while still honouring the caller’s hint.
Please double-check the micro-benchmarks and update the documentation/comments accordingly.

🤖 Prompt for AI Agents
In lib/common/memory/src/mmap_ops.rs around lines 55 to 59, the current code
only applies the madvise advice when populate is false, causing the advice hint
to be ignored when populate is true. To fix this, call madvise with the resolved
advice before calling mmap.populate() so that the kernel advice is always
applied regardless of the populate flag. This preserves the intended read-ahead
behavior and cache locality. Also, verify micro-benchmarks and update comments
or documentation to reflect this change.

@agourlay
Copy link
Member Author

We found a different reason for the IO load discrepancy which is unrelated to PopulateRead.

I still think that our usage of PopulateRead is not bullet proof but I cannot prove it without doubt.

Closing for the time being.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant