Skip to content

Custom RRF k parameter#7065

Merged
coszio merged 9 commits intodevfrom
custom-rrf
Aug 20, 2025
Merged

Custom RRF k parameter#7065
coszio merged 9 commits intodevfrom
custom-rrf

Conversation

@coszio
Copy link
Contributor

@coszio coszio commented Aug 14, 2025

Allows the k parameter of reciprocal rank fusion formula to be provided by user, instead of just hardcoded to 2.

http example request would look like: (edited)

{
    "prefetch": [ ... ],
    "query": { "rrf": { "k": 60 } }
}

or same as before:

{
    "prefetch": [ ... ],
    "query": { "fusion":  "rrf" }
}

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 6

🧹 Nitpick comments (16)
lib/api/src/rest/schema.rs (1)

515-520: RRF k validation is not enforced at REST level; attribute here is ineffective without derive.

You added #[validate(range(min = 1))] on the RrfK(usize) variant, but Fusion does not derive Validate and REST validation is implemented manually in rest/validate.rs. As a result, this attribute does not enforce k ≥ 1 for REST input.

  • Enforce k ≥ 1 in the manual Validate impl (see suggested fix in rest/validate.rs).
  • Optionally, add a schemars constraint so OpenAPI reflects the minimum in generated docs.

Apply this diff to annotate the OpenAPI schema (purely documentation; does not enforce at runtime):

 pub enum Fusion {
     Rrf,
-    #[validate(range(min = 1))]
+    #[validate(range(min = 1))]
+    #[schemars(range(min = 1))]
     RrfK(usize),
     Dbsf,
 }

Additionally, consider updating the doc comment above to mention the object form for RRF with a custom k (e.g., "rrf_k") and an example payload to improve discoverability.

lib/api/src/grpc/proto/points.proto (3)

565-568: Clarify default vs. custom k in the Fusion enum comment.

Given the addition of rrf_custom, the comment should reflect that RRF uses the default k (2), and rrf_custom enables custom k.

Apply this diff to reword the comment:

 enum Fusion {
-    RRF = 0; // Reciprocal Rank Fusion (with k=2)
+    RRF = 0; // Reciprocal Rank Fusion (default k=2). Use `rrf_custom` to set a custom k.
     DBSF = 1; // Distribution-Based Score Fusion
 }

670-673: Document k’s valid range on the new message.

GRPC-side validation enforces k ≥ 1 via build.rs. Make this explicit in the proto comment for better client UX.

Apply this diff to add an explanatory comment:

 message ReciprocalRankFusion {
-    uint32 k = 1;
+    // RRF blending parameter (must be >= 1). Default (when using `Fusion = RRF`) is 2.
+    uint32 k = 1;
 }

674-686: Good addition; preserves backward compatibility and enables custom k.

Adding rrf_custom as a separate oneof branch keeps existing clients working with Fusion=RRF while enabling custom k for those who need it. Field number 10 is free and safe to add.

Minor doc refinement to guide users toward when to use this variant:

-    ReciprocalRankFusion rrf_custom = 10; // Reciprocal Rank Fusion with custom `k` parameter.
+    ReciprocalRankFusion rrf_custom = 10; // Reciprocal Rank Fusion with custom `k` parameter. Prefer `Fusion = RRF` for default k=2.
docs/redoc/master/openapi.json (2)

14981-14989: Clarify Fusion description to document the new object form and defaults.

Consider updating the description to explicitly document both forms and the default behavior when using the string "rrf". This reduces ambiguity for API consumers.

-      "Fusion": {
-        "description": "Fusion algorithm allows to combine results of multiple prefetches.\n\nAvailable fusion algorithms:\n\n* `rrf` - Reciprocal Rank Fusion * `dbsf` - Distribution-Based Score Fusion",
+      "Fusion": {
+        "description": "Fusion algorithm allows combining results of multiple prefetches.\n\nAvailable fusion algorithms:\n\n* `rrf` - Reciprocal Rank Fusion. You can also configure its `k` parameter by using an object form: `{ \"rrf_k\": <uint, min 1> }`. If the plain string `\"rrf\"` is used, the default `k` is applied by the server.\n* `dbsf` - Distribution-Based Score Fusion",

14990-15003: Add example for the parameterized RRF object form.

Providing an inline example improves discoverability in Swagger/ReDoc and for SDK generators.

           {
             "type": "object",
             "required": [
               "rrf_k"
             ],
             "properties": {
               "rrf_k": {
                 "type": "integer",
                 "format": "uint",
-                "minimum": 0
+                "minimum": 1
               }
             },
+            "example": {
+              "rrf_k": 60
+            },
             "additionalProperties": false
           }
lib/segment/src/common/reciprocal_rank_fusion.rs (3)

15-17: Defensive check: guard against k == 0 to avoid division-by-zero

Even though upstream validation enforces k ≥ 1, this low-level helper should defensively assert to prevent accidental misuse and hard-to-debug NaNs/Infs if validation is bypassed in the future.

Apply this diff to add a debug assertion:

 fn position_score(position: usize, k: usize) -> f32 {
+    debug_assert!(k > 0, "RRF k must be >= 1 to avoid division by zero");
     1.0 / (position as f32 + k as f32)
 }

25-28: Document the k parameter contract (k ≥ 1, default = DEFAULT_RRF_K)

The signature change is correct. Please add a short doc note on the k constraints and recommended default to make the API self-explanatory for callers who don’t see upstream validators.


75-125: Float assertions are brittle; prefer approximate comparisons

Hard-coding f32 literals (e.g., 1.0833334) makes tests fragile across platforms/optimizations. Consider approximate comparisons with a small epsilon to reduce flakiness.

For example:

fn approx_eq(a: f32, b: f32, eps: f32) -> bool { (a - b).abs() <= eps }

assert!(approx_eq(scored_points[0].score, 0.5, 1e-6));
assert!(approx_eq(scored_points[0].score, 1.0833334, 1e-6));
docs/grpc/docs.md (2)

239-239: Fix markdown list indentation (MD007)

markdownlint flagged this newly added TOC entry. Align it with the expected 2-space indent.

Apply this diff:

-    - [ReciprocalRankFusion](#qdrant-ReciprocalRankFusion)
+  - [ReciprocalRankFusion](#qdrant-ReciprocalRankFusion)

If this file is generated, please adjust the generator to emit the correct indentation.


4183-4192: Enrich the ReciprocalRankFusion docs with constraints and defaults

Please add a short description noting:

  • k must be ≥ 1 (validation constraint)
  • default k is 2 when using Fusion = RRF (enum)
  • formula reference: score contribution = 1 / (k + rank_index), where rank_index starts at 0

If this page is generated from proto comments, adding the notes to the proto will propagate here.

I can propose proto comment text to include these nuances if helpful.

lib/collection/src/shards/local_shard/query.rs (1)

408-411: Pass-through of k to rrf_scoring is correct; assert invariant to guard accidental k=0

Given upstream validation enforces k ≥ 1, this match arm is functionally correct. Add a debug assertion to catch any accidental bypass of validation.

Apply this diff:

-        let fused = match fusion {
-            FusionInternal::RrfK(k) => rrf_scoring(sources, k),
+        let fused = match fusion {
+            FusionInternal::RrfK(k) => {
+                debug_assert!(k > 0, "RRF k must be >= 1");
+                rrf_scoring(sources, k)
+            },
             FusionInternal::Dbsf => score_fusion(sources, ScoreFusion::dbsf()),
         };
lib/collection/src/collection/query.rs (1)

363-365: Correct: use RrfK(k) and forward k to rrf_scoring

This aligns the collection-level fusion with the parameterized RRF behavior. Keep this consistent with the shard-level fusion_rescore to avoid drift in semantics.

Consider extracting a small helper that applies the common “rrf_scoring + optional score_threshold + limit/offset” pipeline so both code paths share identical logic.

lib/api/src/grpc/qdrant.rs (1)

7107-7111: Doc tweak suggestion: clarify default vs. custom RRF usage.

The comment “Reciprocal Rank Fusion (with k=2)” is good for the enum variant. Consider explicitly referencing that this is the default when using Fusion::Rrf, while Query::RrfCustom allows specifying k ≥ 1. This reduces ambiguity when users scan the generated docs.

If you prefer, I can propose an update in the source .proto docs so it’s reflected across all generated artifacts.

lib/collection/src/operations/universal_query/shard_query.rs (2)

60-60: Consider modeling k as NonZeroU32 (or at least u32) at the type level

k is constrained by the gRPC schema to uint32 and by validation to k ≥ 1. Representing it as NonZeroU32 (or u32) in FusionInternal would:

  • encode the ≥ 1 invariant at the type level, and
  • avoid lossy usize→u32 casts elsewhere.

This is optional but would reduce conversion hazards and improve API consistency across layers.


545-549: Nit: avoid importing Query in a scope where it names the impl target

Within impl From for api::grpc::qdrant::Query, importing Query can be confusing. Either fully qualify or omit the import to avoid shadowing the impl target type name.

Optional readability tweak:

-        use api::grpc::qdrant::{Fusion, Query, ReciprocalRankFusion};
+        use api::grpc::qdrant::{Fusion, ReciprocalRankFusion};
📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

💡 Knowledge Base configuration:

  • MCP integration is disabled by default for public repositories
  • Jira integration is disabled by default for public repositories
  • Linear integration is disabled by default for public repositories

You can enable these sources in your CodeRabbit configuration.

📥 Commits

Reviewing files that changed from the base of the PR and between 9973e41 and 7123ca0.

📒 Files selected for processing (17)
  • docs/grpc/docs.md (4 hunks)
  • docs/redoc/master/openapi.json (1 hunks)
  • lib/api/build.rs (1 hunks)
  • lib/api/src/grpc/proto/points.proto (3 hunks)
  • lib/api/src/grpc/proto/points_internal_service.proto (1 hunks)
  • lib/api/src/grpc/qdrant.rs (5 hunks)
  • lib/api/src/grpc/validate.rs (1 hunks)
  • lib/api/src/rest/schema.rs (1 hunks)
  • lib/api/src/rest/validate.rs (1 hunks)
  • lib/collection/benches/batch_query_bench.rs (2 hunks)
  • lib/collection/src/collection/query.rs (1 hunks)
  • lib/collection/src/operations/universal_query/collection_query.rs (1 hunks)
  • lib/collection/src/operations/universal_query/shard_query.rs (7 hunks)
  • lib/collection/src/shards/local_shard/query.rs (1 hunks)
  • lib/segment/src/common/reciprocal_rank_fusion.rs (4 hunks)
  • src/common/inference/batch_processing_grpc.rs (1 hunks)
  • src/common/inference/query_requests_grpc.rs (1 hunks)
🧰 Additional context used
🧠 Learnings (1)
📚 Learning: 2025-08-10T18:30:02.986Z
Learnt from: generall
PR: qdrant/qdrant#7006
File: lib/collection/src/operations/verification/update.rs:158-174
Timestamp: 2025-08-10T18:30:02.986Z
Learning: In Qdrant's strict mode verification code (lib/collection/src/operations/verification/update.rs), exhaustive pattern matching without `..` is intentionally used for structs like PointsBatch and PointsList. This design pattern ensures compilation fails when new fields are added, forcing developers to explicitly consider how new fields should be handled in the indexed_filter_write method. This provides visibility and compile-time safety for struct evolution.

Applied to files:

  • src/common/inference/batch_processing_grpc.rs
🧬 Code Graph Analysis (6)
lib/collection/benches/batch_query_bench.rs (1)
tests/openapi/helpers/helpers.py (1)
  • reciprocal_rank_fusion (120-145)
lib/api/src/rest/schema.rs (1)
lib/api/src/rest/validate.rs (14)
  • validate (15-21)
  • validate (25-30)
  • validate (34-45)
  • validate (49-59)
  • validate (63-83)
  • validate (87-93)
  • validate (97-101)
  • validate (105-133)
  • validate (137-142)
  • validate (146-150)
  • validate (154-170)
  • validate (174-232)
  • validate (236-247)
  • validate (251-273)
lib/collection/src/collection/query.rs (1)
lib/segment/src/common/reciprocal_rank_fusion.rs (1)
  • rrf_scoring (25-56)
lib/collection/src/operations/universal_query/collection_query.rs (2)
tests/openapi/helpers/helpers.py (1)
  • reciprocal_rank_fusion (120-145)
lib/collection/src/operations/universal_query/shard_query.rs (9)
  • from (537-542)
  • from (546-560)
  • from (564-577)
  • from (581-585)
  • from (589-593)
  • from (647-669)
  • from (673-703)
  • from (707-727)
  • from (731-761)
lib/collection/src/shards/local_shard/query.rs (1)
lib/segment/src/common/reciprocal_rank_fusion.rs (1)
  • rrf_scoring (25-56)
lib/collection/src/operations/universal_query/shard_query.rs (1)
lib/collection/src/operations/universal_query/collection_query.rs (2)
  • from (718-724)
  • from (728-732)
🪛 markdownlint-cli2 (0.17.2)
docs/grpc/docs.md

239-239: Unordered list indentation
Expected: 2; Actual: 4

(MD007, ul-indent)

⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (12)
  • GitHub Check: e2e-tests
  • GitHub Check: test-shard-snapshot-api-s3-minio
  • GitHub Check: test-consistency
  • GitHub Check: test-consensus-compose
  • GitHub Check: integration-tests-consensus
  • GitHub Check: integration-tests
  • GitHub Check: rust-tests-no-rocksdb (ubuntu-latest)
  • GitHub Check: storage-compat-test
  • GitHub Check: lint
  • GitHub Check: rust-tests (windows-latest)
  • GitHub Check: rust-tests (macos-latest)
  • GitHub Check: rust-tests (ubuntu-latest)
🔇 Additional comments (19)
lib/api/build.rs (1)

296-301: LGTM: gRPC validation for k ≥ 1 is correctly wired.

The added constraint on ReciprocalRankFusion.k ensures invalid values are rejected at the gRPC boundary.

lib/segment/src/common/reciprocal_rank_fusion.rs (1)

12-12: DEFAULT_RRF_K public constant looks good

Making the default RRF k value explicit and reusable across crates is a solid move.

src/common/inference/batch_processing_grpc.rs (1)

163-163: Exhaustive handling for RrfCustom is correct

No inference inputs to collect from fusion, so a no-op arm here is appropriate and keeps the match exhaustive as new variants arrive.

lib/collection/benches/batch_query_bench.rs (1)

22-22: Bench updated to parameterized RRF with DEFAULT_RRF_K

Import and usage look correct and keep the benchmark aligned with the new API while preserving previous behavior via the default.

Also applies to: 264-265

docs/grpc/docs.md (2)

4013-4014: New Query.oneof field is clear

Adding rrf_custom with an object to carry k communicates intent and keeps backward compatibility with enum-based RRF.


5232-5233: Enum entry updated to reflect default k=2

Good clarification that the enum-based RRF uses k=2 by default, matching legacy behavior.

lib/api/src/grpc/proto/points_internal_service.proto (1)

282-283: rrf_custom mapping verified — optional rename suggestion

rrf_custom (tag 7) is wired end-to-end; no functional changes required. Renaming to rrf or rrf_params remains an optional nit.

Key locations confirming mappings:

  • src/common/inference/query_requests_grpc.rs:293–295 — Variant::RrfCustom(...) -> Query::Fusion(FusionInternal::RrfK(k as usize))
  • lib/collection/src/operations/universal_query/shard_query.rs:551–554 and 570–572 — FusionInternal::RrfK(k) serialized to Query/Score::RrfCustom(ReciprocalRankFusion { k })
  • lib/collection/src/operations/universal_query/shard_query.rs:611–612 — incoming Score::RrfCustom(...) -> FusionInternal::RrfK(k)
  • lib/api/src/grpc/qdrant.rs:10191 (and 5676) — generated prost message has RrfCustom with tag 7
  • lib/api/src/rest/schema.rs:515–519 — REST enum supports Rrf and RrfK(usize)
  • lib/collection/src/operations/universal_query/collection_query.rs:720–721 — Fromrest::Fusion maps Rrf/RrfK into FusionInternal::RrfK
  • lib/collection/src/shards/local_shard/query.rs:409 — FusionInternal::RrfK(k) passed to rrf_scoring(sources, k)
  • lib/api/src/grpc/validate.rs:349 — RrfCustom included in gRPC validation

[result: mappings present end-to-end; rename is optional]

lib/collection/src/operations/universal_query/collection_query.rs (1)

713-713: RRF k types and default verified — no type-width issues

DEFAULT_RRF_K is pub const DEFAULT_RRF_K: usize = 2, and both REST and internal enums use usize for k, while gRPC boundaries explicitly cast to/from u32 — no implicit-width/portability issue found.

Files/locations checked:

  • lib/segment/src/common/reciprocal_rank_fusion.rs — pub const DEFAULT_RRF_K: usize = 2
  • lib/api/src/rest/schema.rs — pub enum Fusion { Rrf, #[validate(range(min = 1))] RrfK(usize) }
  • lib/collection/src/operations/universal_query/shard_query.rs — pub enum FusionInternal { RrfK(usize), Dbsf } and conversions:
    • impl Fromapi::grpc::qdrant::Fusion maps Rrf -> FusionInternal::RrfK(DEFAULT_RRF_K)
    • Query builders cast k as u32 (k as u32)
    • gRPC -> internal conversion casts k as usize (k as usize)

Recommendation: add an OpenAPI test that exercises a non-default rrf_k (helpers currently hardcode k=2) to prevent regressions — happy to author that test.

lib/api/src/grpc/qdrant.rs (6)

5627-5631: New RRF message looks correct (type + validation).

k: u32 with #[validate(range(min = 1))] matches expected constraints for RRF. Serialization/validation derives align with the rest of the API types.


5632-5635: Consistent derives on Query.

Adding/keeping validator::Validate and serde::Serialize on Query is consistent with adjacent messages and enables nested validation for new variants.


5674-5676: New Query variant RrfCustom added with tag 10 – good addition.

Variant naming and tag assignment are clear and consistent with the rest of the API.


10161-10164: Internal shard Query score oneof extended (tag 7) – looks right.

Including 7 for the custom RRF score path is consistent and maintains wire compatibility.


5637-5638: Query.oneof tag 10 (RrfCustom) — downstream handling verified

RrfCustom is wired end-to-end: gRPC message + validation, REST/OpenAPI validation, conversions to/from internal types, and scoring. Legacy Fusion::Rrf still maps to DEFAULT_RRF_K = 2.

Checked locations:

  • lib/api/src/grpc/qdrant.rs — ReciprocalRankFusion { k: u32, #[validate(range(min = 1))] } and query::Variant::RrfCustom tag = 10 (≈ lines 5627–5676).
  • lib/api/src/grpc/validate.rs — RrfCustom included in query validation match (≈ line 349).
  • lib/collection/src/operations/universal_query/shard_query.rs — From -> Query uses RrfCustom; Query/Score -> FusionInternal maps RrfCustom -> RrfK(k); Fromapi::grpc::qdrant::Fusion -> FusionInternal::RrfK(DEFAULT_RRF_K) (≈ lines 536–613).
  • src/common/inference/query_requests_grpc.rs — Variant::RrfCustom -> Query::Fusion(FusionInternal::RrfK(k as usize)) (≈ lines 293–294).
  • lib/segment/src/common/reciprocal_rank_fusion.rs — DEFAULT_RRF_K = 2 and rrf_scoring implementation.
  • lib/api/src/rest/schema.rs & lib/api/src/rest/validate.rs — REST RrfK range(min=1) and Fusion validation handling.
  • lib/collection/src/operations/universal_query/collection_query.rs — Fromrest::Fusion maps Rrf -> FusionInternal::RrfK(DEFAULT_RRF_K).

No fixes required; changes are consistent and wire-compatible.


10189-10191: RRF: custom k is correctly propagated; legacy Fusion::Rrf uses DEFAULT_RRF_K

Verified: the new RrfCustom variant is mapped to an internal RrfK(k) and that RrfK(k) is passed into rrf_scoring(k). The legacy Fusion::Rrf path maps to RrfK(DEFAULT_RRF_K).

  • src/common/inference/query_requests_grpc.rs — Variant::RrfCustom(grpc::ReciprocalRankFusion { k }) => Query::Fusion(FusionInternal::RrfK(k as usize))
  • lib/collection/src/operations/universal_query/shard_query.rs — Score::RrfCustom(...) => ScoringQuery::Fusion(FusionInternal::RrfK(k as usize)); Fromapi::grpc::qdrant::Fusion maps Fusion::Rrf => FusionInternal::RrfK(DEFAULT_RRF_K)
  • lib/collection/src/shards/local_shard/query.rs — FusionInternal::RrfK(k) => rrf_scoring(sources, k)
  • lib/collection/src/collection/query.rs — FusionInternal::RrfK(k) => rrf_scoring(intermediates, *k)
  • lib/segment/src/common/reciprocal_rank_fusion.rs — pub fn rrf_scoring(..., k: usize) and pub const DEFAULT_RRF_K: usize = 2

No change required.

lib/collection/src/operations/universal_query/shard_query.rs (5)

6-6: LGTM: default RRF k is now explicitly imported

Using DEFAULT_RRF_K here is correct and keeps the default centralized.


118-123: LGTM: RRF requires intermediate results

Returning true for FusionInternal::RrfK(_) is correct since RRF needs per-prefetch rankings to fuse.


157-160: LGTM: expected order is LargeBetter for fusion scorings

RRF and DBSF produce higher-is-better scores; mapping to Order::LargeBetter is correct.


536-541: LGTM: legacy Fusion::Rrf maps to default k

Mapping grpc Fusion::Rrf to FusionInternal::RrfK(DEFAULT_RRF_K) preserves backwards compatibility.


681-681: LGTM: reuse From for shard query serialization

Delegating via Self::from(fusion) removes duplication and stays consistent with the dedicated conversion impl.

Comment on lines +552 to +571
variant: Some(QueryVariant::RrfCustom(ReciprocalRankFusion {
k: k as u32,
})),
},
FusionInternal::Dbsf => Query {
variant: Some(QueryVariant::Fusion(i32::from(Fusion::Dbsf))),
},
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

⚠️ Potential issue

Potential usize→u32 truncation: k may silently overflow

Here k (usize) is cast to u32 with as. If a REST path supplies a large k (usize), this will silently truncate when constructing gRPC messages, yielding incorrect behavior.

Mitigations:

  • Short-term: clamp when converting to gRPC.
  • Longer-term: store k as u32/NonZeroU32 in FusionInternal.

Apply this minimal clamping to avoid truncation:

-                variant: Some(QueryVariant::RrfCustom(ReciprocalRankFusion {
-                    k: k as u32,
-                })),
+                variant: Some(QueryVariant::RrfCustom(ReciprocalRankFusion {
+                    k: u32::try_from(k).unwrap_or(u32::MAX),
+                })),

And similarly below:

-                score: Some(Score::RrfCustom(ReciprocalRankFusion { k: k as u32 })),
+                score: Some(Score::RrfCustom(ReciprocalRankFusion {
+                    k: u32::try_from(k).unwrap_or(u32::MAX),
+                })),

Follow-up: enforce an upper bound at validation time to a sane max (e.g., u32::MAX or a domain-specific cap) to avoid clamping silently.

Also applies to: 570-574

🤖 Prompt for AI Agents
In lib/collection/src/operations/universal_query/shard_query.rs around lines
552-558 (and similarly 570-574), k is being cast from usize to u32 with as which
can silently truncate large values; change the conversion to clamp the usize
into the u32 range before creating the gRPC message (e.g., use
u32::MAX.saturating_min(k as u128) style or u32::try_from(k).unwrap_or(u32::MAX)
equivalent) so the value never truncates unexpectedly, and apply the same
clamping at the other location; additionally add a follow-up to validate/limit k
earlier (e.g., enforce a sensible upper bound) to avoid silent clamping.

@timvisee

This comment was marked as outdated.

@generall
Copy link
Member

Why not

{
    "prefetch": [ ... ],
    "query": { "fusion": { "type": "rrf", "k": 60 } }
}

?

@coszio
Copy link
Contributor Author

coszio commented Aug 15, 2025

I changed it to use the tagged approach to mimic our payload schema params interface, thanks for the suggestions @generall @timvisee 🙌

coderabbitai[bot]

This comment was marked as resolved.

@coderabbitai
Copy link
Contributor

coderabbitai bot commented Aug 20, 2025

📝 Walkthrough

Walkthrough

Introduces parameterized Reciprocal Rank Fusion (RRF) across API, docs, and internals. Adds a new Rrf type (field k: optional uint, validated k ≥ 1) to protobufs, OpenAPI, and REST schemas and exposes it on Query (gRPC) and as an RrfQuery (REST). Internally replaces FusionInternal::Rrf with FusionInternal::RrfK(usize) and adds DEFAULT_RRF_K = 2; updates rrf_scoring to accept k. Updates conversions between REST/gRPC/internal representations, validation paths, batching/inference handling, benches, tests, and documentation/OpenAPI to propagate and document the new k parameter.

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

Possibly related PRs

Suggested reviewers

  • timvisee
  • generall

Tip

🔌 Remote MCP (Model Context Protocol) integration is now available!

Pro plan users can now connect to remote MCP servers from the Integrations page. Connect with popular remote MCPs such as Notion and Linear to add more context to your reviews and chats.

✨ Finishing Touches
  • 📝 Generate Docstrings
🧪 Generate unit tests
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch custom-rrf

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.

Support

Need help? Create a ticket on our support page for assistance with any issues or questions.

CodeRabbit Commands (Invoked using PR/Issue comments)

Type @coderabbitai help to get the list of available commands.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Status, Documentation and Community

  • Visit our Status Page to check the current availability of CodeRabbit.
  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
lib/api/src/grpc/qdrant.rs (1)

10196-10206: Add nested validation to shard-level Query.score

The query_shard_points::Query struct derives Validate but its oneof field isn’t marked #[validate(nested)], so embedded types like FusionParams won’t be checked.

• In lib/api/src/grpc/qdrant.rs, locate the Query struct under pub mod query_shard_points (around line 10196) and apply:

 pub struct Query {
     #[prost(oneof = "query::Score", tags = "1, 2, 3, 4, 5, 6, 7")]
+    #[validate(nested)]
     pub score: ::core::option::Option<query::Score>,
 }

• Also verify the second occurrence (lines 10224–10227) if any Query-typed field derives Validate, and add #[validate(nested)] there too.

• Alternatively, if you’d rather centralize oneof checks in lib/api/src/grpc/validate.rs, implement a ValidateExt for query_shard_points::query::Score so that nested variants (e.g. FusionParams) are validated.

♻️ Duplicate comments (4)
lib/api/src/rest/schema.rs (1)

533-542: Allow omitting type and accept rrf_k as an alias for k

Right now, RrfParams requires the type field at deserialization time, despite having a default enum variant. Also, some examples/comments have used rrf_k as a key. To make the request shape more forgiving and self-explanatory:

  • Default the type field to Rrf when omitted.
  • Accept "rrf_k" as an alias for "k".

This will support both {"fusion": {"type":"rrf","k":60}} and {"fusion": {"k":60}} and be lenient if users follow older examples.

 #[derive(Debug, Default, Serialize, Deserialize, JsonSchema, Validate)]
 #[serde(rename_all = "snake_case")]
 pub struct RrfParams {
-    pub r#type: RrfType,
+    #[serde(default)]
+    pub r#type: RrfType,
 
     /// K parameter for reciprocal rank fusion
     #[validate(range(min = 1))]
-    #[serde(default)]
-    pub k: Option<usize>,
+    #[serde(default, alias = "rrf_k")]
+    pub k: Option<usize>,
 }
lib/api/src/grpc/validate.rs (1)

349-350: Validate nested FusionParams to enforce k ≥ 1 (don’t short-circuit validation)

Currently FusionParams is accepted without validating its payload, allowing k=0 to pass through despite the documented min=1 constraint. Validate the nested message so field-level constraints apply. This aligns with earlier feedback for the old RrfCustom route.

Apply this diff:

-            grpc::query::Variant::Sample(_)
-            | grpc::query::Variant::Fusion(_)
-            | grpc::query::Variant::FusionParams(_)
-            | grpc::query::Variant::OrderBy(_) => Ok(()),
+            grpc::query::Variant::Sample(_)
+            | grpc::query::Variant::Fusion(_)
+            | grpc::query::Variant::OrderBy(_) => Ok(()),
+            grpc::query::Variant::FusionParams(params) => params.validate(),

If FusionParams or its inner RrfParams doesn’t derive Validate, add it and ensure k has range(min = 1). Also consider a negative unit test asserting that a Query with FusionParams { rrf: { k: 0 } } fails validation.

Use this to confirm the build script or derives inject range(min = 1) for RrfParams.k:

#!/bin/bash
# Check for RrfParams.k constraints and Validate derive
rg -n -C3 'message\s+RrfParams\b' lib/api/src/grpc/proto
rg -n -C5 'RrfParams\.k|range\s*\(\s*min\s*=\s*1\s*\)' lib/api
lib/collection/src/operations/universal_query/shard_query.rs (2)

569-579: Avoid usize -> u32 truncation when emitting Query (custom k path)

k is usize internally but serialized as u32. Using as may silently truncate on 64-bit platforms for very large k.

Apply a fallible conversion with clamping to avoid truncation:

-            FusionInternal::RrfK(k) => Query {
+            FusionInternal::RrfK(k) => Query {
                 variant: Some(QueryVariant::FusionParams(FusionParams {
-                    variant: Some(FusionParamsVariant::Rrf(RrfParams { k: Some(k as u32) })),
+                    variant: Some(FusionParamsVariant::Rrf(RrfParams {
+                        k: Some(u32::try_from(k).unwrap_or(u32::MAX)),
+                    })),
                 })),
             },

Optionally, instead of clamping, return an error earlier or change FusionInternal to hold u32/NonZeroU32 (see earlier refactor suggestion).


593-604: Avoid usize -> u32 truncation when emitting query_shard_points::Query (custom k path)

Same truncation risk here when serializing k.

Apply the same fix:

-            FusionInternal::RrfK(k) => Query {
+            FusionInternal::RrfK(k) => Query {
                 score: Some(Score::FusionParams(FusionParams {
-                    variant: Some(FusionParamsVariant::Rrf(RrfParams { k: Some(k as u32) })),
+                    variant: Some(FusionParamsVariant::Rrf(RrfParams {
+                        k: Some(u32::try_from(k).unwrap_or(u32::MAX)),
+                    })),
                 })),
             },
🧹 Nitpick comments (17)
lib/api/src/grpc/proto/points.proto (2)

568-570: Clarify RRF enum doc to reflect default vs parameterized K

The comment “with k=2” is fine as a shorthand, but now that parameterized RRF exists, consider clarifying that RRF (enum) uses the default k=2, while FusionParams.rrf allows overriding k.

-    RRF = 0; // Reciprocal Rank Fusion (with k=2)
+    RRF = 0; // Reciprocal Rank Fusion (default k=2). Use FusionParams.rrf to override k.

695-696: Adding Query.fusion_params is backward compatible; verify precedence rules

Introducing fusion_params alongside the legacy Fusion enum in the same oneof is backwards compatible. Please ensure server-side validation clarifies behavior if clients mistakenly set both (oneof last-write-wins at decode time). If not already handled, consider rejecting requests that provide both in REST to avoid ambiguity.

lib/api/src/rest/schema.rs (1)

706-709: FusionQuery switches to FusionInterface — LGTM; doc follow-up

This enables both legacy string fusion and parameterized object forms. Consider updating any user-facing docs/snippets to show both shapes:

  • "fusion": "rrf" (default k=2)
  • "fusion": { "type": "rrf", "k": 60 }
lib/api/src/grpc/qdrant.rs (2)

5631-5641: RrfParams looks correct; suggest documenting the default.

The optional k with validation is good. Consider documenting that omission defaults to the legacy behavior (k=2) to avoid ambiguity in client usage.

 pub struct RrfParams {
-    /// K parameter for reciprocal rank fusion
+    /// K parameter for reciprocal rank fusion
+    /// If omitted, defaults to 2 (legacy behavior).
     #[prost(uint32, optional, tag = "1")]
     #[validate(range(min = 1))]
     pub k: ::core::option::Option<u32>,
 }

7142-7143: Clarify the legacy k=2 semantics in the Fusion enum doc.

The comment “with k=2” is accurate for the legacy enum-based fusion. Consider making “default k=2” explicit to reduce confusion alongside the new parameterized FusionParams.

-    /// Reciprocal Rank Fusion (with k=2)
+    /// Reciprocal Rank Fusion (default k=2). For custom k use `FusionParams::Rrf`.
lib/collection/benches/batch_query_bench.rs (1)

262-262: Switch to FusionInternal::RrfK(DEFAULT_RRF_K) is correct

Bench inputs now reflect the new internal enum shape. Optional: consider running the bench with a few k values (e.g., [1, 2, DEFAULT_RRF_K, 120]) to observe sensitivity of fusion latency/throughput to k.

lib/collection/src/tests/shard_query.rs (1)

5-5: Add a targeted test to exercise non-default k

Optional but useful: add a case that uses a small k (e.g., k=1) vs a large k to verify that fusion ordering/weights change as expected. This will catch regressions in parameter pluming and scoring sensitivity.

Also applies to: 62-62, 97-97, 144-144, 188-188

docs/grpc/docs.md (5)

170-170: Fix markdown list indentation (MD007).

markdownlint flags these list items for using 4 spaces. Align with 2-space indentation to satisfy MD007, unless this file is auto-generated and lint is intentionally ignored for it.

Also applies to: 251-251


2994-3007: Clarify FusionParams usage and defaults.

Document that FusionParams is the parameterized alternative to Fusion and that RRF’s k defaults to 2 when omitted.

-### FusionParams
-Parameterized fusion of multiple prefetches.
+### FusionParams
+Parameterized fusion of multiple prefetches. Use this to pass algorithm-specific parameters.
+If omitted, defaults are applied (for RRF, k defaults to 2).

 | Field | Type | Label | Description |
 | ----- | ---- | ----- | ----------- |
-| rrf | [RrfParams](#qdrant-RrfParams) |  | Reciprocal Rank Fusion |
+| rrf | [RrfParams](#qdrant-RrfParams) |  | Reciprocal Rank Fusion (set k to override the default; k ≥ 1) |

4031-4031: State mutual exclusivity for fusion vs fusion_params.

Add a short note that Query.fusion and Query.fusion_params are mutually exclusive (they occupy the same oneof).

-| fusion_params | [FusionParams](#qdrant-FusionParams) |  | Parameterized fusion of multiple prefetches. |
+| fusion_params | [FusionParams](#qdrant-FusionParams) |  | Parameterized fusion of multiple prefetches. Mutually exclusive with `fusion`. |

4419-4427: Document default and constraints for RrfParams.k.

Spell out default and minimum accepted value for k.

-### RrfParams
-Parameterized reciprocal rank fusion
+### RrfParams
+Parameterized reciprocal rank fusion. When omitted, `k` defaults to 2 (DEFAULT_RRF_K).

 | Field | Type | Label | Description |
 | ----- | ---- | ----- | ----------- |
-| k | [uint32](#uint32) | optional | K parameter for reciprocal rank fusion |
+| k | [uint32](#uint32) | optional | K parameter for reciprocal rank fusion (k ≥ 1; default 2) |

5252-5252: Cross-reference FusionParams for non-default k.

Help users discover how to customize k.

-| RRF | 0 | Reciprocal Rank Fusion (with k=2) |
+| RRF | 0 | Reciprocal Rank Fusion (with k=2). Use `FusionParams` → `RrfParams` to override `k`. |
docs/redoc/master/openapi.json (3)

15007-15010: Use of FusionInterface is correct; consider adding an inline example for clarity.

This change enables both legacy string and new object forms. Adding an example here will help users and codegen see the intended shapes immediately.

           "fusion": {
-            "$ref": "#/components/schemas/FusionInterface"
+            "$ref": "#/components/schemas/FusionInterface",
+            "example": { "type": "rrf", "k": 60 }
           }

15030-15036: Add a discriminator to FusionParams to improve SDK codegen and validation.

Declaring a discriminator on the tagged union helps generators produce sealed types and improves error messages.

       "FusionParams": {
-        "anyOf": [
+        "anyOf": [
           {
             "$ref": "#/components/schemas/RrfParams"
           }
-        ]
+        ],
+        "discriminator": {
+          "propertyName": "type",
+          "mapping": {
+            "rrf": "#/components/schemas/RrfParams"
+          }
+        }
       },

15037-15055: RrfParams.k: document the default (2) and add an example.

Server uses DEFAULT_RRF_K = 2 when k is unset. Make this explicit in the schema description and provide an example value.

       "RrfParams": {
         "type": "object",
         "required": [
           "type"
         ],
         "properties": {
           "type": {
             "$ref": "#/components/schemas/RrfType"
           },
           "k": {
-            "description": "K parameter for reciprocal rank fusion",
+            "description": "K parameter for reciprocal rank fusion. Defaults to 2 if omitted.",
             "default": null,
             "type": "integer",
             "format": "uint",
+            "example": 60,
             "minimum": 1,
             "nullable": true
           }
         }
       },
lib/collection/src/operations/universal_query/shard_query.rs (2)

60-60: Consider storing k as NonZeroU32 (or u32) to match API surface and invariants

Internal RrfK(usize) forces lossy casts to u32 at the API boundary and allows 0, which the proto forbids. Using NonZeroU32 (or plain u32) would remove casts and encode the invariant in the type.

Example enum pivot (requires touch-ups at call sites):

-    RrfK(usize),
+    RrfK(std::num::NonZeroU32),

Benefits: fewer casts, no silent truncation risk, and k ≥ 1 guaranteed by type.


563-584: Optional: Deduplicate Fusion→gRPC encoding between Query and query_shard_points::Query

Both impls are nearly identical. Consider extracting a small helper to DRY up the construction of Fusion/FusionParams with the k==DEFAULT_RRF_K fast-path. This reduces maintenance overhead.

If you want, I can sketch a small internal helper API to unify these two impls.

Also applies to: 586-608

📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

💡 Knowledge Base configuration:

  • MCP integration is disabled by default for public repositories
  • Jira integration is disabled by default for public repositories
  • Linear integration is disabled by default for public repositories

You can enable these sources in your CodeRabbit configuration.

📥 Commits

Reviewing files that changed from the base of the PR and between 2b4f19f and e862414.

📒 Files selected for processing (19)
  • docs/grpc/docs.md (6 hunks)
  • docs/redoc/master/openapi.json (2 hunks)
  • lib/api/build.rs (1 hunks)
  • lib/api/src/grpc/proto/points.proto (3 hunks)
  • lib/api/src/grpc/proto/points_internal_service.proto (1 hunks)
  • lib/api/src/grpc/qdrant.rs (5 hunks)
  • lib/api/src/grpc/validate.rs (1 hunks)
  • lib/api/src/rest/schema.rs (2 hunks)
  • lib/api/src/rest/validate.rs (2 hunks)
  • lib/collection/benches/batch_query_bench.rs (2 hunks)
  • lib/collection/src/collection/query.rs (1 hunks)
  • lib/collection/src/operations/universal_query/collection_query.rs (1 hunks)
  • lib/collection/src/operations/universal_query/planned_query.rs (6 hunks)
  • lib/collection/src/operations/universal_query/shard_query.rs (8 hunks)
  • lib/collection/src/shards/local_shard/query.rs (1 hunks)
  • lib/collection/src/tests/shard_query.rs (5 hunks)
  • lib/segment/src/common/reciprocal_rank_fusion.rs (4 hunks)
  • src/common/inference/batch_processing_grpc.rs (1 hunks)
  • src/common/inference/query_requests_grpc.rs (1 hunks)
🧰 Additional context used
🧠 Learnings (6)
📚 Learning: 2025-08-15T15:56:36.821Z
Learnt from: coszio
PR: qdrant/qdrant#7065
File: lib/collection/src/operations/universal_query/shard_query.rs:0-0
Timestamp: 2025-08-15T15:56:36.821Z
Learning: Internal gRPC APIs in Qdrant don't require additional runtime validation when proto-level validation is already configured via build.rs range constraints.

Applied to files:

  • lib/api/build.rs
  • lib/collection/src/operations/universal_query/shard_query.rs
📚 Learning: 2025-08-10T18:31:56.855Z
Learnt from: generall
PR: qdrant/qdrant#7006
File: lib/collection/src/operations/point_ops.rs:501-528
Timestamp: 2025-08-10T18:31:56.855Z
Learning: In Qdrant, batch operations validate that `ids`, `vectors`, and `payloads` (if present) have matching lengths at the REST API level in `lib/api/src/rest/validate.rs` through the `Validate` trait implementation for `Batch`. This validation happens before data is converted to internal structures like `PointInsertOperationsInternal`, so methods operating on these internal structures can safely assume the lengths match.

Applied to files:

  • lib/api/src/rest/validate.rs
📚 Learning: 2025-08-10T18:26:12.443Z
Learnt from: generall
PR: qdrant/qdrant#7006
File: docs/redoc/master/openapi.json:13645-13652
Timestamp: 2025-08-10T18:26:12.443Z
Learning: In qdrant, docs/redoc/master/openapi.json is auto-generated from the REST schemas. To change field docs, edit lib/api/src/rest/schema.rs (e.g., add doc comments or #[schemars(description = ...)]). Specifically, UpdateVectors.update_filter lacked a description and should state: "If specified, only update vectors for points that match this filter; points not matching the filter are left unchanged."

Applied to files:

  • lib/api/src/rest/schema.rs
📚 Learning: 2025-08-10T18:26:33.017Z
Learnt from: generall
PR: qdrant/qdrant#7006
File: docs/redoc/master/openapi.json:10393-10401
Timestamp: 2025-08-10T18:26:33.017Z
Learning: In qdrant, docs/redoc/master/openapi.json is auto-generated. Any description changes must be made in the source (e.g., lib/api/src/rest/schema.rs and/or lib/api/src/grpc/proto/points.proto), not edited directly in the JSON.

Applied to files:

  • lib/api/src/rest/schema.rs
📚 Learning: 2025-08-10T18:25:16.206Z
Learnt from: generall
PR: qdrant/qdrant#7006
File: docs/redoc/master/openapi.json:10626-10634
Timestamp: 2025-08-10T18:25:16.206Z
Learning: In Qdrant, docs/redoc/master/openapi.json is autogenerated from the Rust REST schema definitions (e.g., lib/api/src/rest/schema.rs). To change OpenAPI descriptions, update the Rust struct field doc comments/attributes rather than editing the JSON directly.

Applied to files:

  • lib/api/src/rest/schema.rs
📚 Learning: 2025-07-11T11:35:21.549Z
Learnt from: generall
PR: qdrant/qdrant#6854
File: lib/segment/src/index/query_estimator.rs:320-327
Timestamp: 2025-07-11T11:35:21.549Z
Learning: In test code for Qdrant's query estimator (lib/segment/src/index/query_estimator.rs), simplified ID resolution logic using `id.to_string().parse().unwrap()` is acceptable for testing purposes and doesn't need to match production code's `id_tracker.internal_id()` approach. Test code can use mock implementations that serve the testing goals.

Applied to files:

  • lib/collection/src/operations/universal_query/planned_query.rs
🧬 Code Graph Analysis (11)
src/common/inference/query_requests_grpc.rs (1)
lib/collection/src/operations/universal_query/shard_query.rs (7)
  • try_from (228-265)
  • try_from (271-297)
  • try_from (354-360)
  • try_from (366-378)
  • try_from (384-390)
  • try_from (396-418)
  • try_from (424-527)
lib/api/src/grpc/validate.rs (1)
src/tonic/api/points_api.rs (1)
  • query (587-608)
lib/collection/src/operations/universal_query/collection_query.rs (1)
lib/collection/src/operations/universal_query/shard_query.rs (9)
  • from (555-560)
  • from (564-583)
  • from (587-607)
  • from (611-615)
  • from (619-623)
  • from (677-699)
  • from (703-733)
  • from (737-757)
  • from (761-791)
lib/api/src/rest/validate.rs (1)
lib/api/src/grpc/validate.rs (13)
  • validate (14-14)
  • validate (19-21)
  • validate (29-31)
  • validate (39-41)
  • validate (49-54)
  • validate (58-64)
  • validate (68-74)
  • validate (78-85)
  • validate (89-97)
  • validate (101-112)
  • validate (116-123)
  • validate (127-134)
  • validate (138-145)
lib/api/src/rest/schema.rs (1)
lib/api/src/rest/validate.rs (16)
  • validate (15-21)
  • validate (25-30)
  • validate (34-45)
  • validate (49-59)
  • validate (63-83)
  • validate (87-93)
  • validate (97-101)
  • validate (105-109)
  • validate (113-118)
  • validate (122-150)
  • validate (154-159)
  • validate (163-167)
  • validate (171-187)
  • validate (191-249)
  • validate (253-264)
  • validate (268-290)
lib/collection/src/operations/universal_query/planned_query.rs (1)
lib/collection/src/shards/local_shard/query.rs (1)
  • rescore (272-398)
lib/collection/src/shards/local_shard/query.rs (1)
lib/segment/src/common/reciprocal_rank_fusion.rs (1)
  • rrf_scoring (25-56)
lib/segment/src/common/reciprocal_rank_fusion.rs (1)
lib/segment/src/common/score_fusion.rs (1)
  • point (172-182)
lib/collection/src/collection/query.rs (1)
lib/segment/src/common/reciprocal_rank_fusion.rs (1)
  • rrf_scoring (25-56)
lib/collection/src/operations/universal_query/shard_query.rs (2)
tests/openapi/helpers/helpers.py (1)
  • reciprocal_rank_fusion (120-145)
lib/collection/src/collection/query.rs (1)
  • query (46-67)
lib/api/src/grpc/qdrant.rs (1)
lib/api/src/grpc/validate.rs (8)
  • validate (14-14)
  • validate (19-21)
  • validate (29-31)
  • validate (39-41)
  • validate (49-54)
  • validate (58-64)
  • validate (68-74)
  • validate (78-85)
🪛 markdownlint-cli2 (0.17.2)
docs/grpc/docs.md

170-170: Unordered list indentation
Expected: 2; Actual: 4

(MD007, ul-indent)


251-251: Unordered list indentation
Expected: 2; Actual: 4

(MD007, ul-indent)

⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (12)
  • GitHub Check: e2e-tests
  • GitHub Check: test-shard-snapshot-api-s3-minio
  • GitHub Check: integration-tests
  • GitHub Check: test-consistency
  • GitHub Check: test-consensus-compose
  • GitHub Check: integration-tests-consensus
  • GitHub Check: rust-tests (windows-latest)
  • GitHub Check: rust-tests (macos-latest)
  • GitHub Check: rust-tests (ubuntu-latest)
  • GitHub Check: rust-tests-no-rocksdb (ubuntu-latest)
  • GitHub Check: lint
  • GitHub Check: storage-compat-test
🔇 Additional comments (38)
lib/api/build.rs (1)

300-301: RrfParams.k validation added — LGTM

Enforcing k >= 1 at the proto-build layer is correct and sufficient. This matches the new RRF parameter semantics and avoids k=0 division by zero in rrf scoring.

lib/api/src/grpc/proto/points.proto (2)

672-676: RrfParams shape and optional k — LGTM

Using optional uint32 k with proto3 optional is appropriate. Combined with build-time validation (k >= 1) this prevents invalid inputs and keeps defaulting to 2 in downstream layers.


677-683: FusionParams oneof scaffolding looks good

Oneof with rrf = 1 is a solid forward-compatible shape for adding more fusion types later. No issues spotted.

lib/api/src/grpc/proto/points_internal_service.proto (1)

282-283: Internal Query adds FusionParams — LGTM

Propagating FusionParams into the internal score oneof is aligned with the external proto. Given it reuses the RrfParams validation via build.rs, no extra internal validation is needed.

lib/segment/src/common/reciprocal_rank_fusion.rs (1)

12-17: Parameterized RRF with DEFAULT_RRF_K — LGTM

  • DEFAULT_RRF_K = 2 exposes a clear default.
  • position_score(pos, k) and rrf_scoring(..., k) are correct and avoid div-by-zero with k>=1.
  • Tests cover the default path and preserve prior behavior.

No functional issues found.

Also applies to: 25-29, 34-35

lib/api/src/rest/schema.rs (1)

520-525: REST FusionParams wrapper — LGTM

Untagged enum with a single Rrf(RrfParams) variant keeps the JSON concise and future-proofs additional fusion param types.

lib/api/src/rest/validate.rs (3)

12-12: LGTM: new Fusion wrapper imports are correct

Importing FusionInterface and FusionParams here aligns REST validation with the new tagged payload shape.


112-119: LGTM: composed validation covers both legacy string and tagged object forms

Delegating through FusionInterface cleanly validates both "fusion": "rrf" and "fusion": { "rrf": { "k": ... } } payloads. Mirrors the gRPC validation style.


104-110: Resolved: RrfParams already enforces k ≥ 1

The RrfParams struct in both REST and gRPC schemas includes a #[validate(range(min = 1))] on the k field, ensuring that any payload with k < 1 will be rejected:

  • lib/api/src/rest/schema.rs: line 539 (#[validate(range(min = 1))])
  • lib/api/src/grpc/qdrant.rs: line 5639 (#[validate(range(min = 1))])

No further changes are needed here.

lib/api/src/grpc/qdrant.rs (2)

5666-5669: Backward-compatible FusionParams adapter verified

  • lib/collection/src/operations/universal_query/shard_query.rs (lines 371–374): TryFrom<grpc::FusionParams> maps k.map(...).unwrap_or(DEFAULT_RRF_K)—defaults to DEFAULT_RRF_K when k is None.
  • lib/api/src/grpc/qdrant.rs (lines 571–577): maps FusionInternal::RrfK(DEFAULT_RRF_K) → legacy Fusion::Rrf tag and FusionInternal::RrfK(k) (when k != DEFAULT_RRF_K) → new FusionParams tag.
  • lib/api/src/rest/schema.rs: FusionInterface is a serde(untagged) enum supporting both the legacy Fusion string and the new FusionParams object.
  • lib/collection/src/operations/universal_query/collection_query.rs (lines 710–720 & 726–733): From<rest::Fusion> and From<rest::FusionParams>FusionInternal cover both forms.

All conversions and adapters correctly handle both legacy and new tags.


5643-5649: Enforce nested validation on FusionParams and wire it through Query.validate

The FusionParams struct already derives validator::Validate, but its variant field isn’t marked #[validate(nested)], and the generated Query validator currently skips the FusionParams arm, so inner RrfParams.k ≥ 1 checks never run on the gRPC path.

Please update:

  • lib/api/src/grpc/qdrant.rs
    • Add #[validate(nested)] to the variant field of FusionParams.
  • lib/api/src/grpc/validate.rs
    • In impl Validate for super::qdrant::query::Variant, replace the FusionParams(_) => Ok(()) arm with FusionParams(q) => q.validate() (or insert it before the grouped Ok(()) cases).

Example diffs:

--- a/lib/api/src/grpc/qdrant.rs
@@
 #[derive(validator::Validate)]
 #[derive(serde::Serialize)]
 #[allow(clippy::derive_partial_eq_without_eq)]
 #[derive(Clone, PartialEq, ::prost::Message)]
 pub struct FusionParams {
     #[prost(oneof = "fusion_params::Variant", tags = "1")]
+    #[validate(nested)]
     pub variant: ::core::option::Option<fusion_params::Variant>,
 }
--- a/lib/api/src/grpc/validate.rs
@@ impl Validate for super::qdrant::query::Variant {
-            grpc::query::Variant::FusionParams(_) => Ok(()),
+            grpc::query::Variant::FusionParams(q) => q.validate(),
             grpc::query::Variant::OrderBy(_)    => Ok(()),
⛔ Skipped due to learnings
Learnt from: coszio
PR: qdrant/qdrant#7065
File: lib/collection/src/operations/universal_query/shard_query.rs:0-0
Timestamp: 2025-08-15T15:56:36.821Z
Learning: Internal gRPC APIs in Qdrant don't require additional runtime validation when proto-level validation is already configured via build.rs range constraints.
src/common/inference/batch_processing_grpc.rs (1)

163-164: Exhaustive match: correctly no-ops for FusionParams during inference batching

Acknowledging FusionParams alongside Fusion as a no-op here is correct; fusion settings shouldn’t contribute inference inputs. This keeps the match exhaustive and behavior consistent.

lib/collection/benches/batch_query_bench.rs (1)

22-22: Importing DEFAULT_RRF_K keeps benches aligned with the new parameterized RRF

Good call to reference the canonical default from segment::common; avoids drift.

lib/collection/src/tests/shard_query.rs (1)

5-5: Tests updated to RrfK(DEFAULT_RRF_K): good coverage for the new enum

All RRF paths now exercise the parameterized variant with the default k, preserving prior semantics. Assertions remain meaningful and unchanged.

Also applies to: 62-62, 97-97, 144-144, 188-188

lib/collection/src/operations/universal_query/planned_query.rs (5)

433-433: Use of DEFAULT_RRF_K improves clarity and avoids magic numbers.

Importing DEFAULT_RRF_K rather than inlining 2 keeps tests aligned with the implementation default.


665-673: Tests updated to use RrfK(DEFAULT_RRF_K).

This matches the new parameterized FusionInternal variant and keeps expected behavior unchanged.


723-739: Error case test for fusion without prefetch remains valid.

Using FusionInternal::RrfK(DEFAULT_RRF_K) preserves the validation behavior.


769-782: REST params ignored when prefetch present: assertion remains correct.

Mapping to RrfK(DEFAULT_RRF_K) doesn’t affect the test’s intent.


985-1001: Nested fusion tests correctly migrated to RrfK(DEFAULT_RRF_K).

Great to see rescore expectations updated under the new variant.

Also applies to: 1036-1043

lib/collection/src/shards/local_shard/query.rs (3)

409-411: Correctly pass k into rrf_scoring for parameterized RRF.

The new FusionInternal::RrfK(k) arm plumbs k into the scorer, maintaining ordering and threshold logic below.


401-424: Retain score_threshold and limit semantics post-fusion.

Applying threshold with take_while after RRF sorting and before limit is consistent with previous behavior.


271-398: Upstream k ≥ 1 validation confirmed
Both REST and gRPC layers enforce a minimum of 1 for RRF’s k parameter, so this code path cannot receive k = 0:

  • lib/api/src/rest/schema.rs:539
    #[validate(range(min = 1))] pub k: Option<usize>

  • lib/api/src/grpc/qdrant.rs:5639
    #[validate(range(min = 1))] pub k: Option<u32>

Since k is guaranteed ≥ 1, no additional checks are needed in the rescoring implementation.

lib/collection/src/operations/universal_query/collection_query.rs (4)

714-723: Map legacy REST Fusion::Rrf to RrfK(DEFAULT_RRF_K).

This preserves backward compatibility while moving internals to the parameterized variant.


726-734: Add Fromrest::FusionParams → FusionInternal with defaulting.

Good defaulting to DEFAULT_RRF_K when k is None; ignoring r#type is fine in the tagged union.


736-745: Fromrest::FusionInterface cleanly handles both legacy and parameterized shapes.

This matches the API design discussed in the PR comments and keeps the call site simple.


668-706: Validate ‘using’ incompatibility with fusion unchanged.

Keeping the guard that Fusion queries cannot be combined with ‘using’ avoids undefined behavior.

lib/collection/src/collection/query.rs (2)

363-365: Correctly route parameterized RRF to rrf_scoring with k.

This brings collection-level fusion in sync with shard-level changes.


359-373: Preserve thresholding behavior for root fusion queries.

Threshold is applied after fusion and before offset/limit handling; consistent with previous semantics.

docs/redoc/master/openapi.json (2)

15012-15021: Union for FusionInterface is well-structured and backwards compatible.

Mixing string enum (Fusion) and params object (FusionParams) via anyOf matches existing schema patterns (e.g., WithPayloadInterface) and preserves legacy clients.


15056-15061: RrfType enum is clear and follows the repo’s tagged-union pattern.

No issues. This keeps the object form self-describing and future-safe.

lib/collection/src/operations/universal_query/shard_query.rs (8)

6-6: LGTM: Local DEFAULT_RRF_K import keeps behavior consistent across layers

Importing DEFAULT_RRF_K here ensures shard-level logic stays aligned with the segment-level default.


120-121: LGTM: RRF requires intermediate results

RrfK(_) correctly signals that prefetch/intermediate results are needed.


158-159: LGTM: Ordering is large-better for fusion

RrfK(_) correctly maps to descending order by score.


363-379: LGTM: FusionParams decoding, with sane defaulting

  • Handles missing variant with a clear InvalidArgument.
  • k: Option -> usize is safe.
  • None -> DEFAULT_RRF_K preserves legacy behavior.

557-557: LGTM: Legacy Fusion::Rrf maps to default k

Mapping Fusion::Rrf to RrfK(DEFAULT_RRF_K) preserves backward compatibility.


641-643: LGTM: Accept FusionParams on the inbound gRPC path

This cleanly integrates the new FusionParams variant without duplicating parsing logic.


710-710: LGTM: Delegate Fusion serialization via From

Nice reuse; keeps one source of truth for Fusion→gRPC encoding.


569-579: ✅ RRF k cast audit complete

No unexpected usize→u32 casts were found beyond the two intended instances:

  • lib/collection/src/operations/universal_query/shard_query.rs:
    • Line 576 – RrfParams { k: Some(k as u32) }
    • Line 600 – RrfParams { k: Some(k as u32) }

All other branches correctly handle DEFAULT_RRF_K without casting.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

♻️ Duplicate comments (1)
lib/collection/src/operations/universal_query/shard_query.rs (1)

556-574: Avoid usize → u32 truncation when serializing k to gRPC

Current code uses k as u32 which will silently truncate large usize values. Clamp into the u32 domain to preserve correctness.

Apply this diff in both conversions:

-            FusionInternal::RrfK(k) => Query {
-                variant: Some(QueryVariant::Rrf(Rrf { k: Some(k as u32) })),
+            FusionInternal::RrfK(k) => Query {
+                variant: Some(QueryVariant::Rrf(Rrf {
+                    k: Some(k.min(u32::MAX as usize) as u32),
+                })),
             },
-            FusionInternal::RrfK(k) => Query {
-                score: Some(Score::Rrf(Rrf { k: Some(k as u32) })),
+            FusionInternal::RrfK(k) => Query {
+                score: Some(Score::Rrf(Rrf {
+                    k: Some(k.min(u32::MAX as usize) as u32),
+                })),
             },

Notes:

  • From can’t fail; clamping is preferable to silent truncation.
  • Longer-term, making k a u32/NonZeroU32 internally would remove this edge entirely.

Also applies to: 576-595

🧹 Nitpick comments (10)
lib/api/src/grpc/proto/points.proto (2)

568-569: Clarify the “default parameters” note for RRF

Since a parameterized RRF is now available via Query.rrf, consider clarifying here that Fusion::RRF implies RRF with the default k (e.g., k=2). This will help clients choose between Fusion::RRF and the object form Query.rrf.

-    RRF = 0; // Reciprocal Rank Fusion (with default parameters)
+    RRF = 0; // Reciprocal Rank Fusion (uses default parameters, e.g. k=2)

672-676: Document k’s constraints and default

The field is optional and validated elsewhere; documenting the expected constraints here improves discoverability for SDK consumers.

-message Rrf {
-    optional uint32 k = 1; // K parameter for reciprocal rank fusion
-}
+message Rrf {
+    // K parameter for reciprocal rank fusion
+    // Must be >= 1. If not provided, the server uses a default (e.g., k=2).
+    optional uint32 k = 1;
+}
lib/collection/src/operations/universal_query/collection_query.rs (1)

726-731: Map REST RRF object to internal, but consider clamping/validating k

The REST layer validates k ≥ 1, but adding a small guard here would protect against accidental misuse if a future path bypassed REST validation.

-            FusionInternal::RrfK(k.unwrap_or(DEFAULT_RRF_K))
+            FusionInternal::RrfK(k.filter(|k| *k >= 1).unwrap_or(DEFAULT_RRF_K))
lib/api/src/rest/schema.rs (3)

519-524: Nit: consider documenting that this tag always deserializes to “rrf”

RrfType only has one variant, so requiring the field in JSON is a bit redundant unless you plan multiple RRF subtypes later.

Add a short doc comment, or make its presence optional (see next comment).


526-535: Make type optional and improve ergonomics; accept rrf_k as an alias

Requiring "type":"rrf" inside the object is redundant given the dedicated RRF wrapper. Also consider accepting rrf_k as an alias to ease migration from earlier examples.

 #[derive(Debug, Default, Serialize, Deserialize, JsonSchema, Validate)]
 #[serde(rename_all = "snake_case")]
 pub struct Rrf {
-    pub r#type: RrfType,
+    #[serde(default)]
+    pub r#type: RrfType,
 
     /// K parameter for reciprocal rank fusion
     #[validate(range(min = 1))]
-    #[serde(default)]
+    #[serde(default, alias = "rrf_k")]
     pub k: Option<usize>,
 }

537-542: Consider accepting both object shapes: { "type": "rrf", "k": ... } and { "rrf": { "k": ... } }

If you intend to support the nested/tagged shape discussed in the PR comments, you can accept both without breaking the existing object form.

 #[derive(Debug, Serialize, Deserialize, JsonSchema)]
 #[serde(untagged)]
 pub enum FusionInterface {
     Fusion(Fusion),
-    RRF(Rrf),
+    RRF(Rrf),
+    // Also accept the nested/tagged shape: { "rrf": { "k": ... } }
+    RrfTagged(RrfTagged),
 }

Add this helper outside of the changed hunk (for maintainers to apply):

#[derive(Debug, Serialize, Deserialize, JsonSchema)]
#[serde(rename_all = "snake_case")]
pub struct RrfTagged {
    pub rrf: Rrf,
}

Documentation: update examples to show both

  • "fusion": "rrf"
  • "fusion": { "type": "rrf", "k": 60 }
  • "fusion": { "rrf": { "k": 60 } }
lib/api/src/grpc/qdrant.rs (2)

5631-5641: Consider documenting the default k in the proto comment

Nit: The doc says “K parameter for reciprocal rank fusion” but doesn’t mention the default (2). Adding that in the proto comment (not here, since this file is generated) will help SDK users.


7123-7126: Clarify Fusion::Rrf docs — nice touch

The note “with default parameters” makes it clear when to use the enum vs. the parameterized message. Consider also mentioning the default k value in the proto docs for extra clarity.

lib/collection/src/operations/universal_query/shard_query.rs (2)

60-60: Consider encoding k’s invariants in the type (NonZeroU32/u32) instead of usize

Storing k as usize forces repeated lossy casts to u32 at the gRPC boundary. Using u32 or NonZeroU32 internally would:

  • Eliminate truncation risk at serialization time.
  • Encode the k ≥ 1 invariant at the type level (if NonZeroU32), aligning with proto validation.

363-372: LGTM: Robust conversion from gRPC Rrf, defaulting to DEFAULT_RRF_K

Mapping Option -> usize with a default is fine here. Style nit: usize::from(k) avoids an as cast, but current code is clear enough.

📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

💡 Knowledge Base configuration:

  • MCP integration is disabled by default for public repositories
  • Jira integration is disabled by default for public repositories
  • Linear integration is disabled by default for public repositories

You can enable these sources in your CodeRabbit configuration.

📥 Commits

Reviewing files that changed from the base of the PR and between e862414 and 0206078.

📒 Files selected for processing (11)
  • lib/api/build.rs (1 hunks)
  • lib/api/src/grpc/proto/points.proto (3 hunks)
  • lib/api/src/grpc/proto/points_internal_service.proto (1 hunks)
  • lib/api/src/grpc/qdrant.rs (5 hunks)
  • lib/api/src/grpc/validate.rs (1 hunks)
  • lib/api/src/rest/schema.rs (2 hunks)
  • lib/api/src/rest/validate.rs (2 hunks)
  • lib/collection/src/operations/universal_query/collection_query.rs (1 hunks)
  • lib/collection/src/operations/universal_query/shard_query.rs (8 hunks)
  • src/common/inference/batch_processing_grpc.rs (1 hunks)
  • src/common/inference/query_requests_grpc.rs (1 hunks)
🚧 Files skipped from review as they are similar to previous changes (4)
  • lib/api/src/grpc/proto/points_internal_service.proto
  • lib/api/build.rs
  • lib/api/src/grpc/validate.rs
  • src/common/inference/batch_processing_grpc.rs
🧰 Additional context used
🧠 Learnings (5)
📚 Learning: 2025-08-10T18:31:56.855Z
Learnt from: generall
PR: qdrant/qdrant#7006
File: lib/collection/src/operations/point_ops.rs:501-528
Timestamp: 2025-08-10T18:31:56.855Z
Learning: In Qdrant, batch operations validate that `ids`, `vectors`, and `payloads` (if present) have matching lengths at the REST API level in `lib/api/src/rest/validate.rs` through the `Validate` trait implementation for `Batch`. This validation happens before data is converted to internal structures like `PointInsertOperationsInternal`, so methods operating on these internal structures can safely assume the lengths match.

Applied to files:

  • lib/api/src/rest/validate.rs
📚 Learning: 2025-08-10T18:26:12.443Z
Learnt from: generall
PR: qdrant/qdrant#7006
File: docs/redoc/master/openapi.json:13645-13652
Timestamp: 2025-08-10T18:26:12.443Z
Learning: In qdrant, docs/redoc/master/openapi.json is auto-generated from the REST schemas. To change field docs, edit lib/api/src/rest/schema.rs (e.g., add doc comments or #[schemars(description = ...)]). Specifically, UpdateVectors.update_filter lacked a description and should state: "If specified, only update vectors for points that match this filter; points not matching the filter are left unchanged."

Applied to files:

  • lib/api/src/rest/schema.rs
📚 Learning: 2025-08-10T18:26:33.017Z
Learnt from: generall
PR: qdrant/qdrant#7006
File: docs/redoc/master/openapi.json:10393-10401
Timestamp: 2025-08-10T18:26:33.017Z
Learning: In qdrant, docs/redoc/master/openapi.json is auto-generated. Any description changes must be made in the source (e.g., lib/api/src/rest/schema.rs and/or lib/api/src/grpc/proto/points.proto), not edited directly in the JSON.

Applied to files:

  • lib/api/src/rest/schema.rs
📚 Learning: 2025-08-10T18:25:16.206Z
Learnt from: generall
PR: qdrant/qdrant#7006
File: docs/redoc/master/openapi.json:10626-10634
Timestamp: 2025-08-10T18:25:16.206Z
Learning: In Qdrant, docs/redoc/master/openapi.json is autogenerated from the Rust REST schema definitions (e.g., lib/api/src/rest/schema.rs). To change OpenAPI descriptions, update the Rust struct field doc comments/attributes rather than editing the JSON directly.

Applied to files:

  • lib/api/src/rest/schema.rs
📚 Learning: 2025-08-15T15:56:36.821Z
Learnt from: coszio
PR: qdrant/qdrant#7065
File: lib/collection/src/operations/universal_query/shard_query.rs:0-0
Timestamp: 2025-08-15T15:56:36.821Z
Learning: Internal gRPC APIs in Qdrant don't require additional runtime validation when proto-level validation is already configured via build.rs range constraints.

Applied to files:

  • lib/collection/src/operations/universal_query/shard_query.rs
🧬 Code Graph Analysis (5)
src/common/inference/query_requests_grpc.rs (1)
lib/collection/src/operations/universal_query/shard_query.rs (7)
  • try_from (228-265)
  • try_from (271-297)
  • try_from (354-360)
  • try_from (366-371)
  • try_from (377-383)
  • try_from (389-411)
  • try_from (417-520)
lib/api/src/rest/validate.rs (1)
lib/api/src/grpc/validate.rs (13)
  • validate (14-14)
  • validate (19-21)
  • validate (29-31)
  • validate (39-41)
  • validate (49-54)
  • validate (58-64)
  • validate (68-74)
  • validate (78-85)
  • validate (89-97)
  • validate (101-112)
  • validate (116-123)
  • validate (127-134)
  • validate (138-145)
lib/collection/src/operations/universal_query/collection_query.rs (1)
lib/collection/src/operations/universal_query/shard_query.rs (9)
  • from (548-553)
  • from (557-573)
  • from (577-594)
  • from (598-602)
  • from (606-610)
  • from (664-686)
  • from (690-720)
  • from (724-744)
  • from (748-778)
lib/api/src/rest/schema.rs (1)
lib/api/src/rest/validate.rs (15)
  • validate (15-21)
  • validate (25-30)
  • validate (34-45)
  • validate (49-59)
  • validate (63-83)
  • validate (87-93)
  • validate (97-101)
  • validate (105-110)
  • validate (114-142)
  • validate (146-151)
  • validate (155-159)
  • validate (163-179)
  • validate (183-241)
  • validate (245-256)
  • validate (260-282)
lib/collection/src/operations/universal_query/shard_query.rs (2)
tests/openapi/helpers/helpers.py (1)
  • reciprocal_rank_fusion (120-145)
lib/collection/src/operations/universal_query/collection_query.rs (4)
  • from (718-723)
  • from (727-730)
  • from (734-739)
  • from (743-747)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (12)
  • GitHub Check: integration-tests-consensus
  • GitHub Check: integration-tests
  • GitHub Check: test-shard-snapshot-api-s3-minio
  • GitHub Check: e2e-tests
  • GitHub Check: rust-tests (macos-latest)
  • GitHub Check: test-consistency
  • GitHub Check: test-consensus-compose
  • GitHub Check: rust-tests (windows-latest)
  • GitHub Check: rust-tests-no-rocksdb (ubuntu-latest)
  • GitHub Check: rust-tests (ubuntu-latest)
  • GitHub Check: lint
  • GitHub Check: storage-compat-test
🔇 Additional comments (17)
lib/api/src/grpc/proto/points.proto (1)

688-689: Nested validation for Query::Rrf is already wired
The impl Validate for super::qdrant::query::Variant in lib/api/src/grpc/validate.rs includes:

    grpc::query::Variant::Rrf(q) => q.validate(),

No further changes are required.

lib/collection/src/operations/universal_query/collection_query.rs (3)

713-713: Good: centralize default via DEFAULT_RRF_K

Importing DEFAULT_RRF_K keeps the REST mapping aligned with the segment implementation’s canonical default.


720-723: Backwards compatibility preserved for string “rrf”

Mapping Fusion::Rrf to RrfK(DEFAULT_RRF_K) preserves prior behavior for the shorthand string form.


733-740: Accept both Fusion and parameterized RRF interfaces

This is the right abstraction to support both “rrf” (default k) and object RRF (custom k). Make sure docs/examples show both shapes.

lib/api/src/rest/validate.rs (2)

12-12: Import looks fine

Bringing FusionInterface into scope is required by the new Validate impl below.


104-111: Validate wrapper delegates correctly to inner variants

This ensures the object form of RRF gets its k validated (via the derived Validate on the Rrf struct).

lib/api/src/rest/schema.rs (1)

701-702: Schema switch to FusionInterface is correct

This unlocks both the string and object forms for fusion. Ensure OpenAPI examples include the parameterized RRF object to make the new capability visible.

lib/api/src/grpc/qdrant.rs (4)

5631-5641: Add parameterized RRF message (k ≥ 1) — looks correct

The new Rrf message with an optional k and validator range(min = 1) is appropriate. Using Option keeps backward compatibility (default handled elsewhere).


5647-5650: Extend Query oneof tags to include Rrf (tag 10) — good

The tag set update matches the new Variant::Rrf addition and maintains uniqueness across 1–10.


10178-10208: ✅ Shard-level RRF scoring path validated

  • The new Score::Rrf (tag 7) is correctly deserialized into FusionInternal::RrfK(k) in
    lib/collection/src/operations/universal_query/shard_query.rs:367-370.
  • Both shard and collection query engines invoke rrf_scoring(..., k):
    lib/collection/src/shards/local_shard/query.rs:409
    lib/collection/src/collection/query.rs:363
  • Defaulting of k when absent is handled via .unwrap_or(DEFAULT_RRF_K) in both conversion layers.

All routes from the gRPC tag through to rrf_scoring with the provided (or default) k are in place.


5684-5687: ✅ End-to-end Rrf variant handling confirmed

All of the following checks are in place for the new Query::Variant::Rrf (tag 10):

• gRPC validation:
lib/api/src/grpc/validate.rs handles grpc::query::Variant::Rrf(q) => q.validate()
– Validator rules in build.rs include ("Rrf.k", "range(min = 1)")

• gRPC⇄internal conversion:
– Inference request mapping (common/inference/query_requests_grpc.rs):
Variant::Rrf(rrf) => Query::Fusion(FusionInternal::try_from(rrf)?)
TryFrom<grpc::Rrf> for FusionInternal (shard_query.rs) produces FusionInternal::RrfK(k…)
– Reverse mapping emits QueryVariant::Rrf(Rrf { k: Some(k) })

• REST support:
From<rest::Rrf> for FusionInternal correctly translates the optional k
– REST validator (lib/api/src/rest/validate.rs) includes Fusion::Rrf

• Protobuf definitions:
points.proto defines message Rrf { optional uint32 k = 1; }
oneof variant in points.proto contains Rrf rrf = 10;

No further changes needed.

lib/collection/src/operations/universal_query/shard_query.rs (6)

6-6: LGTM: Centralized default is correctly imported

Pulling in DEFAULT_RRF_K avoids magic numbers and keeps defaults consistent across layers.


120-123: LGTM: Correctly marks RRF as requiring intermediate results

RrfK needs per-prefetch ranking info; returning true here is correct and keeps behavior aligned with DBFS.


158-159: LGTM: Ordering semantics preserved for RRF

FusionInternal::RrfK uses descending order (large-better), consistent with score-boosting semantics.


550-552: LGTM: Back-compat default mapping (Rrf → RrfK(DEFAULT_RRF_K))

Keeps wire format stable for the default case to avoid breaking rolling upgrades.


628-630: LGTM: Dedicated Rrf score path handled

Accepting Score::Rrf(rrf) and reusing FusionInternal::try_from(rrf) keeps the parsing centralized and symmetric with the Query path. Given the proto-level range(min=1) validation, no extra runtime checks here are necessary.


697-697: LGTM: Reuse the FusionInternal → gRPC conversion

Delegating ScoringQuery::Fusion to Self::from(fusion) avoids duplication and stays in sync with serialization logic.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Nitpick comments (3)
docs/grpc/docs.md (3)

250-250: Add Rrf to TOC — looks correct; watch markdownlint MD007 on this line

The anchor and placement are correct. markdownlint flagged ul-indent (expected 2 spaces, found 4). Since this file appears generated from proto docs, prefer ignoring markdownlint for generated docs or disabling MD007 for this path rather than hand-tweaking a single line which would make the TOC inconsistent with the rest.

If you want to silence the linter globally for generated gRPC docs, add docs/grpc/docs.md to the linter ignore, or disable MD007 for this path in your markdownlint config.


4015-4016: Clarify the purpose and defaults for the new rrf field on Query

Consider documenting that this field enables configurable RRF, its default when k is omitted, and the valid range for k.

Apply this diff to improve clarity (ideally via proto comments and re-generating this file):

-| rrf | [Rrf](#qdrant-Rrf) |  | Parameterized RRF fusion |
+| rrf | [Rrf](#qdrant-Rrf) |  | Reciprocal Rank Fusion with configurable k (defaults to 2; k must be ≥ 1) |

Note: If this file is generated, update the comment on the rrf field in points.proto and re-generate docs.


4402-4411: Document default and validation for Rrf.k

The new Rrf message is useful. Please surface the default (2) and the validation bound (k ≥ 1) to prevent guesswork by API consumers.

Apply this diff (preferably by changing proto comments and re-generating):

-### Rrf
-Parameterized reciprocal rank fusion
+### Rrf
+Reciprocal Rank Fusion (RRF) with configurable smoothing parameter.
@@
-| k | [uint32](#uint32) | optional | K parameter for reciprocal rank fusion |
+| k | [uint32](#uint32) | optional | Smoothing parameter k for RRF. Optional; defaults to 2 if unset. Must be ≥ 1. |
📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

💡 Knowledge Base configuration:

  • MCP integration is disabled by default for public repositories
  • Jira integration is disabled by default for public repositories
  • Linear integration is disabled by default for public repositories

You can enable these sources in your CodeRabbit configuration.

📥 Commits

Reviewing files that changed from the base of the PR and between 0206078 and 5986c0f.

📒 Files selected for processing (2)
  • docs/grpc/docs.md (4 hunks)
  • docs/redoc/master/openapi.json (2 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
  • docs/redoc/master/openapi.json
🧰 Additional context used
🪛 markdownlint-cli2 (0.17.2)
docs/grpc/docs.md

250-250: Unordered list indentation
Expected: 2; Actual: 4

(MD007, ul-indent)

🔇 Additional comments (1)
docs/grpc/docs.md (1)

5236-5236: Enum doc tweak for Fusion.RRF — LGTM

“Reciprocal Rank Fusion (with default parameters)” accurately distinguishes enum-based default RRF from the parameterized Rrf message. No further changes needed here.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (10)
lib/api/src/grpc/qdrant.rs (3)

5631-5641: Add RRF message with validation: good shape and semantics

Optional k with min ≥ 1 is appropriate and keeps backward compatibility by allowing omission (use default). One nit: consider documenting the default value for k here (e.g., “If omitted, defaults to 2”) so it propagates into generated docs for clients. That doc should be added in the .proto, not this generated file.


7123-7127: Doc tweak clarifies legacy Fusion enum usage

Updating the Fusion enum doc to “Reciprocal Rank Fusion (with default parameters)” is correct given the new parameterized path. Consider adding a short cross-reference in the .proto docs (not here) indicating “for parameterized RRF use the Query::Rrf message.”


5637-5641: Optional cap for k (non-blocking suggestion)

If you want guardrails against extreme values (e.g., accidental k=0 already blocked, but very large k could flatten weights), consider adding an upper bound in the validator (e.g., max = 65535). This is optional; the current type is correct.

docs/redoc/master/openapi.json (3)

15012-15021: Use oneOf instead of anyOf for FusionInterface to improve validation and codegen.

Only one of the two shapes can ever match (string enum vs object), so oneOf is a better semantic fit and yields clearer SDK types.

Apply this diff:

-      "FusionInterface": {
-        "anyOf": [
+      "FusionInterface": {
+        "oneOf": [
           {
             "$ref": "#/components/schemas/Fusion"
           },
           {
             "$ref": "#/components/schemas/Rrf"
           }
-        ]
+        ]
       },

15030-15048: Document and surface the default K (2) for RRF in the schema; add example for clarity.

Server defaults to K=2 if omitted. Reflecting this in OpenAPI avoids ambiguity and improves SDK DX. Keep nullable: true to preserve the ability to explicitly send null while code-generated clients still know the default when omitted.

Apply this diff:

       "Rrf": {
         "type": "object",
         "required": [
           "type"
         ],
         "properties": {
           "type": {
             "$ref": "#/components/schemas/RrfType"
           },
           "k": {
-            "description": "K parameter for reciprocal rank fusion",
-            "default": null,
+            "description": "K parameter for Reciprocal Rank Fusion. If omitted, defaults to 2.",
+            "default": 2,
             "type": "integer",
             "format": "uint",
             "minimum": 1,
             "nullable": true
           }
         }
+        ,
+        "example": {
+          "type": "rrf",
+          "k": 60
+        }
       },

15049-15054: Optional: Inline RrfType to reduce schema surface area.

RrfType is a single-value enum used only here. Consider inlining to cut indirection. Fine to keep as-is if it matches your schema style.

Example inline alternative (no change required if you prefer the current style):

-          "type": {
-            "$ref": "#/components/schemas/RrfType"
-          },
+          "type": {
+            "type": "string",
+            "enum": ["rrf"]
+          },
docs/grpc/docs.md (4)

250-250: TOC entry for Rrf looks correct; consider silencing markdownlint MD007 for generated docs

The anchor resolves to the new Rrf section. markdownlint flagged list indentation here (MD007), but indentation matches surrounding generated entries. Prefer excluding generated gRPC docs from lint or configuring MD007 rather than hand-editing this file.

If you want to silence the linter for generated files, add an ignore to your markdownlint config (example for markdownlint-cli2):

# .markdownlint-cli2.yaml
+globsIgnore:
+  - "docs/grpc/**"

4015-4016: Clarify precedence between Query.fusion (RRF) and Query.rrf

There are now two ways to request RRF: the Fusion enum with default parameters and the parameterized Rrf field. Please document precedence/mutual exclusivity (e.g., if rrf is set, it overrides Fusion=RRF, or they are mutually exclusive).

Since this file is generated from .proto comments, update the proto doc comments and regenerate. For example, adjust the field description:

-| rrf | [Rrf](#qdrant-Rrf) |  | Parameterized RRF fusion |
+| rrf | [Rrf](#qdrant-Rrf) |  | Parameterized RRF fusion. If provided, it takes precedence over `fusion = RRF` (default-parameter RRF). |

Would you like me to propose concrete .proto comment changes once you confirm the intended precedence?


5236-5236: Enum description tweak: cross-reference the parameterized Rrf

The “with default parameters” note is good. Consider adding a short pointer to the new Rrf message so users discover how to customize k.

Apply via proto comments (regenerate docs):

-| RRF | 0 | Reciprocal Rank Fusion (with default parameters) |
+| RRF | 0 | Reciprocal Rank Fusion (with default parameters). See [Rrf](#qdrant-Rrf) to customize `k`. |

4402-4411: Document k constraints and default in Rrf proto

To surface validation and default behavior in the generated docs, update the .proto comment for k in lib/api/src/grpc/proto/points.proto:

  • File: lib/api/src/grpc/proto/points.proto
     message Rrf {
  • optional uint32 k = 1; // K parameter for reciprocal rank fusion
  • optional uint32 k = 1; // K parameter for reciprocal rank fusion (must be ≥ 1, defaults to 2)
    }

This change ensures the docs show:
- “k must be ≥ 1” (enforced by `build.rs`/validators), and  
- “default = 2” (from `DEFAULT_RRF_K`)  

After updating this comment, regenerate the gRPC docs to apply.

</blockquote></details>

</blockquote></details>

<details>
<summary>📜 Review details</summary>

**Configuration used: CodeRabbit UI**
**Review profile: CHILL**
**Plan: Pro**

**💡 Knowledge Base configuration:**

- MCP integration is disabled by default for public repositories
- Jira integration is disabled by default for public repositories
- Linear integration is disabled by default for public repositories

You can enable these sources in your CodeRabbit configuration.

<details>
<summary>📥 Commits</summary>

Reviewing files that changed from the base of the PR and between 5986c0fdca6de1b02619066b753a0a4e66f2abd3 and a018d669ced233d13119bb314351849c1fc85141.

</details>

<details>
<summary>📒 Files selected for processing (19)</summary>

* `docs/grpc/docs.md` (4 hunks)
* `docs/redoc/master/openapi.json` (2 hunks)
* `lib/api/build.rs` (1 hunks)
* `lib/api/src/grpc/proto/points.proto` (3 hunks)
* `lib/api/src/grpc/proto/points_internal_service.proto` (1 hunks)
* `lib/api/src/grpc/qdrant.rs` (5 hunks)
* `lib/api/src/grpc/validate.rs` (1 hunks)
* `lib/api/src/rest/schema.rs` (2 hunks)
* `lib/api/src/rest/validate.rs` (2 hunks)
* `lib/collection/benches/batch_query_bench.rs` (2 hunks)
* `lib/collection/src/collection/query.rs` (1 hunks)
* `lib/collection/src/operations/universal_query/collection_query.rs` (1 hunks)
* `lib/collection/src/operations/universal_query/planned_query.rs` (6 hunks)
* `lib/collection/src/operations/universal_query/shard_query.rs` (8 hunks)
* `lib/collection/src/shards/local_shard/query.rs` (1 hunks)
* `lib/collection/src/tests/shard_query.rs` (5 hunks)
* `lib/segment/src/common/reciprocal_rank_fusion.rs` (4 hunks)
* `src/common/inference/batch_processing_grpc.rs` (1 hunks)
* `src/common/inference/query_requests_grpc.rs` (1 hunks)

</details>

<details>
<summary>🚧 Files skipped from review as they are similar to previous changes (16)</summary>

* lib/api/src/grpc/proto/points_internal_service.proto
* lib/collection/src/collection/query.rs
* lib/api/src/grpc/validate.rs
* src/common/inference/query_requests_grpc.rs
* lib/collection/benches/batch_query_bench.rs
* lib/api/src/grpc/proto/points.proto
* lib/api/build.rs
* lib/collection/src/operations/universal_query/planned_query.rs
* src/common/inference/batch_processing_grpc.rs
* lib/collection/src/operations/universal_query/collection_query.rs
* lib/collection/src/tests/shard_query.rs
* lib/api/src/rest/validate.rs
* lib/collection/src/shards/local_shard/query.rs
* lib/segment/src/common/reciprocal_rank_fusion.rs
* lib/api/src/rest/schema.rs
* lib/collection/src/operations/universal_query/shard_query.rs

</details>

<details>
<summary>🧰 Additional context used</summary>

<details>
<summary>🪛 markdownlint-cli2 (0.17.2)</summary>

<details>
<summary>docs/grpc/docs.md</summary>

250-250: Unordered list indentation
Expected: 2; Actual: 4

(MD007, ul-indent)

</details>

</details>

</details>

<details>
<summary>🔇 Additional comments (3)</summary><blockquote>

<details>
<summary>lib/api/src/grpc/qdrant.rs (2)</summary>

`10177-10180`: **Extend QueryShardPoints::Query.Score tags to include 7: OK**

Tag addition looks correct and maintains existing tags. No collisions observed.

---

`5647-5649`: **Validation for Rrf is wired, conversions need manual confirmation**

- lib/api/src/grpc/validate.rs includes  
`grpc::query::Variant::Rrf(q) => q.validate()`  
so inbound validation is in place.

I couldn’t locate the gRPC↔internal conversion implementations for the new Rrf variant. Please verify (or add) in your conversion modules that:

• Inbound (gRPC→internal):  
  grpc::query::Variant::Rrf(q)  
    → `FusionInternal::RrfK(q.k.unwrap_or(DEFAULT_RRF_K))`

• Outbound (internal→gRPC), for both Query and QueryShardPoints:  
  – `FusionInternal::RrfK(k) if k == DEFAULT_RRF_K`  
     → `grpc::query::Variant::Rrf`  
  – `FusionInternal::RrfK(k) if k != DEFAULT_RRF_K`  
     → `grpc::query::Variant::Fusion(Rrf(k as i32))`

Ensure these mappings live alongside your other `TryFrom`/`From` impls (e.g. in lib/api/src/grpc/convert_*.rs).

</details>
<details>
<summary>docs/redoc/master/openapi.json (1)</summary>

`15007-15010`: **LGTM: FusionQuery now references FusionInterface (backward compatible).**

This cleanly enables both the legacy string enum and the new object form without breaking existing clients.

</details>

</blockquote></details>

</details>

<!-- This is an auto-generated comment by CodeRabbit for review status -->

Comment on lines +10206 to 10208
#[prost(message, tag = "7")]
Rrf(super::super::Rrf),
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Verification agent

🧩 Analysis chain

Add QueryShardPoints::Query::Score::Rrf variant: OK; ensure mapping/validation

gRPC internal query path now supports parameterized RRF. Please confirm:

  • Validation for Rrf.k is invoked in the internal shard path
  • Conversion to internal scoring passes the resolved k to rrf_scoring

🏁 Script executed:

#!/bin/bash
set -euo pipefail

# Ensure shard query conversions handle Rrf
rg -n -C3 'query_shard_points::query::Score::Rrf' lib -g '!**/target/**' || true

# Ensure rrf_scoring takes k and is fed from the new path
rg -n -C3 'rrf_scoring\s*\(' lib -g '!**/target/**' || true

Length of output: 5368


🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo "=== FusionInternal enum definition ==="
rg -n -C5 'enum FusionInternal' lib/collection/src/operations/universal_query/shard_query.rs

echo
echo "=== TryFrom impls in shard_query.rs ==="
rg -n -C5 'impl TryFrom' lib/collection/src/operations/universal_query/shard_query.rs

Length of output: 2997


Add RRF variant mapping OK; add validation for k

The new Score::Rrf arm correctly maps into FusionInternal::RrfK and the resolved k is passed into rrf_scoring. However, there’s no guard against invalid values (e.g. zero or negative). Please add early validation in the TryFrom<grpc::Rrf> impl:

• File: lib/collection/src/operations/universal_query/shard_query.rs
Location: impl TryFrom<grpc::Rrf> for FusionInternal (around lines 363–371)

• Replace:

let grpc::Rrf { k } = rrf;
Ok(FusionInternal::RrfK(
    k.map(|k| k as usize).unwrap_or(DEFAULT_RRF_K),
))

With:

let grpc::Rrf { k } = rrf;
let resolved_k = match k {
    Some(k) if k > 0 => k as usize,
    Some(k)        => return Err(Status::invalid_argument(
                          format!("invalid RRF k {}; must be > 0", k)
                       )),
    None           => DEFAULT_RRF_K,
};
Ok(FusionInternal::RrfK(resolved_k))

This ensures clients can’t pass an invalid k, while still defaulting when it’s omitted.

🤖 Prompt for AI Agents
In lib/collection/src/operations/universal_query/shard_query.rs around lines 363
to 371, the TryFrom<grpc::Rrf> impl currently maps k to FusionInternal::RrfK
without validating its value; change it to validate k: destructure grpc::Rrf { k
}, then match k to produce resolved_k where Some(k) if k > 0 maps to k as usize,
Some(k) with k <= 0 returns Err(Status::invalid_argument(...)) with a message
like "invalid RRF k {}; must be > 0", and None yields DEFAULT_RRF_K; finally
return Ok(FusionInternal::RrfK(resolved_k)).

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

♻️ Duplicate comments (1)
lib/api/src/grpc/qdrant.rs (1)

5631-5641: RRF message + k validation added: looks good; confirm upstream defaulting and guardrails

Introducing Rrf { k: Option<u32> } with #[validate(range(min = 1))] is correct for payload validation. Please ensure:

  • None is resolved to DEFAULT_RRF_K.
  • Some(0) (or any non-positive) is rejected in the internal conversion path as previously discussed.

This was raised earlier for TryFrom<grpc::Rrf> for FusionInternal and still applies here.

#!/bin/bash
set -euo pipefail

# 1) Validate that gRPC validators call q.validate() for the new Query::Rrf path (top-level)
rg -n -C3 'query::Variant::Rrf\(' lib/api/src/grpc/validate.rs || true

# 2) Validate that gRPC validators call q.validate() for the shard scoring path
rg -n -C3 'query::Score::Rrf\(' lib/api/src/grpc/validate.rs || true

# 3) Check TryFrom<grpc::Rrf> mapping validates k > 0,
#    defaults to DEFAULT_RRF_K when None, and maps to FusionInternal::RrfK
rg -n -C5 'impl\s+TryFrom<.*grpc::Rrf.*>\s+for\s+.*' lib -g '!**/target/**' || true
rg -n -C5 'FusionInternal::RrfK' lib/collection/src/operations/universal_query/shard_query.rs || true
rg -n -C5 'DEFAULT_RRF_K' lib/collection/src/operations/universal_query/shard_query.rs || true

# 4) Ensure rrf_scoring takes k and is fed properly
rg -n -C3 'rrf_scoring\s*\(' lib -g '!**/target/**' || true
🧹 Nitpick comments (10)
lib/api/src/grpc/qdrant.rs (1)

7123-7123: Doc nit: consider stating the actual default k (2) explicitly

The comment “with default parameters” could be made more explicit to reduce ambiguity, e.g., “with default k=2”.

docs/grpc/docs.md (4)

250-250: ToC entry for Rrf looks good; watch markdownlint MD007 (indent).

The new entry links to the correct anchor. However, markdownlint flagged unordered list indentation at this position (Expected: 2; Actual: 4). If this file is generated, consider excluding docs/grpc from linting or adjusting the rule; otherwise, re-indent to satisfy MD007.


4015-4016: Clarify how Query.rrf interacts with Query.fusion (precedence and defaults).

Having both a Fusion enum and an Rrf message can confuse users about which one to set and what happens if both are provided. Recommend stating that rrf enables RRF with an optional k, that the default is used when unspecified, and which takes precedence when both are set.

Apply this diff to expand the description inline:

-| rrf | [Rrf](#qdrant-Rrf) |  | Parameterized reciprocal rank fusion |
+| rrf | [Rrf](#qdrant-Rrf) |  | Parameterized reciprocal rank fusion. When set, enables RRF with optional k. If omitted and `fusion` = RRF is set, the default k is used. If both are set, `rrf.k` takes precedence. |

4404-4411: Document validation and default for Rrf.k (min and default).

The docs should state constraints and defaults to prevent misuse and align with the implementation (k ≥ 1, default k = 2).

Apply this diff:

 ### Rrf
-Parameterized reciprocal rank fusion
+Parameterized reciprocal rank fusion
+
+Default: k = 2. Validation: k must be >= 1.

 | Field | Type | Label | Description |
 | ----- | ---- | ----- | ----------- |
-| k | [uint32](#uint32) | optional | K parameter for reciprocal rank fusion |
+| k | [uint32](#uint32) | optional | K parameter for reciprocal rank fusion (min: 1, default: 2) |

5236-5236: Make Fusion enum doc explicit about the default k.

Since RRF now has a configurable k, calling out the default value in the enum description improves clarity.

Apply this diff:

-| RRF | 0 | Reciprocal Rank Fusion (with default parameters) |
+| RRF | 0 | Reciprocal Rank Fusion (default k = 2) |
docs/redoc/master/openapi.json (4)

14813-14815: Query shape for parameterized RRF diverges from PR examples—confirm intended UX and update docs accordingly

The schema adds RrfQuery as a top-level query variant ({ "rrf": { "k": ... } }). The PR text and discussion also mention shapes under fusion, e.g.:

  • {"query":{"fusion":{"type":"rrf","k":60}}} (tagged)
  • or nested {"query":{"fusion":{"rrf":{"k":123}}}}.

As modeled, the valid payloads per this OpenAPI are:

  • Shorthand: {"query":{"fusion":"rrf"}}
  • Parameterized: {"query":{"rrf":{"k":60}}}

If this is the finalized shape, please align the PR description and docs with the top-level rrf variant. If instead the goal is to nest RRF under fusion, the schema should be refactored to model a tagged/nested union under fusion rather than a separate RrfQuery variant.


15016-15021: Update Fusion description to reflect configurable RRF k (default 2), not fixed k=2

The current description still says “rrf — ... (with k=2)”, which can mislead users now that k is configurable.

Apply this diff to adjust the description:

-        "description": "Fusion algorithm allows to combine results of multiple prefetches.\n\nAvailable fusion algorithms:\n\n* `rrf` - Reciprocal Rank Fusion (with k=2) * `dbsf` - Distribution-Based Score Fusion",
+        "description": "Fusion algorithm allows combining results of multiple prefetches.\n\nAvailable fusion algorithms:\n\n* `rrf` - Reciprocal Rank Fusion (k configurable, default 2)\n* `dbsf` - Distribution-Based Score Fusion",

15023-15033: Add a short description to RrfQuery for clarity in generated docs/SDKs

Adding a description helps generated clients and documentation.

       "RrfQuery": {
+        "description": "Perform Reciprocal Rank Fusion with optional parameters.",
         "type": "object",
         "required": [
           "rrf"
         ],
         "properties": {
           "rrf": {
             "$ref": "#/components/schemas/Rrf"
           }
         }
       },

15035-15047: Consider documenting the default k=2 on the Rrf schema (and optionally set default to 2)

Server-side defaults to k=2 when omitted. Reflecting that here improves client UX and codegen. Option A only updates description; Option B also sets a default.

Option A (description only):

       "Rrf": {
         "description": "Parameters for Reciprocal Rank Fusion",
         "type": "object",
         "properties": {
           "k": {
-            "description": "K parameter for reciprocal rank fusion",
+            "description": "K parameter for reciprocal rank fusion. Default is 2.",
             "default": null,
             "type": "integer",
             "format": "uint",
             "minimum": 1,
             "nullable": true
           }
         }
       },

Option B (set default, keep nullable if server accepts explicit null):

           "k": {
-            "description": "K parameter for reciprocal rank fusion",
-            "default": null,
+            "description": "K parameter for reciprocal rank fusion. Default is 2.",
+            "default": 2,
             "type": "integer",
             "format": "uint",
             "minimum": 1,
             "nullable": true
           }

If the server treats explicit null the same as omission (i.e., use default), Option B is safe; otherwise prefer Option A.

src/common/inference/query_requests_rest.rs (1)

269-270: Add conversion tests for RRF (custom k and default parity)

To lock in the behavior of both payload shapes and ensure defaulting remains consistent, please append the following tests to the bottom of the tests module in src/common/inference/query_requests_rest.rs:

#[test]
fn test_convert_query_with_inferred_rrf_custom_k() {
    let inferred = create_test_inferred_batch();
    let query = rest::QueryInterface::Query(rest::Query::Rrf(rest::RrfQuery {
        rrf: rest::Rrf { k: Some(60) },
    }));

    let result = super::convert_query_with_inferred(query, &inferred).unwrap();
    match result {
        Query::Fusion(FusionInternal::RrfK(k)) => assert_eq!(k, 60),
        other => panic!("Expected Fusion::RrfK(60), got {:?}", other),
    }
}

#[test]
fn test_convert_query_with_inferred_rrf_default_matches_shorthand() {
    let inferred = create_test_inferred_batch();

    // Nested RRF with no k (should use default)
    let nested = rest::QueryInterface::Query(rest::Query::Rrf(rest::RrfQuery {
        rrf: rest::Rrf { k: None },
    }));

    // Shorthand "fusion": "rrf" (should use the same default)
    let shorthand = rest::QueryInterface::Query(rest::Query::Fusion(rest::FusionQuery {
        fusion: rest::Fusion::Rrf,
    }));

    let nested_res = super::convert_query_with_inferred(nested, &inferred).unwrap();
    let shorthand_res = super::convert_query_with_inferred(shorthand, &inferred).unwrap();

    let k_nested = match nested_res {
        Query::Fusion(FusionInternal::RrfK(k)) => k,
        other => panic!("Expected Fusion::RrfK(_), got {:?}", other),
    };
    let k_short = match shorthand_res {
        Query::Fusion(FusionInternal::RrfK(k)) => k,
        other => panic!("Expected Fusion::RrfK(_), got {:?}", other),
    };

    assert_eq!(k_nested, k_short, "Default k must be equal across both shapes");
}

These tests will ensure:

  • A custom k is passed through correctly.
  • The default k used in nested RRF matches the default when using the Fusion shorthand.
📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

💡 Knowledge Base configuration:

  • MCP integration is disabled by default for public repositories
  • Jira integration is disabled by default for public repositories
  • Linear integration is disabled by default for public repositories

You can enable these sources in your CodeRabbit configuration.

📥 Commits

Reviewing files that changed from the base of the PR and between a018d66 and f01ec0b.

📒 Files selected for processing (9)
  • docs/grpc/docs.md (4 hunks)
  • docs/redoc/master/openapi.json (2 hunks)
  • lib/api/src/grpc/proto/points.proto (3 hunks)
  • lib/api/src/grpc/qdrant.rs (5 hunks)
  • lib/api/src/rest/schema.rs (4 hunks)
  • lib/api/src/rest/validate.rs (2 hunks)
  • lib/collection/src/operations/universal_query/collection_query.rs (1 hunks)
  • src/common/inference/batch_processing.rs (1 hunks)
  • src/common/inference/query_requests_rest.rs (1 hunks)
🚧 Files skipped from review as they are similar to previous changes (3)
  • lib/api/src/grpc/proto/points.proto
  • lib/api/src/rest/schema.rs
  • lib/collection/src/operations/universal_query/collection_query.rs
🧰 Additional context used
🧠 Learnings (1)
📚 Learning: 2025-08-10T18:31:56.855Z
Learnt from: generall
PR: qdrant/qdrant#7006
File: lib/collection/src/operations/point_ops.rs:501-528
Timestamp: 2025-08-10T18:31:56.855Z
Learning: In Qdrant, batch operations validate that `ids`, `vectors`, and `payloads` (if present) have matching lengths at the REST API level in `lib/api/src/rest/validate.rs` through the `Validate` trait implementation for `Batch`. This validation happens before data is converted to internal structures like `PointInsertOperationsInternal`, so methods operating on these internal structures can safely assume the lengths match.

Applied to files:

  • lib/api/src/rest/validate.rs
🧬 Code Graph Analysis (1)
src/common/inference/query_requests_rest.rs (1)
lib/collection/src/operations/universal_query/collection_query.rs (3)
  • from (718-723)
  • from (727-730)
  • from (734-738)
🪛 markdownlint-cli2 (0.17.2)
docs/grpc/docs.md

250-250: Unordered list indentation
Expected: 2; Actual: 4

(MD007, ul-indent)

⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (12)
  • GitHub Check: test-consistency
  • GitHub Check: test-shard-snapshot-api-s3-minio
  • GitHub Check: e2e-tests
  • GitHub Check: test-consensus-compose
  • GitHub Check: integration-tests-consensus
  • GitHub Check: integration-tests
  • GitHub Check: rust-tests (ubuntu-latest)
  • GitHub Check: lint
  • GitHub Check: storage-compat-test
  • GitHub Check: rust-tests (windows-latest)
  • GitHub Check: rust-tests (macos-latest)
  • GitHub Check: rust-tests-no-rocksdb (ubuntu-latest)
🔇 Additional comments (8)
lib/api/src/grpc/qdrant.rs (4)

5647-5647: Query oneof tag extended to include Rrf (tag 10): LGTM

No tag collisions observed within Query oneof. Backwards compatibility is preserved via the existing Fusion enum variant.


5684-5686: New Query::Rrf variant preserves shorthand while enabling parameters

Adding a dedicated Rrf(super::Rrf) alongside the existing Fusion(Fusion::Rrf) shorthand is the right approach for backwards compatibility and clarity.


10178-10179: Shard path oneof tags extended to include Rrf: LGTM

QueryShardPoints::Query::Score now includes tag 7 for Rrf. This aligns the shard scoring path with the top-level query path.


10205-10207: Rrf variant validation confirmed

The validate.rs matcher includes:

grpc::query::Variant::Rrf(q) => q.validate(),

so shard-path validation for Rrf is already invoked. No further changes required.

lib/api/src/rest/validate.rs (2)

8-10: Import move to super module looks correct.

Switching NamedVectorStruct and BatchVectorStruct to super::* aligns with the module layout and avoids cross-crate coupling. No issues spotted.


31-41: Confirm RRF k validation and default fallback

The nested RrfQuery payload is correctly covered by rrf.validate():

  • lib/api/src/rest/schema.rs:523–526
    • pub struct Rrf derives Validate, with #[validate(range(min = 1))] on the k field and #[serde(default)]
  • lib/api/src/rest/schema.rs:695–698
    • pub struct RrfQuery derives Validate and nests pub rrf: Rrf

Please verify that when converting a missing or None k into the gRPC/Qdrant request (in lib/api/src/rest/conversions.rs), the code applies the default constant (e.g. DEFAULT_RRF_K = 2)—for example via rrf.k.unwrap_or(DEFAULT_RRF_K).
If it doesn’t, consider adding that fallback so omission of k still yields the intended default at execution time.

src/common/inference/batch_processing.rs (1)

101-105: LGTM: RRF is correctly excluded from inference batching

Treating RRF like other non-inference query variants (OrderBy/Fusion/Formula/Sample) is correct; it should not contribute any inference inputs.

src/common/inference/query_requests_rest.rs (1)

269-270: LGTM: REST RRF variant is consistently mapped to internal Fusion

Mapping rest::Query::Rrf to Query::Fusion(FusionInternal::from(rrf.rrf)) keeps parity with the existing "fusion": "rrf" shorthand and unifies both shapes in the internal layer.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Nitpick comments (2)
docs/redoc/master/openapi.json (2)

15016-15022: Clarify Fusion doc to point users to RrfQuery for custom k

The description says “rrf … with default parameters”, but doesn’t tell users how to pass custom parameters. Add a one-liner to steer them to the new RrfQuery shape to reduce ambiguity.

Proposed diff:

       "Fusion": {
-        "description": "Fusion algorithm allows to combine results of multiple prefetches.\n\nAvailable fusion algorithms:\n\n* `rrf` - Reciprocal Rank Fusion (with default parameters) * `dbsf` - Distribution-Based Score Fusion",
+        "description": "Fusion algorithm allows to combine results of multiple prefetches.\n\nAvailable fusion algorithms:\n\n* `rrf` - Reciprocal Rank Fusion (with default parameters)\n* `dbsf` - Distribution-Based Score Fusion\n\nNote:\n- Use the shorthand string form `\"fusion\": \"rrf\"` to apply defaults (k=2).\n- To customize RRF parameters, use the dedicated RrfQuery variant: `{ \"rrf\": { \"k\": <uint>=2 } }`.",
         "type": "string",
         "enum": [
           "rrf",
           "dbsf"
         ]
       },

15023-15047: RrfQuery/Rrf schema looks good; consider documenting default and adding examples

  • k uses unsigned integer with minimum 1 and nullable for “use default” – aligns with server validation and defaulting behavior. Nice.
  • Minor doc polish: explicitly state the default (2) and add examples to improve SDK codegen docs and UX.

Proposed diff:

       "RrfQuery": {
         "type": "object",
         "required": [
           "rrf"
         ],
         "properties": {
           "rrf": {
             "$ref": "#/components/schemas/Rrf"
           }
-        }
+        },
+        "example": { "rrf": { "k": 60 } }
       },
       "Rrf": {
-        "description": "Parameters for Reciprocal Rank Fusion",
+        "description": "Parameters for Reciprocal Rank Fusion. If `k` is omitted or null, the server uses the default k=2.",
         "type": "object",
         "properties": {
           "k": {
             "description": "K parameter for reciprocal rank fusion",
             "default": null,
             "type": "integer",
             "format": "uint",
             "minimum": 1,
-            "nullable": true
+            "nullable": true,
+            "example": 60
           }
         }
       },
📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

💡 Knowledge Base configuration:

  • MCP integration is disabled by default for public repositories
  • Jira integration is disabled by default for public repositories
  • Linear integration is disabled by default for public repositories

You can enable these sources in your CodeRabbit configuration.

📥 Commits

Reviewing files that changed from the base of the PR and between f01ec0b and e3db1a3.

📒 Files selected for processing (2)
  • docs/redoc/master/openapi.json (2 hunks)
  • lib/api/src/rest/schema.rs (4 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
  • lib/api/src/rest/schema.rs
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (12)
  • GitHub Check: storage-compat-test
  • GitHub Check: e2e-tests
  • GitHub Check: test-shard-snapshot-api-s3-minio
  • GitHub Check: integration-tests-consensus
  • GitHub Check: test-consistency
  • GitHub Check: integration-tests
  • GitHub Check: test-consensus-compose
  • GitHub Check: lint
  • GitHub Check: rust-tests-no-rocksdb (ubuntu-latest)
  • GitHub Check: rust-tests (windows-latest)
  • GitHub Check: rust-tests (macos-latest)
  • GitHub Check: rust-tests (ubuntu-latest)
🔇 Additional comments (1)
docs/redoc/master/openapi.json (1)

14813-14815: RRF added to Query union looks correct and non-breaking

Including RrfQuery alongside existing Query variants preserves the legacy "fusion": "rrf" flow while enabling the new parameterized path. Good addition.

@coszio coszio merged commit a2d3270 into dev Aug 20, 2025
16 checks passed
@coszio coszio deleted the custom-rrf branch August 20, 2025 19:05
timvisee pushed a commit that referenced this pull request Aug 26, 2025
* allow custom K parameter for RRF

* generate grpc docs and openapi

* use tagged type approach for parametrized fusions

* use params approach in grpc

* simplify api structure

* upd schema

* nits

* rest: parameterized rrf as query variant

* consistency in doc comments

---------

Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
@timvisee timvisee mentioned this pull request Nov 14, 2025
@coderabbitai coderabbitai bot mentioned this pull request Feb 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants