Skip to content

Implement logging of slow requests for scroll, facet, count, and retrieve#7291

Merged
generall merged 8 commits intodevfrom
slow-requests-log-2
Oct 27, 2025
Merged

Implement logging of slow requests for scroll, facet, count, and retrieve#7291
generall merged 8 commits intodevfrom
slow-requests-log-2

Conversation

@generall
Copy link
Member

No description provided.

@generall generall requested a review from timvisee September 22, 2025 21:12
@generall generall changed the base branch from master to dev September 22, 2025 21:13
@qdrant qdrant deleted a comment from coderabbitai bot Sep 23, 2025
}
}

pub struct ScrollRequestLoggable(pub Value);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we have a custom wrapping type here, not like the others? Is it just because we don't have a proper type for it?

Maybe it's worth to implement this - a type with all the shared parameters for the scroll variants. That way we also prevent forgetting to log/hash new parameters in the future.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, the source function operates with individual fields. Agreed, that it is not that fancy

coderabbitai[bot]

This comment was marked as resolved.

coderabbitai[bot]

This comment was marked as resolved.

@qdrant qdrant deleted a comment from coderabbitai bot Sep 23, 2025
@timvisee timvisee mentioned this pull request Sep 23, 2025
coderabbitai[bot]

This comment was marked as resolved.

@qdrant qdrant deleted a comment from coderabbitai bot Sep 24, 2025
@generall generall force-pushed the slow-requests-log-2 branch from 9789eff to 035cf61 Compare October 27, 2025 09:00
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (2)
lib/collection/src/collection/clean.rs (1)

286-296: Same issue: borrowed temporaries across await — bind locals first.

Replace &false.into() with locals to avoid lifetime across .await; explicit Type::from(false) preferred.

-        let mut ids = match shard
-            .local_scroll_by_id(
-                offset,
-                CLEAN_BATCH_SIZE + 1,
-                &false.into(),
-                &false.into(),
-                None,
-                None,
-                None,
-                HwMeasurementAcc::disposable(),
-            )
+        // Bind booleans to concrete types to avoid borrowing temporaries across .await
+        let with_payload: WithPayloadInterface = WithPayloadInterface::from(false);
+        let with_vector: WithVector = WithVector::from(false);
+        let mut ids = match shard
+            .local_scroll_by_id(
+                offset,
+                CLEAN_BATCH_SIZE + 1,
+                &with_payload,
+                &with_vector,
+                None,               // read_consistency
+                None,               // timeout
+                HwMeasurementAcc::disposable(),
+            )
             .await
         {
lib/collection/src/shards/remote_shard.rs (1)

1321-1326: Bug: facet uses raw timeout instead of processed_timeout

This bypasses validation/rounding and can set an invalid gRPC timeout. Use processed_timeout.

-                if let Some(timeout) = timeout {
-                    request.set_timeout(timeout);
+                if let Some(timeout) = processed_timeout {
+                    request.set_timeout(timeout);
                 }
♻️ Duplicate comments (2)
lib/collection/src/shards/replica_set/mod.rs (1)

935-945: Borrowed temporary across await: bind WithPayload/WithVector before calling local_scroll_by_id.

Passing &false.into() creates a temporary borrowed across .await and will fail to compile. Bind locals and pass refs; also follow guideline to prefer Type::from(false) over .into().

Apply:

             let mut points = local_shard
                 .get()
                 .local_scroll_by_id(
                     Some(current_offset),
                     BATCH_SIZE + 1,
-                    &false.into(),
-                    &false.into(),
+                    {
+                        // Prefer explicit Type::from(false)
+                        let with_payload: WithPayloadInterface = WithPayloadInterface::from(false);
+                        &with_payload
+                    },
+                    {
+                        let with_vector: WithVector = WithVector::from(false);
+                        &with_vector
+                    },
                     Some(&filter),
                     &self.search_runtime,
                     None,
                     hw_measurement_acc.clone(),
                 )
                 .await?;

As per coding guidelines.

lib/collection/src/shards/local_shard/shard_ops.rs (1)

370-390: Facet: same note about reducing logged payloads

Apply the same redaction approach here to keep slow-request entries compact.

🧹 Nitpick comments (5)
lib/collection/src/shards/local_shard/scroll.rs (1)

140-140: Visibility inconsistency: pub methods named internal_*.

The methods are renamed to internal_scroll_by_id and internal_scroll_by_field, suggesting they are intended for internal use only. However, they retain pub visibility, making them accessible outside the crate. While these methods are currently called only within lib/collection/ (in shard_ops.rs), the naming convention contradicts the visibility. Consider restricting visibility to pub(crate) to align the public API with intent, consistent with how pub(super) is already used in this file.

Also applies to: 223-223

lib/collection/src/operations/loggable.rs (1)

54-100: Loggable for Scroll/Facet/Count/Retrieve with struct-level hashing — good fit for dedup.

Approach looks solid. Consider adding quick unit tests to assert identical requests produce identical request_hash across runs to catch accidental field additions.

lib/collection/src/shards/local_shard/shard_ops.rs (1)

290-313: Consider trimming logged request payloads for large ID sets

If PointRequestInternal can be heavy (large ids), prefer logging a redacted/summary form (e.g., request.remove_details() or equivalent) to reduce collector pressure. If such helper doesn’t exist, consider adding one to Loggable.

lib/collection/src/shards/shard_trait.rs (1)

66-74: Minor naming nit

Parameter name hardware_accumulator differs from hw_measurement_acc used elsewhere; consider unifying for consistency.

lib/collection/src/shards/remote_shard.rs (1)

971-982: Prefer explicit From over Into conversions

To align with repo guidelines, prefer explicit target types with Type::from(x) over x.into() for offset, vector, and ids conversions. Example: offset.map(TargetType::from(o)). Based on coding guidelines

Also applies to: 1180-1187, 1073-1081

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 6c4ad11 and 035cf61.

📒 Files selected for processing (16)
  • lib/api/src/rest/schema.rs (1 hunks)
  • lib/collection/src/collection/clean.rs (1 hunks)
  • lib/collection/src/collection/point_ops.rs (2 hunks)
  • lib/collection/src/operations/loggable.rs (2 hunks)
  • lib/collection/src/operations/types.rs (3 hunks)
  • lib/collection/src/shards/dummy_shard.rs (2 hunks)
  • lib/collection/src/shards/forward_proxy_shard.rs (4 hunks)
  • lib/collection/src/shards/local_shard/scroll.rs (4 hunks)
  • lib/collection/src/shards/local_shard/shard_ops.rs (8 hunks)
  • lib/collection/src/shards/proxy_shard.rs (2 hunks)
  • lib/collection/src/shards/queue_proxy_shard.rs (3 hunks)
  • lib/collection/src/shards/remote_shard.rs (4 hunks)
  • lib/collection/src/shards/replica_set/mod.rs (1 hunks)
  • lib/collection/src/shards/replica_set/read_ops.rs (2 hunks)
  • lib/collection/src/shards/shard_trait.rs (1 hunks)
  • lib/segment/src/data_types/facets.rs (1 hunks)
🧰 Additional context used
📓 Path-based instructions (2)
**/*.rs

📄 CodeRabbit inference engine (.github/review-rules.md)

**/*.rs: Prefer explicit SomeType::from(x) over implicit x.into() in Rust code
Do not use transmute_from_u8, transmute_to_u8, transmute_from_u8_to_slice, transmute_from_u8_to_mut_slice, transmute_to_u8_slice in new code; use bytemuck or zerocopy instead

Files:

  • lib/collection/src/collection/clean.rs
  • lib/collection/src/shards/local_shard/scroll.rs
  • lib/api/src/rest/schema.rs
  • lib/collection/src/shards/replica_set/mod.rs
  • lib/collection/src/operations/loggable.rs
  • lib/segment/src/data_types/facets.rs
  • lib/collection/src/shards/queue_proxy_shard.rs
  • lib/collection/src/shards/shard_trait.rs
  • lib/collection/src/operations/types.rs
  • lib/collection/src/collection/point_ops.rs
  • lib/collection/src/shards/proxy_shard.rs
  • lib/collection/src/shards/replica_set/read_ops.rs
  • lib/collection/src/shards/dummy_shard.rs
  • lib/collection/src/shards/local_shard/shard_ops.rs
  • lib/collection/src/shards/remote_shard.rs
  • lib/collection/src/shards/forward_proxy_shard.rs
**/src/**/*.rs

📄 CodeRabbit inference engine (.github/review-rules.md)

**/src/**/*.rs: Prefer exhaustive match arms over a catch-all _ arm to avoid missing new enum variants (except in tests/benchmarks or when provably safe)
Prefer explicit field ignoring with : _ over .. in struct patterns (except in tests/benchmarks or when provably safe)

Files:

  • lib/collection/src/collection/clean.rs
  • lib/collection/src/shards/local_shard/scroll.rs
  • lib/api/src/rest/schema.rs
  • lib/collection/src/shards/replica_set/mod.rs
  • lib/collection/src/operations/loggable.rs
  • lib/segment/src/data_types/facets.rs
  • lib/collection/src/shards/queue_proxy_shard.rs
  • lib/collection/src/shards/shard_trait.rs
  • lib/collection/src/operations/types.rs
  • lib/collection/src/collection/point_ops.rs
  • lib/collection/src/shards/proxy_shard.rs
  • lib/collection/src/shards/replica_set/read_ops.rs
  • lib/collection/src/shards/dummy_shard.rs
  • lib/collection/src/shards/local_shard/shard_ops.rs
  • lib/collection/src/shards/remote_shard.rs
  • lib/collection/src/shards/forward_proxy_shard.rs
🧬 Code graph analysis (9)
lib/collection/src/operations/loggable.rs (2)
lib/collection/src/profiling/slow_requests_log.rs (3)
  • to_log_value (211-213)
  • request_name (215-217)
  • request_hash (219-221)
lib/collection/src/profiling/slow_requests_collector.rs (1)
  • new (24-35)
lib/collection/src/shards/queue_proxy_shard.rs (7)
lib/collection/src/shards/dummy_shard.rs (1)
  • local_scroll_by_id (118-130)
lib/collection/src/shards/forward_proxy_shard.rs (1)
  • local_scroll_by_id (490-514)
lib/collection/src/shards/local_shard/shard_ops.rs (1)
  • local_scroll_by_id (189-211)
lib/collection/src/shards/proxy_shard.rs (1)
  • local_scroll_by_id (252-276)
lib/collection/src/shards/remote_shard.rs (1)
  • local_scroll_by_id (1012-1027)
lib/collection/src/shards/replica_set/read_ops.rs (1)
  • local_scroll_by_id (46-90)
lib/collection/src/shards/shard_trait.rs (1)
  • local_scroll_by_id (36-46)
lib/collection/src/shards/shard_trait.rs (7)
lib/collection/src/shards/dummy_shard.rs (1)
  • local_scroll_by_id (118-130)
lib/collection/src/shards/forward_proxy_shard.rs (1)
  • local_scroll_by_id (490-514)
lib/collection/src/shards/local_shard/shard_ops.rs (1)
  • local_scroll_by_id (189-211)
lib/collection/src/shards/proxy_shard.rs (1)
  • local_scroll_by_id (252-276)
lib/collection/src/shards/queue_proxy_shard.rs (2)
  • local_scroll_by_id (285-308)
  • local_scroll_by_id (620-644)
lib/collection/src/shards/remote_shard.rs (1)
  • local_scroll_by_id (1012-1027)
lib/collection/src/shards/replica_set/read_ops.rs (1)
  • local_scroll_by_id (46-90)
lib/collection/src/collection/point_ops.rs (1)
lib/collection/src/operations/types.rs (2)
  • default (444-453)
  • default (1550-1552)
lib/collection/src/shards/proxy_shard.rs (7)
lib/collection/src/shards/dummy_shard.rs (1)
  • local_scroll_by_id (118-130)
lib/collection/src/shards/forward_proxy_shard.rs (1)
  • local_scroll_by_id (490-514)
lib/collection/src/shards/local_shard/shard_ops.rs (1)
  • local_scroll_by_id (189-211)
lib/collection/src/shards/queue_proxy_shard.rs (2)
  • local_scroll_by_id (285-308)
  • local_scroll_by_id (620-644)
lib/collection/src/shards/remote_shard.rs (1)
  • local_scroll_by_id (1012-1027)
lib/collection/src/shards/replica_set/read_ops.rs (1)
  • local_scroll_by_id (46-90)
lib/collection/src/shards/shard_trait.rs (1)
  • local_scroll_by_id (36-46)
lib/collection/src/shards/replica_set/read_ops.rs (5)
lib/collection/src/shards/dummy_shard.rs (1)
  • local_scroll_by_id (118-130)
lib/collection/src/shards/forward_proxy_shard.rs (1)
  • local_scroll_by_id (490-514)
lib/collection/src/shards/local_shard/shard_ops.rs (1)
  • local_scroll_by_id (189-211)
lib/collection/src/shards/remote_shard.rs (1)
  • local_scroll_by_id (1012-1027)
lib/collection/src/shards/shard_trait.rs (1)
  • local_scroll_by_id (36-46)
lib/collection/src/shards/dummy_shard.rs (7)
lib/collection/src/shards/forward_proxy_shard.rs (1)
  • local_scroll_by_id (490-514)
lib/collection/src/shards/local_shard/shard_ops.rs (1)
  • local_scroll_by_id (189-211)
lib/collection/src/shards/proxy_shard.rs (1)
  • local_scroll_by_id (252-276)
lib/collection/src/shards/queue_proxy_shard.rs (2)
  • local_scroll_by_id (285-308)
  • local_scroll_by_id (620-644)
lib/collection/src/shards/remote_shard.rs (1)
  • local_scroll_by_id (1012-1027)
lib/collection/src/shards/replica_set/read_ops.rs (1)
  • local_scroll_by_id (46-90)
lib/collection/src/shards/shard_trait.rs (1)
  • local_scroll_by_id (36-46)
lib/collection/src/shards/local_shard/shard_ops.rs (7)
lib/collection/src/operations/types.rs (4)
  • timeout (939-946)
  • default_with_payload (434-436)
  • bad_input (955-959)
  • default_limit (430-432)
lib/collection/src/profiling/interface.rs (1)
  • log_request_to_collector (13-32)
lib/collection/src/shards/dummy_shard.rs (1)
  • local_scroll_by_id (118-130)
lib/collection/src/shards/forward_proxy_shard.rs (1)
  • local_scroll_by_id (490-514)
lib/collection/src/shards/proxy_shard.rs (1)
  • local_scroll_by_id (252-276)
lib/collection/src/shards/remote_shard.rs (1)
  • local_scroll_by_id (1012-1027)
lib/collection/src/shards/shard_trait.rs (1)
  • local_scroll_by_id (36-46)
lib/collection/src/shards/forward_proxy_shard.rs (7)
lib/collection/src/shards/dummy_shard.rs (1)
  • local_scroll_by_id (118-130)
lib/collection/src/shards/local_shard/shard_ops.rs (1)
  • local_scroll_by_id (189-211)
lib/collection/src/shards/proxy_shard.rs (1)
  • local_scroll_by_id (252-276)
lib/collection/src/shards/queue_proxy_shard.rs (2)
  • local_scroll_by_id (285-308)
  • local_scroll_by_id (620-644)
lib/collection/src/shards/remote_shard.rs (1)
  • local_scroll_by_id (1012-1027)
lib/collection/src/shards/replica_set/read_ops.rs (1)
  • local_scroll_by_id (46-90)
lib/collection/src/shards/shard_trait.rs (1)
  • local_scroll_by_id (36-46)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (11)
  • GitHub Check: test-shard-snapshot-api-s3-minio
  • GitHub Check: lint
  • GitHub Check: test-consistency
  • GitHub Check: e2e-tests
  • GitHub Check: rust-tests (windows-latest)
  • GitHub Check: test-consensus-compose
  • GitHub Check: rust-tests-no-rocksdb (ubuntu-latest)
  • GitHub Check: rust-tests (macos-latest)
  • GitHub Check: rust-tests (ubuntu-latest)
  • GitHub Check: integration-tests-consensus
  • GitHub Check: integration-tests
🔇 Additional comments (26)
lib/segment/src/data_types/facets.rs (1)

12-12: Deriving Hash for FacetParams looks good; verify nested Hashes.

Confirm that all nested types (JsonPath, Filter) implement Hash to avoid derive failures and to keep request_hash stable.

lib/collection/src/operations/types.rs (1)

404-427: Hash derives on internal request types enable stable request hashing.

LGTM. Please double‑check all nested fields implement Hash (e.g., WithPayloadInterface, WithVector, OrderByInterface) so these derives don’t break builds. Optionally derive Eq alongside PartialEq+Hash for semantic completeness.

Also applies to: 537-547, 865-874

lib/api/src/rest/schema.rs (1)

503-509: Add Hash on OrderByInterface — OK; ensure dependencies hashable.

Make sure segment::data_types::order_by::OrderBy and JsonPath also implement Hash to keep this compile-safe and deterministic.

lib/collection/src/operations/loggable.rs (1)

71-83: Blanket impl for Arc is handy.

Nice ergonomics for passing Arc into Box. No issues spotted.

lib/collection/src/shards/proxy_shard.rs (2)

237-249: New ProxyShard::scroll_by forwarder — LGTM.

Correctly forwards Arc with runtime + timeout; consistent with ShardOperation.


265-274: local_scroll_by_id forwarder updated to new signature — LGTM.

OrderBy removal aligns with trait changes; params match shard impls.

lib/collection/src/collection/point_ops.rs (1)

262-291: scroll_by refactor: limit bump and Arc dispatch — looks correct; two nits.

  • Logic for next_page via limit+1 when no order_by is sound; Arc avoids cloning.
  • Nit: local_only derived once and passed — good. Consider a small test for order_by paging (no offset) to prevent regressions.

Also applies to: 296-306, 321-344, 346-357

lib/collection/src/shards/local_shard/shard_ops.rs (5)

137-141: Good guard: offset not allowed with order_by

Prevents ambiguous paging. Clear error guidance.


150-154: Initialization and conversion look sound

Defaulting limit and converting order_by via OrderBy::from keeps internal semantics consistent.


155-168: Branching by order_by is clean and readable

Clear separation between ID-ordered and field-ordered flows.

Also applies to: 170-182


184-187: Logging closure: avoid clone — good

Passing the Arc request directly keeps allocation minimal and follows prior feedback.


250-273: Count instrumentation is correct

Timer placement and conditional logging align with slow-request goals.

lib/collection/src/shards/forward_proxy_shard.rs (3)

476-488: scroll_by delegation LGTM

Forwards the Arc cleanly to the wrapped shard with consistent params.


186-206: Efficient batch read path

Use of local_scroll_by_id with vectors+payload enabled, plus +1 paging, is correct.


263-274: Preselection optimization is solid

Scrolling IDs only before retrieving full records minimizes transfer cost.

lib/collection/src/shards/shard_trait.rs (2)

24-31: New scroll_by signature is consistent with internal request refactor

Arc across shards standardizes the API.


32-46: Documented exclusion from rate limits/logging — good

Clear intent for local_scroll_by_id as an internal-only pathway.

lib/collection/src/shards/dummy_shard.rs (1)

118-130: Stub aligns with DummyShard semantics

Returning the configured service_error is expected.

lib/collection/src/shards/remote_shard.rs (1)

1003-1007: is_payload_required usage is correct

Passing payload necessity to try_record_from_grpc avoids unnecessary decoding.

lib/collection/src/shards/queue_proxy_shard.rs (5)

31-32: LGTM! Import changes align with refactoring.

The addition of ScrollRequestInternal to the imports is correct and necessary for the updated method signatures.


272-282: LGTM! Clean refactoring to consolidated request object.

The signature change from individual scroll parameters to Arc<ScrollRequestInternal> simplifies the API and the forwarding implementation is correct.


285-308: LGTM! Removal of order_by parameter is consistent with trait changes.

The updated signature correctly removes the order_by parameter and properly forwards to the inner implementation, aligning with the ShardOperation trait definition.


607-618: LGTM! Inner implementation properly forwards with new signature.

The forwarding to wrapped_shard.scroll_by with the consolidated Arc<ScrollRequestInternal> is correct.


620-644: LGTM! Inner forwarding implementation is correct.

The removal of order_by and forwarding to wrapped_shard.local_scroll_by_id is consistent with the trait changes.

lib/collection/src/shards/replica_set/read_ops.rs (2)

17-43: LGTM! Well-implemented scroll_by with consolidated request.

The new signature using Arc<ScrollRequestInternal> is clean, and the implementation correctly clones the Arc in the closure and delegates to the shard with appropriate parameters.


46-90: LGTM! Signature update is correct with appropriate hardcoded local_only.

The removal of order_by and local_only parameters from the signature is correct. Hardcoding local_only to true (line 87) is appropriate for a method explicitly named local_scroll_by_id.

Comment on lines 107 to +116
/// Forward read-only `scroll_by` to `wrapped_shard`
async fn scroll_by(
&self,
_: Arc<ScrollRequestInternal>,
_: &Handle,
_: Option<Duration>,
_: HwMeasurementAcc,
) -> CollectionResult<Vec<RecordInternal>> {
self.dummy()
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

Doc comment mismatch

Comment says “Forward read-only scroll_by to wrapped_shard”, but DummyShard has no wrapped shard and returns an error. Update the doc to reflect dummy behavior.

-    /// Forward read-only `scroll_by` to `wrapped_shard`
+    /// Dummy implementation of `scroll_by` that always returns an error
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
/// Forward read-only `scroll_by` to `wrapped_shard`
async fn scroll_by(
&self,
_: Arc<ScrollRequestInternal>,
_: &Handle,
_: Option<Duration>,
_: HwMeasurementAcc,
) -> CollectionResult<Vec<RecordInternal>> {
self.dummy()
}
/// Dummy implementation of `scroll_by` that always returns an error
async fn scroll_by(
&self,
_: Arc<ScrollRequestInternal>,
_: &Handle,
_: Option<Duration>,
_: HwMeasurementAcc,
) -> CollectionResult<Vec<RecordInternal>> {
self.dummy()
}
🤖 Prompt for AI Agents
In lib/collection/src/shards/dummy_shard.rs around lines 107 to 116, the doc
comment incorrectly states that scroll_by is "Forward read-only `scroll_by` to
`wrapped_shard`" even though DummyShard has no wrapped shard and instead returns
an error; update the doc comment to accurately describe that this implementation
always returns a dummy/error (or unimplemented) response for scroll_by,
explaining that DummyShard does not support scrolling and returns the dummy
error via self.dummy().

Comment on lines +953 to 982
let ScrollRequestInternal {
filter,
offset,
limit,
with_payload,
with_vector,
order_by,
} = request.as_ref();

let with_payload = with_payload
.as_ref()
.cloned()
.unwrap_or_else(ScrollRequestInternal::default_with_payload);

let is_payload_required = with_payload.is_required();
let order_by = order_by.clone().map(OrderBy::from);
let filter = filter.clone();

let scroll_points = ScrollPoints {
collection_name: self.collection_id.clone(),
filter: filter.map(|f| f.clone().into()),
filter: filter.map(api::grpc::qdrant::Filter::from),
offset: offset.map(|o| o.into()),
limit: Some(limit as u32),
with_payload: Some(with_payload_interface.clone().into()),
limit: limit.map(|x| x as u32),
with_payload: Some(WithPayloadSelector::from(with_payload)),
with_vectors: Some(with_vector.clone().into()),
read_consistency: None,
shard_key_selector: None,
order_by: order_by.map(|o| o.clone().into()),
order_by: order_by.map(api::grpc::qdrant::OrderBy::from),
timeout: processed_timeout.map(|t| t.as_secs()),
};
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Potential truncation: usize → u32 cast for limit

Casting with as can wrap on 64‑bit platforms for very large limits. Validate with try_from and return BadInput on overflow.

-        let scroll_points = ScrollPoints {
+        // Validate limit fits into u32 expected by gRPC
+        let limit_u32 = limit
+            .map(|x| u32::try_from(*x).map_err(|_| CollectionError::bad_input("limit exceeds u32::MAX")) )
+            .transpose()?;
+
+        let scroll_points = ScrollPoints {
             collection_name: self.collection_id.clone(),
             filter: filter.map(api::grpc::qdrant::Filter::from),
             offset: offset.map(|o| o.into()),
-            limit: limit.map(|x| x as u32),
+            limit: limit_u32,
             with_payload: Some(WithPayloadSelector::from(with_payload)),
             with_vectors: Some(with_vector.clone().into()),
             read_consistency: None,
             shard_key_selector: None,
             order_by: order_by.map(api::grpc::qdrant::OrderBy::from),
             timeout: processed_timeout.map(|t| t.as_secs()),
         };

Committable suggestion skipped: line range outside the PR's diff.

🤖 Prompt for AI Agents
In lib/collection/src/shards/remote_shard.rs around lines 953 to 982, the code
uses "limit.map(|x| x as u32)" which can silently truncate/wrap on 64-bit
platforms; replace the unchecked cast with a fallible conversion (e.g.,
u32::try_from) and return a BadInput error if the value overflows or is
otherwise invalid. Specifically, map limit to Result<Option<u32>, Error> (or
validate before building ScrollPoints), convert using try_from, propagate or
convert the failure into the appropriate BadInput gRPC/HTTP error response, and
then pass the validated u32 into ScrollPoints.limit; preserve existing timeout
and other fields.

@generall generall merged commit b037910 into dev Oct 27, 2025
15 checks passed
@generall generall deleted the slow-requests-log-2 branch October 27, 2025 10:19
timvisee added a commit that referenced this pull request Nov 14, 2025
…ieve (#7291)

* reorder arguments of scroll_by

* add slow request log for scroll request

* add slow request log for facet request

* add slow request log for count request

* add slow request log for retrieve request

* Remove obsolete clone

* generic implementation of Loggable for Arc

* Refactor scroll by arguments (#7296)

* separate internal scroll operation from the one used by user

* fmt

---------

Co-authored-by: Tim Visée <tim+github@visee.me>
@timvisee timvisee mentioned this pull request Nov 14, 2025
@Huarong
Copy link

Huarong commented Nov 21, 2025

Do we need a config to enable slow this request feature?

@qdrant qdrant deleted a comment from coderabbitai bot Nov 21, 2025
@timvisee
Copy link
Member

timvisee commented Nov 21, 2025

Do we need a config to enable slow this request feature?

No. This is enabled by default since Qdrant 1.16.0. See GET /profiler/slow_requests.

@Huarong
Copy link

Huarong commented Nov 21, 2025

@timvisee Thanks the great work. Maybe we should update the API documents

@generall
Copy link
Member Author

Maybe we should update the API documents

At this point we don't want to document this API, as it might change or even be removed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants