Skip to content

Add function for indexed_only decision#7463

Closed
timvisee wants to merge 11 commits intodevfrom
metrics-unindexed-vectors-improved
Closed

Add function for indexed_only decision#7463
timvisee wants to merge 11 commits intodevfrom
metrics-unindexed-vectors-improved

Conversation

@timvisee
Copy link
Member

Improves #7307

Add a in_indexed_only_search() function to segments and their indices. Allow us to use just this function to decide how indexed_only behaves.

This specifically improves the way we list excluded points for #7307 by keeping the indexed_only decision within segments.

All Submissions:

  • Contributions should target the dev branch. Did you create your branch from dev?
  • Have you followed the guidelines in our Contributing document?
  • Have you checked to ensure there aren't other open Pull Requests for the same update/change?

Comment on lines +550 to +572
// TODO(timvisee): enable indexed_only handling with proxies
// // If the proxy segment has no deleted points, direct call to wrapped segment
// if self.deleted_points.is_empty() {
// return self.wrapped_segment.get().read().in_indexed_only_search(
// vector_name,
// search_optimized_threshold_kb,
// filter,
// hw_counter,
// );
// }

// // Incorporate deleted point count in decision
// let wrapped_filter = Self::add_deleted_points_condition_to_filter(
// filter,
// self.deleted_points.keys().copied(),
// );

// self.wrapped_segment.get().read().in_indexed_only_search(
// vector_name,
// search_optimized_threshold_kb,
// Some(&wrapped_filter),
// hw_counter,
// )
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The current behavior is actually the above. But it might be more correct to use this commented-out section instead.

I'd have to think about it for a bit.

If we do enable this, we must also adjust how we use the parameter in search.

@timvisee timvisee changed the title Metrics unindexed vectors improved Add function for indexed_only decision Oct 28, 2025
@timvisee timvisee force-pushed the metrics-unindexed-vectors-improved branch from af7ac72 to 3d55bce Compare October 28, 2025 11:49
@timvisee timvisee force-pushed the metrics-unindexed-vectors branch from 938b102 to 2f25d7d Compare October 28, 2025 16:33
@timvisee timvisee force-pushed the metrics-unindexed-vectors-improved branch from e1ddc22 to a0fbeb2 Compare October 28, 2025 16:33
@timvisee
Copy link
Member Author

We can postpone merging this until after the Qdrant 1.16 release. That gives us a bit more time to decide upon #7463 (review).

@JojiiOfficial JojiiOfficial force-pushed the metrics-unindexed-vectors branch 3 times, most recently from 6a6386c to e7cbba6 Compare October 29, 2025 15:41
Base automatically changed from metrics-unindexed-vectors to dev October 29, 2025 15:44
@timvisee timvisee added the do not merge Pull requests blocked on an external event label Nov 12, 2025
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This name seems complicated and depends on a specific query parameter.

Suggested change
pub unindexed_vectors: Option<HashMap<String, usize>>,

Do we even need to expose this?

Idea: do not expose this new metric, and instead, consider those unoptimized segments with less than full_scan_threshold as optimized, and include them in the total_optimized_points count.

@timvisee
Copy link
Member Author

This is stale, and I'm not sure it's worth the effort to revive it

@timvisee timvisee closed this Jan 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

do not merge Pull requests blocked on an external event

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants