Skip to content

Add replica metrics#7301

Merged
timvisee merged 9 commits intodevfrom
metrics-active-replica
Oct 29, 2025
Merged

Add replica metrics#7301
timvisee merged 9 commits intodevfrom
metrics-active-replica

Conversation

@JojiiOfficial
Copy link
Contributor

@JojiiOfficial JojiiOfficial commented Sep 24, 2025 •
edited
Loading

Adds the following new metrics to the /metrics api

# HELP active_total_replicas_min minimum number of replicas across shards
# TYPE active_total_replicas_min gauge
active_replicas_min 2

# HELP active_total_replicas_max maximum number of replicas across shards
# TYPE active_total_replicas_max gauge
active_replicas_max 2

@timvisee
Copy link
Member

timvisee commented Sep 24, 2025 •
edited
Loading 

# HELP active_expected_replicas accumulated total number of expected replicas across all shards
# TYPE active_expected_replicas gauge
active_expected_replicas 4

Doesn't it make more sense to count this per collection?

@JojiiOfficial
Copy link
Contributor Author

JojiiOfficial commented Sep 24, 2025

I made this metric not per-collection to keep consistent with the other metrics. Additionally, we can compare this parameter against active_total_replicas more easily if we don't measure it on per-collection level.

@JojiiOfficial JojiiOfficial mentioned this pull request Sep 24, 2025
Merged
@JojiiOfficial JojiiOfficial force-pushed the optimizer-trigger-count-metric branch from e1f6f5b to 424a232 Compare September 25, 2025 06:54
@JojiiOfficial JojiiOfficial force-pushed the optimizer-trigger-count-metric branch from f3b7c36 to c099206 Compare September 25, 2025 09:23
timvisee
timvisee reviewed Sep 25, 2025
View reviewed changes
Copy link
Member

@timvisee timvisee left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we correctly handle resharding. See my review remark below.

Happy to assist you with this one to form the logic here.

generall
generall previously requested changes Sep 25, 2025
View reviewed changes
src/common/metrics.rs
.filter(|i| i.status == TrackerStatus::Optimizing)
.count();

let min_max_active_replicas = collection
Copy link
Member

@generall generall Sep 25, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's move replica counting into a separate function/file. This one becomes bulky

Copy link
Contributor Author

@JojiiOfficial JojiiOfficial Oct 29, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd suggest to do this in a separate PR once the other ones are merged too. This way rebasing and merging with the other currently open PRs is significantly easier.

timvisee
timvisee reviewed Oct 1, 2025
View reviewed changes
Copy link
Member

@timvisee timvisee left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Happy to approve once the above review remarks have been resolved ✔️

@JojiiOfficial JojiiOfficial force-pushed the metrics-active-replica branch from ee21c86 to 33545d9 Compare October 29, 2025 09:03
@timvisee timvisee requested a review from generall October 29, 2025 11:53
@JojiiOfficial JojiiOfficial force-pushed the metrics-active-replica branch from 7044a84 to df0c11e Compare October 29, 2025 13:35
@JojiiOfficial JojiiOfficial changed the base branch from optimizer-trigger-count-metric to dev October 29, 2025 13:36
coderabbitai[bot]

This comment was marked as resolved.

Sign in to view

coderabbitai[bot]

This comment was marked as resolved.

Sign in to view

@qdrant qdrant deleted a comment from coderabbitai bot Oct 29, 2025
@timvisee timvisee merged commit 36bbc3a into dev Oct 29, 2025
12 checks passed
@timvisee timvisee deleted the metrics-active-replica branch October 29, 2025 14:59
@qdrant qdrant deleted a comment from coderabbitai bot Oct 29, 2025
timvisee added a commit that referenced this pull request Nov 14, 2025
@JojiiOfficial @timvisee
Add replica metrics (#7301)
69d3198 
* Add replica metrics

* Clippy

* Properly ignore shards with resharding status

* Include 0 as value in metrics

* Remove total_active_replicas and total_expected_replicas

* Improve minimum active replica count in case of resharding

* Reorder metrics calls min < max (to trigger CI)

* Fix metrics help

* Fix typo

---------

Co-authored-by: timvisee <tim@visee.me>
@timvisee timvisee mentioned this pull request Nov 14, 2025
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@coderabbitai coderabbitai[bot] coderabbitai[bot] left review comments

@timvisee timvisee timvisee approved these changes

@generall generall Awaiting requested review from generall

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@JojiiOfficial @timvisee @generall