feat(general): add stats to maintenance run - CompactIndexesStats #4948
Conversation
|
This task performs quite differently for V0 and V1 format, and is much simpler for V1 format. Let me know if we want the stats to be more informative for V0 format. |
There was a problem hiding this comment.
Pull Request Overview
This PR adds observability statistics for the "drop deleted indexes" maintenance task by introducing a new DropDeletedIndexesStats struct. The changes enable tracking when deleted indexes are dropped from the repository, which helps with troubleshooting and understanding repository behavior over time.
Key changes:
- Introduces
DropDeletedIndexesStatsstruct to track when deleted indexes are dropped - Updates
CompactIndexesmethod signature to return statistics instead of just an error - Adds comprehensive test coverage for the new statistics functionality
Reviewed Changes
Copilot reviewed 14 out of 14 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
repo/maintenancestats/stats_compact_indexes.go |
New file defining the DropDeletedIndexesStats struct with JSON serialization and summary methods |
repo/maintenancestats/builder.go |
Adds support for building DropDeletedIndexesStats from serialized data |
repo/maintenancestats/builder_test.go |
Test cases for serialization/deserialization of the new stats type |
repo/content/indexblob/index_blob.go |
Updates Manager interface to return stats from Compact method |
repo/content/indexblob/index_blob_manager_v0.go |
Updates V0 manager implementation to return DropDeletedIndexesStats |
repo/content/indexblob/index_blob_manager_v1.go |
Updates V1 manager implementation to return DropDeletedIndexesStats |
repo/content/content_manager_indexes.go |
Updates CompactIndexes to handle the new return type |
repo/maintenance/drop_deleted_contents.go |
Updates function signature to return stats |
repo/maintenance/index_compaction.go |
Updates to handle new return type from CompactIndexes |
repo/maintenance/maintenance_run.go |
Updates task runner to handle stats return value |
cli/command_index_optimize.go |
Updates CLI command to handle new return type |
repo/repository_test.go |
Updates test calls to handle new return signature |
repo/content/content_manager_test.go |
Updates test calls to handle new return signature |
tests/repository_stress_test/repository_stress_test.go |
Updates test calls to handle new return signature |
That's OK. There is NO need to include details about the V0 implementation at the moment. What is desirable is to clearly call out the 2 conceptually separate maintenance tasks. |
ec0df39 to
f96c942
Compare
f96c942 to
1c3d221
Compare
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## master #4948 +/- ##
==========================================
+ Coverage 75.86% 77.96% +2.09%
==========================================
Files 470 543 +73
Lines 37301 31333 -5968
==========================================
- Hits 28299 24429 -3870
+ Misses 7071 4856 -2215
- Partials 1931 2048 +117 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
|
Thanks! |
Maintenance is critical for healthy of the repository.
On the other hand, Maintenance is complex, because it runs multiple sub tasks each may generate different results according to the maintenance policy. The results may include deleting/combining/adding data/metadata to the repository.
It is worthy to add more observability for these tasks for below reasons:
There will be a serial of PRs to add observability info for each sub task.
The current PR add the stats info for CompactIndexesStats sub task.