Skip to content

Conversation

@Lyndon-Li
Copy link
Contributor

Maintenance is critical for healthy of the repository.
On the other hand, Maintenance is complex, because it runs multiple sub tasks each may generate different results according to the maintenance policy. The results may include deleting/combining/adding data/metadata to the repository.

It is worthy to add more observability for these tasks for below reasons:

  • It is helpful for troubleshooting. Any data change to the repository is critical, the observability info helps to understand what happened during the maintenance and why that happened
  • It is helpful for users to understand/predict the repo's behavior. The repo data may be stored in a public cloud for which costs are sensitive to scale/duration of data stored. On the other hand, repository has its own policy to manage the data, so the data is not deleted until it is safe enough according to the policy. The observability info helps users to understand how much data is in-use, how much data is out of use and when it is deleted

There will be a serial of PRs to add observability info for each sub task.
The current PR add the stats info for DeleteUnreferencedPacks sub task.

@Lyndon-Li Lyndon-Li marked this pull request as ready for review November 5, 2025 05:42
@Lyndon-Li Lyndon-Li force-pushed the delete-unreferenced-pack-stats branch from ce39da6 to 2fb19a5 Compare November 5, 2025 05:55
Copy link
Collaborator

@julio-lopez julio-lopez left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good overall.
Requested change in the stats field names.

@julio-lopez julio-lopez requested a review from Copilot November 5, 2025 06:11
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR adds observability statistics for the DeleteUnreferencedPacks maintenance task, enabling better tracking of blob deletion operations during repository maintenance.

Key Changes:

  • Introduces a new DeleteUnreferencedPacksStats struct to capture unreferenced, deleted, and retained blob counts and sizes
  • Updates DeleteUnreferencedBlobs to return statistics instead of just a count
  • Adds tracking for retained blobs (those preserved due to cutoff time, minimum age, or active session constraints)

Reviewed Changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
repo/maintenancestats/stats_delete_unreferenced_packs.go Defines the new stats structure with JSON serialization and summary methods
repo/maintenancestats/builder_test.go Adds test cases for serializing/deserializing the new stats type
repo/maintenancestats/builder.go Registers the new stats kind in the builder factory
repo/maintenance/maintenance_run.go Simplifies task runners to directly return stats from DeleteUnreferencedBlobs
repo/maintenance/blob_gc.go Implements stats collection including retained blob tracking and returns structured stats
cli/command_blob_gc.go Updates CLI command to use the new stats structure

@codecov
Copy link

codecov bot commented Nov 5, 2025

Codecov Report

❌ Patch coverage is 83.33333% with 7 lines in your changes missing coverage. Please review.
✅ Project coverage is 77.99%. Comparing base (cb455c6) to head (18b935f).
⚠️ Report is 731 commits behind head on master.

Files with missing lines Patch % Lines
repo/maintenance/blob_gc.go 78.26% 5 Missing ⚠️
cli/command_blob_gc.go 0.00% 2 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #4955      +/-   ##
==========================================
+ Coverage   75.86%   77.99%   +2.13%     
==========================================
  Files         470      544      +74     
  Lines       37301    31353    -5948     
==========================================
- Hits        28299    24455    -3844     
+ Misses       7071     4851    -2220     
- Partials     1931     2047     +116     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@Lyndon-Li Lyndon-Li force-pushed the delete-unreferenced-pack-stats branch from 2fb19a5 to 84b7e75 Compare November 5, 2025 06:21
@Lyndon-Li Lyndon-Li force-pushed the delete-unreferenced-pack-stats branch from 84b7e75 to 18b935f Compare November 5, 2025 06:26
@Lyndon-Li Lyndon-Li requested a review from julio-lopez November 5, 2025 06:33
@julio-lopez julio-lopez merged commit db48fad into kopia:master Nov 5, 2025
23 checks passed
@julio-lopez
Copy link
Collaborator

@julio-lopez
Copy link
Collaborator

Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants