Skip to content

Conversation

@kdkasad
Copy link
Contributor

@kdkasad kdkasad commented Jun 2, 2025

The dirRelativePath variable is actually the path to the file being checked, but was treated as if it was the path to the parent directory, causing the filename to be duplicated in log messages. This PR fixes that.

Here is the output of remove-files on stock Kopia 0.20.1:

$ kopia snapshot fix remove-files --source kopia@box --filename encoded-video --filename thumbs
Listing snapshots for source "kopia@box"...
Processing snapshot kopia@box:/
  2025-05-15 12:00:57 PDT unchanged (120c714f22c88c61f4c36e10e5efab02)
will remove file nas/Photos/encoded-video/encoded-video
will remove file nas/Photos/thumbs/thumbs
  2025-05-22 21:01:48 PDT replaced manifest from 065a7cbe18b93062138638af7e5fad60 to 065a7cbe18b93062138638af7e5fad60
    diff k74105c4e53c951be0faa3eec704e54af kb5e20e9ebd675a28eb747215f2b0e781
    delta:-167.6 GB
will remove file nas/Photos/encoded-video/encoded-video
will remove file nas/Photos/thumbs/thumbs
  2025-05-27 22:20:01 PDT replaced manifest from dc8c0fc2f5c976e48826541f51ac63d2 to dc8c0fc2f5c976e48826541f51ac63d2
    diff k1050367e8bdaf5e99d1e45724f1fdaa3 kac9c390168dbf70b95735fcc5f35675a
    delta:-169.2 GB
  2025-06-01 16:35:18 PDT unchanged (f8135d3d4af33c53e1dd0453ce35607e)
Fixed 2 snapshots, but snapshot manifests were not updated. Pass --commit to update snapshots.

Evidently, the last path component is duplicated. The paths printed do not actually exist, but the paths printed minus the duplicated last path component do exist.

For demonstration of correctness of the fix, here is the output after adding a log statement to print dirRelativePath and ent.Name:

     48 DEBUG rewriting {"dirRelativePath":"boot/grub/i386-pc/blocklist.mod","ent.Name":"blocklist.mod"}
     49 DEBUG rewriting {"dirRelativePath":"boot/grub/i386-pc/boot.img","ent.Name":"boot.img"}
     50 DEBUG rewriting {"dirRelativePath":"boot/grub/i386-pc/boot.mod","ent.Name":"boot.mod"}
     51 DEBUG rewriting {"dirRelativePath":"boot/grub/i386-pc/bsd.mod","ent.Name":"bsd.mod"}
     52 DEBUG rewriting {"dirRelativePath":"boot/grub/i386-pc/bswap_test.mod","ent.Name":"bswap_test.mod"}
     53 DEBUG rewriting {"dirRelativePath":"boot/grub/i386-pc/btrfs.mod","ent.Name":"btrfs.mod"}
     54 DEBUG rewriting {"dirRelativePath":"boot/grub/i386-pc/bufio.mod","ent.Name":"bufio.mod"}
     55 DEBUG rewriting {"dirRelativePath":"boot/grub/i386-pc/cat.mod","ent.Name":"cat.mod"}
     56 DEBUG rewriting {"dirRelativePath":"boot/grub/i386-pc/cbfs.mod","ent.Name":"cbfs.mod"}
     57 DEBUG rewriting {"dirRelativePath":"boot/grub/i386-pc/cbls.mod","ent.Name":"cbls.mod"}
     58 DEBUG rewriting {"dirRelativePath":"boot/grub/i386-pc/cbmemc.mod","ent.Name":"cbmemc.mod"}
     59 DEBUG rewriting {"dirRelativePath":"boot/grub/i386-pc/cbtable.mod","ent.Name":"cbtable.mod"}
     60 DEBUG rewriting {"dirRelativePath":"boot/grub/i386-pc/cbtime.mod","ent.Name":"cbtime.mod"}
     61 DEBUG rewriting {"dirRelativePath":"boot/grub/i386-pc/chain.mod","ent.Name":"chain.mod"}
     62 DEBUG rewriting {"dirRelativePath":"boot/grub/i386-pc/cmdline_cat_test.mod","ent.Name":"cmdline_cat_test.mod"}
     63 DEBUG rewriting {"dirRelativePath":"boot/grub/i386-pc/cmosdump.mod","ent.Name":"cmosdump.mod"}
     64 DEBUG rewriting {"dirRelativePath":"boot/grub/i386-pc/cmostest.mod","ent.Name":"cmostest.mod"}
     65 DEBUG rewriting {"dirRelativePath":"boot/grub/i386-pc/cmp.mod","ent.Name":"cmp.mod"}
     66 DEBUG rewriting {"dirRelativePath":"boot/grub/i386-pc/cmp_test.mod","ent.Name":"cmp_test.mod"}
     67 DEBUG rewriting {"dirRelativePath":"boot/grub/i386-pc/command.lst","ent.Name":"command.lst"}
     68 DEBUG rewriting {"dirRelativePath":"boot/grub/i386-pc/configfile.mod","ent.Name":"configfile.mod"}
     69 DEBUG rewriting {"dirRelativePath":"boot/grub/i386-pc/core.img","ent.Name":"core.img"}
     70 DEBUG rewriting {"dirRelativePath":"boot/grub/i386-pc/cpio.mod","ent.Name":"cpio.mod"}
     71 DEBUG rewriting {"dirRelativePath":"boot/grub/i386-pc/cpio_be.mod","ent.Name":"cpio_be.mod"}
     72 DEBUG rewriting {"dirRelativePath":"boot/grub/i386-pc/cpuid.mod","ent.Name":"cpuid.mod"}
     73 DEBUG rewriting {"dirRelativePath":"boot/grub/i386-pc/crc64.mod","ent.Name":"crc64.mod"}

It can be seen that the path is the path of the file being checked, not the parent directory.

@codecov
Copy link

codecov bot commented Jun 2, 2025

Codecov Report

Attention: Patch coverage is 85.71429% with 1 line in your changes missing coverage. Please review.

Project coverage is 76.40%. Comparing base (cb455c6) to head (b70be5f).
Report is 563 commits behind head on master.

Files with missing lines Patch % Lines
cli/command_snapshot_fix_remove_files.go 66.66% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #4638      +/-   ##
==========================================
+ Coverage   75.86%   76.40%   +0.53%     
==========================================
  Files         470      529      +59     
  Lines       37301    40160    +2859     
==========================================
+ Hits        28299    30683    +2384     
- Misses       7071     7457     +386     
- Partials     1931     2020      +89     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@kdkasad kdkasad force-pushed the fix/rewrite-files-log-path branch from 70b8b69 to e76141c Compare June 3, 2025 17:40
@kdkasad
Copy link
Contributor Author

kdkasad commented Jun 3, 2025

Updated to (hopefully) fix lint errors.

@kdkasad kdkasad requested a review from jkowalski June 3, 2025 17:41
The dirRelativePath variable is actually the path to the file being
checked, but was treated as if it was the path to the parent directory,
causing the filename to be duplicated in log messages.
@kdkasad kdkasad force-pushed the fix/rewrite-files-log-path branch from e76141c to b70be5f Compare June 3, 2025 17:50
@julio-lopez julio-lopez requested a review from Copilot June 5, 2025 04:59
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR fixes duplicated filenames in the output of kopia snapshot fix commands by using the full file path instead of reconstructing it manually.

  • Renamed dirRelativePath to pathFromRoot in both rewriteEntry methods.
  • Updated log and verifier calls to use pathFromRoot directly, removing manual concatenation.

Reviewed Changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.

File Description
cli/command_snapshot_fix_remove_files.go Renamed parameter and adjusted log messages to use the full path from the root.
cli/command_snapshot_fix_invalid_files.go Similarly renamed parameter and updated verifier, log, and callback to use full path.
Comments suppressed due to low confidence (3)

cli/command_snapshot_fix_remove_files.go:30

  • [nitpick] Consider renaming pathFromRoot to a more concise name like entryPath or filePath to clearly reflect that this is the full file path being processed.
func (c *commandSnapshotFixRemoveFiles) rewriteEntry(ctx context.Context, pathFromRoot string, ent *snapshot.DirEntry) (*snapshot.DirEntry, error) {

cli/command_snapshot_fix_invalid_files.go:35

  • [nitpick] Consider renaming pathFromRoot to a concise name like entryPath or filePath for consistency and readability.
func (c *commandSnapshotFixInvalidFiles) rewriteEntry(ctx context.Context, pathFromRoot string, ent *snapshot.DirEntry) (*snapshot.DirEntry, error) {

cli/command_snapshot_fix_remove_files.go:30

  • Add unit tests for rewriteEntry to verify that the correct full file paths are logged when removing files, ensuring this fix is covered by tests.
func (c *commandSnapshotFixRemoveFiles) rewriteEntry(ctx context.Context, pathFromRoot string, ent *snapshot.DirEntry) (*snapshot.DirEntry, error) {

for _, id := range c.removeObjectIDs {
if ent.ObjectID.String() == id {
log(ctx).Infof("will remove file %v", path.Join(dirRelativePath, ent.Name))
log(ctx).Infof("will remove file %v", pathFromRoot)
Copy link

Copilot AI Jun 5, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use %s instead of %v when formatting a string path in Infof to make the intention explicit and avoid relying on default formatting.

Suggested change
log(ctx).Infof("will remove file %v", pathFromRoot)
log(ctx).Infof("will remove file %s", pathFromRoot)

Copilot uses AI. Check for mistakes.

if matched {
log(ctx).Infof("will remove file %v", path.Join(dirRelativePath, ent.Name))
log(ctx).Infof("will remove file %v", pathFromRoot)
Copy link

Copilot AI Jun 5, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use %s instead of %v when formatting a string path in Infof to make the intention explicit and avoid relying on default formatting.

Suggested change
log(ctx).Infof("will remove file %v", pathFromRoot)
log(ctx).Infof("will remove file %s", pathFromRoot)

Copilot uses AI. Check for mistakes.
if err := c.verifier.VerifyFile(ctx, ent.ObjectID, fname); err != nil {
log(ctx).Warnf("removing invalid file %v due to: %v", fname, err)
if err := c.verifier.VerifyFile(ctx, ent.ObjectID, pathFromRoot); err != nil {
log(ctx).Warnf("removing invalid file %v due to: %v", pathFromRoot, err)
Copy link

Copilot AI Jun 5, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use %s instead of %v when logging a string path to make the formatting clearer and more intentional.

Suggested change
log(ctx).Warnf("removing invalid file %v due to: %v", pathFromRoot, err)
log(ctx).Warnf("removing invalid file %s due to: %v", pathFromRoot, err)

Copilot uses AI. Check for mistakes.
@julio-lopez julio-lopez merged commit e5e64e9 into kopia:master Jun 5, 2025
23 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants