Skip to content

dump: add --compress flag to compress archives#5054

Merged
MichaelEischer merged 4 commits intorestic:masterfrom
phillipp:dump-compress-zip
Oct 16, 2024
Merged

dump: add --compress flag to compress archives#5054
MichaelEischer merged 4 commits intorestic:masterfrom
phillipp:dump-compress-zip

Conversation

@phillipp
Copy link
Copy Markdown
Contributor

@phillipp phillipp commented Sep 13, 2024

What does this PR change? What problem does it solve?

When using dump and exporting an archive, the archive was not compressed. This PR adds a flag to do so.

When using dump to export a zip archive, the archive was not compressed. This PR enables compression for these archives.

Was the change previously discussed in an issue or on the forum?

No, I don't think so.

Checklist

  • I have read the contribution guidelines.
  • I have enabled maintainer edits.
  • I have added tests for all code changes.
  • I have added documentation for relevant changes (in the manual).
  • There's a new file in changelog/unreleased/ that describes the changes for our users (see template).
  • I have run gofmt on the code in all commits.
  • All commit messages are formatted in the same style as the other commits in the repo.
  • I'm done! This pull request is ready for review.

Target string
Archive string
Target string
CompressZipArchive bool
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just Compress should be fine. Considering, we can compress a .tar after archiving.

initSingleSnapshotFilter(flags, &dumpOptions.SnapshotFilter)
flags.StringVarP(&dumpOptions.Archive, "archive", "a", "tar", "set archive `format` as \"tar\" or \"zip\"")
flags.StringVarP(&dumpOptions.Target, "target", "t", "", "write the output to target `path`")
flags.BoolVar(&dumpOptions.CompressZipArchive, "compress-zip", false, "compress zip archive contents")
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we do a shorthand c as well? You can check if there any conflicts with the global options.

Comment on lines +3 to +6
# Describe the problem in the past tense, the new behavior in the present
# tense. Mention the affected commands, backends, operating systems, etc.
# Focus on user-facing behavior, not the implementation.
# Use "Restic now ..." instead of "We have changed ...".
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These comments can be excluded from the changelog.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, must have slipped by...

@phillipp
Copy link
Copy Markdown
Contributor Author

@konidev20 It would only be a little more effort to add compression to the tar stream as well, I guess that would be more consistent when renaming the flag to --compress (-c is unused, so we can use that too). Do you agree? because then I would add that?

@phillipp
Copy link
Copy Markdown
Contributor Author

I've added compression for .tar as well and updated the docs and flag name.

I can squash the commits later for better readability

@phillipp phillipp changed the title dump: add --compress-zip flag to compress zip archives dump: add --compress flag to compress archives Sep 13, 2024
Copy link
Copy Markdown
Contributor

@konidev20 konidev20 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. I will let Michael do the final review.

Copy link
Copy Markdown
Member

@MichaelEischer MichaelEischer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The changes look good so far, I just have a few small remarks. Also please squash the commits.

initSingleSnapshotFilter(flags, &dumpOptions.SnapshotFilter)
flags.StringVarP(&dumpOptions.Archive, "archive", "a", "tar", "set archive `format` as \"tar\" or \"zip\"")
flags.StringVarP(&dumpOptions.Target, "target", "t", "", "write the output to target `path`")
flags.BoolVarP(&dumpOptions.Compress, "compress", "c", false, "compress archive contents. When enabled, the DEFLATE algorithm is applied for \"zip\" archives, and the gzip algorithm for \"tar\" archives, resulting in a .tar.gz or .tgz file. (default: false)")
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This description is IMO way too long and verbose. This inline help text in the CLI should be short and concise, with verbose documentation being in the regular docs.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah but where can I add something to the docs but not to the CLI, i thought the docs where generated from the cli source? Am I missing something?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The sources for the docs at restic.readthedocs.io are in the docs/ folder in this repo, see https://github.com/restic/restic/tree/master/doc .

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, and they are auto-generated, see #5054 (comment)

No changes to the docs itself allowed

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes and no.

  • The docs at restic.readthedocs.io are indeed auto-generated by RTD, but the sources for them are at https://github.com/restic/restic/tree/master/doc and you are totally fine to edit them to add contents to the official documentation for restic, when it makes sense to do so.
  • The documentation for/in the CLI of restic is auto-generated too, as mentioned in the comment you linked to, and those files should not be edited, that is correct. But they are a different thing than the official docs on RTD.

@rawtaz
Copy link
Copy Markdown
Contributor

rawtaz commented Sep 15, 2024

@MichaelEischer and I were discussing this PR earlier today and would like to make some changes. Namely:

  • Make DEFLATE compression of zip archives the default, i.e. without needing to be enabled with an option or such.
  • Leave out compression of tar archives for now, the reason being that it is so simple to compress the tar archive afterwards, e.g. by restic dump ... | gzip.
  • Leave out the --compress option as it is no longer needed.

@phillipp
Copy link
Copy Markdown
Contributor Author

Updated the PR, much simpler now.

Copy link
Copy Markdown
Member

@MichaelEischer MichaelEischer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. I've applied the suggestion from @greatroar and minimally extended the test case.

@MichaelEischer MichaelEischer added this pull request to the merge queue Oct 16, 2024
Merged via the queue into restic:master with commit 618f306 Oct 16, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants