Skip to content

New archiver code#1494

Merged
fd0 merged 30 commits intomasterfrom
new-archiver
Apr 28, 2018
Merged

New archiver code#1494
fd0 merged 30 commits intomasterfrom
new-archiver

Conversation

@fd0
Copy link
Copy Markdown
Member

@fd0 fd0 commented Dec 23, 2017 •
edited
Loading

What is the purpose of this change? What does it change?

The commits in this PR replace the current archiver code with a rewritten, newly implemented archiver. The archiver is the part of restic which traverses the files and dirs, saves them to the repo and create snapshots. This is my WIP code, do not use it.

Was the change discussed in an issue or in the forum before?

Closes #549, #1286, #446, #1344, #1416, #1456, #1145, #1160

TODO

  • Add new archiver code
    • Excludes
    • Concurrency
    • Progress reporting
    • Abstract file system, allow reading from stdin with the same archiver code
    • Don't save atime by default
    • Upload intermediate indexes
    • UI polishing
    • Duration estimate
    • Display summary at the end of the backup run
  • Remove old archiver code
  • Remove other unused code
    * internal/pipe
    * internal/walk

Checklist

  • I have read the Contribution Guidelines
  • I have added tests for all changes in this PR
  • I have added documentation for the changes (in the manual)
  • There's an entry in the CHANGELOG.md file that describe the changes for our users
  • I have run gofmt on the code in all commits
  • All commit messages are formatted in the same style as the other commits in the repo
  • I'm done, this Pull Request is ready for review

@fd0 fd0 added the PR:WIP label Dec 23, 2017
@fd0 fd0 force-pushed the new-archiver branch 2 times, most recently from d61d5c1 to 061b5e8 Compare December 24, 2017 20:30
@fd0 fd0 mentioned this pull request Dec 27, 2017
Open
@fd0 fd0 force-pushed the new-archiver branch 2 times, most recently from cbc3dd9 to 2936f4b Compare January 3, 2018 21:29
@restic restic deleted a comment from codecov-io Jan 5, 2018
This was referenced Jan 5, 2018
Closed
Closed
@MJDSys MJDSys mentioned this pull request Jan 12, 2018
Closed
This was referenced Jan 13, 2018
Closed
Closed
@dimejo dimejo mentioned this pull request Jan 16, 2018
Closed
This was referenced Jan 19, 2018
Closed
Closed
@fd0 fd0 mentioned this pull request Jan 28, 2018
Closed
fd0 added a commit that referenced this pull request Jan 28, 2018
@fd0
Remove archiver progress "data processed" bandwith
cbfa516 
This commit removes the bandwidth displayed during backup process. It is
misleading and seldomly correct, because it's neither the "read
bandwidth" (only for the very first backup) nor the "upload bandwidth".
Many users are confused about (and rightly so), c.f. #1581, #1033, #1591

We'll eventually replace this display with something more relevant when
 #1494 is done.
@fd0 fd0 mentioned this pull request Jan 28, 2018
Closed
This was referenced Feb 20, 2018
Closed
Closed
Closed
@fd0 fd0 force-pushed the new-archiver branch 4 times, most recently from 594bed1 to 2c092d7 Compare March 30, 2018 15:30
@restic restic deleted a comment from houndci-bot Mar 30, 2018
@restic restic deleted a comment from houndci-bot Mar 30, 2018
@restic restic deleted a comment from houndci-bot Mar 30, 2018
fd0 added 4 commits April 25, 2018 14:42
@fd0
Add new archiver code
f279731
@fd0
Use new archiver code in tests
38926d8
@fd0
Add workaround for symlinked temp dir on darwin
a5c0cf2 
Chdir to the tempdir, then use os.Getwd() to get the name that
filepath.Abs() uses (and stores in the Snapshot).
@fd0
Add flags to set verbosity
9fac2ca
@fd0 fd0 force-pushed the new-archiver branch 2 times, most recently from 6012b35 to 7fe0493 Compare April 27, 2018 21:10
@mholt
Copy link
Copy Markdown
Contributor

mholt commented Apr 28, 2018

WOW this is a lot of code! Nice work, @fd0! Exciting to see this coming together, you're almost done!!

@fd0
Copy link
Copy Markdown
Member Author

fd0 commented Apr 28, 2018

I'm done, I'll merge this when the CI tests pass :)

@fd0 fd0 removed the PR:WIP label Apr 28, 2018
@fd0 fd0 merged commit c3cc5d7 into master Apr 28, 2018
fd0 added a commit that referenced this pull request Apr 28, 2018
@fd0
Merge pull request #1494 from restic/new-archiver
56e394a 
New archiver code
@fd0 fd0 deleted the new-archiver branch April 28, 2018 20:25
@fd0
Copy link
Copy Markdown
Member Author

fd0 commented Apr 28, 2018

And it's in! Wohoo! Four months of work :)

@fd0 fd0 mentioned this pull request May 5, 2018
Closed
@whereisaaron
Copy link
Copy Markdown

whereisaaron commented May 19, 2018

Now... about that #21 compression @fd0 😄

@fd0 fd0 mentioned this pull request May 22, 2018
Open
@whereisaaron whereisaaron mentioned this pull request May 23, 2018
Closed
@fd0 fd0 mentioned this pull request May 26, 2018
Closed
@fd0 fd0 mentioned this pull request Jun 8, 2018
Closed
@zcalusic zcalusic mentioned this pull request Jul 11, 2018
Closed
@MichaelEischer MichaelEischer mentioned this pull request Jan 9, 2021
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

1 more reviewer

@houndci-bot houndci-bot houndci-bot left review comments

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Unable to backup/restore files/dirs with same name