Skip to content
This repository was archived by the owner on Jul 24, 2024. It is now read-only.

Add lastbackupts argument to support incremental backup/restore#109

Merged
5kbpers merged 21 commits intopingcap:masterfrom
YangKeao:incre_backup
Dec 30, 2019
Merged

Add lastbackupts argument to support incremental backup/restore#109
5kbpers merged 21 commits intopingcap:masterfrom
YangKeao:incre_backup

Conversation

@YangKeao
Copy link
Member

@YangKeao YangKeao commented Dec 12, 2019

This PR has two part: backup and restore:

Most modification on backup is :

  1. Set StartVersion for BackupRequest

Most modification on restore is:

  1. Filter files with lastbackupts. Import only file after lastbackupts

Close #85

Signed-off-by: Yang Keao <keao.yang@yahoo.com>
Signed-off-by: Yang Keao <keao.yang@yahoo.com>
@YangKeao YangKeao requested a review from 5kbpers December 12, 2019 10:34
@YangKeao YangKeao changed the title [WIP] Add lastbackupts argument to support incremental backup/restore Add lastbackupts argument to support incremental backup/restore Dec 13, 2019
Signed-off-by: Yang Keao <keao.yang@yahoo.com>
Signed-off-by: Yang Keao <keao.yang@yahoo.com>
Signed-off-by: Yang Keao <keao.yang@yahoo.com>
Signed-off-by: Yang Keao <keao.yang@yahoo.com>
Signed-off-by: Yang Keao <keao.yang@yahoo.com>
Signed-off-by: Yang Keao <keao.yang@yahoo.com>
YangKeao and others added 4 commits December 15, 2019 14:17
Signed-off-by: 5kbpers <tangminghua@pingcap.com>
Signed-off-by: 5kbpers <tangminghua@pingcap.com>
Signed-off-by: 5kbpers <tangminghua@pingcap.com>
github.com/pingcap/check v0.0.0-20190102082844-67f458068fc8
github.com/pingcap/errors v0.11.4
github.com/pingcap/kvproto v0.0.0-20191212073621-373b0fec09a1
github.com/pingcap/kvproto v0.0.0-20191212110315-d6a9d626988c
Copy link
Member

@overvenus overvenus Dec 13, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please use kvproto@release-3.1 release-3.1 is no longer required since #112

cmd/validate.go Outdated
origin sha256 is %s`, file.Name, hex.EncodeToString(s[:]), hex.EncodeToString(file.Sha256))
backup data checksum failed: %s may be changed
calculated sha256 is %s,
origin sha256 is %s`,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please remove the leading \ts, otherwise, the log looks weird.

								backup data checksum failed: %s may be changed

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed

Signed-off-by: Yang Keao <keao.yang@yahoo.com>
Signed-off-by: Yang Keao <keao.yang@yahoo.com>
@overvenus
Copy link
Member

/run-integration-tests

@5kbpers
Copy link
Contributor

5kbpers commented Dec 26, 2019

/run-integration-tests

@codecov
Copy link

codecov bot commented Dec 26, 2019

Codecov Report

Merging #109 into master will decrease coverage by 4.68%.
The diff coverage is 67.27%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #109      +/-   ##
==========================================
- Coverage   77.56%   72.88%   -4.69%     
==========================================
  Files          33       33              
  Lines        3976     3223     -753     
==========================================
- Hits         3084     2349     -735     
+ Misses        597      585      -12     
+ Partials      295      289       -6
Impacted Files Coverage Δ
pkg/restore/client.go 72.9% <100%> (-1.59%) ⬇️
pkg/restore/util.go 87.5% <100%> (-4.14%) ⬇️
pkg/backup/client.go 73.47% <100%> (-7.61%) ⬇️
pkg/restore/db.go 46.42% <100%> (-9.46%) ⬇️
cmd/validate.go 67.48% <30.76%> (-8.57%) ⬇️
cmd/restore.go 64.34% <56.25%> (-13.79%) ⬇️
cmd/backup.go 67.76% <60%> (-13.28%) ⬇️
pkg/backup/safe_point.go 64.28% <0%> (-14.67%) ⬇️
pkg/utils/worker.go 72% <0%> (-12%) ⬇️
... and 8 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b362e1d...4d441e7. Read the comment docs.

Copy link
Contributor

@5kbpers 5kbpers left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Member

@overvenus overvenus left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rest LGTM

Signed-off-by: 5kbpers <tangminghua@pingcap.com>
Copy link
Member

@overvenus overvenus left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:shipit:

@5kbpers 5kbpers merged commit a9f72a1 into pingcap:master Dec 30, 2019
overvenus pushed a commit to overvenus/br-1 that referenced this pull request Dec 29, 2020
…p#109)

A few tests need to relaxed due to the coarser ID assignment
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants