*: support parsing BACKUP and RESTORE statements by kennytm · Pull Request #746 · pingcap/parser

kennytm · 2020-02-17T21:08:09Z

What problem does this PR solve?

Added the BACKUP, RESTORE and SHOW BACKUP/RESTORE statements to parser.

BACKUP DATABASE * TO 'local:///tmp/archive01/';
BACKUP DATABASE `a` TO 'local:///tmp/archive02/';
BACKUP TABLE `a`.`b` TO 'local:///tmp/archive03/';
RESTORE DATABASE * FROM 'local:///tmp/archive01/';
RESTORE DATABASE `a` FROM 'local:///tmp/archive02/';
RESTORE TABLE `a`.`b` FROM 'local:///tmp/archive03/';
BACKUP DATABASE * FULL TO 'local:///tmp/archive04/';
BACKUP DATABASE * INCREMENTAL UNTIL TIMESTAMP '2020-02-02 14:15:16' TO 'local:///tmp/archive05/';
BACKUP DATABASE * INCREMENTAL UNTIL TIMESTAMP_ORACLE 1234567890 TO 'local:///tmp/archive06/';

BACKUP DATABASE * TO 's3://my_bucket/path' 
    RATE_LIMIT = 100 MB/SECOND 
    SNAPSHOT = 2 HOUR AGO 
    CONCURRENCY = 32 
    S3_REGION = 'us-west-1' 
    S3_USE_ACCELERATE_ENDPOINT = 1;

SHOW BACKUP;
SHOW RESTORE;
SHOW BACKUP LIKE 'b01234';
SHOW BACKUP WHERE status <> 'running';

These are needed for integrating BR into TiDB (PR will come later).

(Note: because the new unresolved keyword S3_USE_ACCELERATE_ENDPOINT is longer than all existing tokens, forcing gofmt to replace the entire token map, I've taken advantage of this to sort the entire tokenMap)

What is changed and how it works?

↑

Check List

Tests

Unit test

Code changes

Side effects

Related changes

Need to update the documentation
Need to be included in the release note

codecov · 2020-02-17T21:28:10Z

Codecov Report

Merging #746 into master will increase coverage by 0.21%.
The diff coverage is 81.42%.

@@            Coverage Diff             @@
##           master     #746      +/-   ##
==========================================
+ Coverage   77.82%   78.03%   +0.21%     
==========================================
  Files          40       40              
  Lines       14421    14632     +211     
==========================================
+ Hits        11223    11418     +195     
- Misses       2514     2535      +21     
+ Partials      684      679       -5

tangenta

LGTM

tangenta · 2020-02-18T09:10:14Z

parser.y

+	}
+|	"RATE_LIMIT" EqOpt LengthNum "MB" '/' "SECOND"
+	{
+		// TODO: check overflow?


How can it overflow ❔

@tangenta RATE_LIMIT = 9223372036854775807 MB/SECOND... admittedly this is not a real issue 😅

tangenta · 2020-02-18T09:14:18Z

parser_test.go

+		{"backup database * to 'noop://' rate_limit 500 MB/second snapshot 5 minute ago", true, "BACKUP DATABASE * TO 'noop://' RATE_LIMIT = 500 MB/SECOND SNAPSHOT = 300000000 MICROSECOND AGO"},
+		{"restore table g from 'noop://' concurrency 40 checksum 0 online 1", true, "RESTORE TABLE `g` FROM 'noop://' CONCURRENCY = 40 CHECKSUM = 0 ONLINE = 1"},
+		{
+			// FIXME: should we really include the access key in the Restore() text???


Should we also handle SetPwdStmt.Restore()?

😱. Perhaps deal with this later (e.g. add a WriteSecureString() method). This needs to be considered together with how TiDB uses Restore().

tiancaiamao · 2020-03-12T11:32:03Z

LGTM
With this feature implemented, preparing some test cases can be much faster.

ast/misc.go

IANTHEREAL · 2020-03-14T03:30:25Z

@kennytm please resolve conflicts, it seems to be merged soon

kennytm · 2020-03-15T11:33:39Z

PTAL again @tangenta @tiancaiamao thanks.

tangenta

LGTM

tiancaiamao · 2020-03-16T02:50:09Z

LGTM

* *: support parsing BACKUP and RESTORE statements * parser: add SNAPSHOT = 'str' option to BACKUP statement

kennytm added type/enhancement New feature or request status/PTAL labels Feb 17, 2020

kennytm requested a review from a team February 17, 2020 21:08

ghost requested review from tangenta and removed request for a team February 17, 2020 21:08

*: support parsing BACKUP and RESTORE statements

f9e078c

kennytm force-pushed the kennytm/backup-restore-statements branch from e62fb52 to f9e078c Compare February 17, 2020 21:24

tangenta reviewed Feb 18, 2020

View reviewed changes

tangenta added the status/LGT1 LGT1 label Feb 18, 2020

kennytm added 2 commits February 18, 2020 19:59

Merge branch 'master' into kennytm/backup-restore-statements

eb71593

Merge branch 'master' into kennytm/backup-restore-statements

237ca03

kennytm mentioned this pull request Mar 11, 2020

RFC + Support of BACKUP and RESTORE statements pingcap/tidb#15274

Merged

3pointer reviewed Mar 13, 2020

View reviewed changes

ast/misc.go Show resolved Hide resolved

kennytm added 2 commits March 15, 2020 18:23

Merge branch 'master' into kennytm/backup-restore-statements

9bcdc07

parser: add SNAPSHOT = 'str' option to BACKUP statement

c96fe86

kennytm removed the status/PTAL label Mar 15, 2020

tangenta reviewed Mar 16, 2020

View reviewed changes

tiancaiamao approved these changes Mar 16, 2020

View reviewed changes

Merge branch 'master' into kennytm/backup-restore-statements

ebeb937

kennytm merged commit c3235db into master Mar 16, 2020

kennytm deleted the kennytm/backup-restore-statements branch March 16, 2020 08:09

kennytm mentioned this pull request Mar 18, 2020

parser: recognize BACKUP … SNAPSHOT = TSO in additional to SNAPSHOT = 'TIMESTAMP' #782

Merged

kennytm mentioned this pull request Apr 13, 2020

Design syntax for using Lightning from SQL pingcap/tidb-lightning#293

Closed

sre-bot mentioned this pull request Apr 30, 2020

RFC + Support of BACKUP and RESTORE statements (#15274) pingcap/tidb#16960

Merged

tiancaiamao pushed a commit to tiancaiamao/parser that referenced this pull request Apr 27, 2021

*: support parsing BACKUP and RESTORE statements (pingcap#746)

ac9e370

* *: support parsing BACKUP and RESTORE statements * parser: add SNAPSHOT = 'str' option to BACKUP statement

lyonzhi pushed a commit to lyonzhi/parser that referenced this pull request Apr 25, 2024

*: support parsing BACKUP and RESTORE statements (pingcap#746)

e807cbc

* *: support parsing BACKUP and RESTORE statements * parser: add SNAPSHOT = 'str' option to BACKUP statement

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

*: support parsing BACKUP and RESTORE statements#746

*: support parsing BACKUP and RESTORE statements#746
kennytm merged 6 commits intomasterfrom
kennytm/backup-restore-statements

kennytm commented Feb 17, 2020 •

edited

Loading

Uh oh!

codecov bot commented Feb 17, 2020 •

edited

Loading

Uh oh!

tangenta left a comment

Uh oh!

tangenta Feb 18, 2020

Uh oh!

kennytm Feb 18, 2020

Uh oh!

tangenta Feb 18, 2020

Uh oh!

kennytm Feb 18, 2020

Uh oh!

tiancaiamao commented Mar 12, 2020

Uh oh!

Uh oh!

IANTHEREAL commented Mar 14, 2020

Uh oh!

kennytm commented Mar 15, 2020

Uh oh!

tangenta left a comment

Uh oh!

tiancaiamao commented Mar 16, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Conversation

kennytm commented Feb 17, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What problem does this PR solve?

What is changed and how it works?

Check List

Uh oh!

codecov bot commented Feb 17, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

tangenta left a comment

Choose a reason for hiding this comment

Uh oh!

tangenta Feb 18, 2020

Choose a reason for hiding this comment

Uh oh!

kennytm Feb 18, 2020

Choose a reason for hiding this comment

Uh oh!

tangenta Feb 18, 2020

Choose a reason for hiding this comment

Uh oh!

kennytm Feb 18, 2020

Choose a reason for hiding this comment

Uh oh!

tiancaiamao commented Mar 12, 2020

Uh oh!

Uh oh!

IANTHEREAL commented Mar 14, 2020

Uh oh!

kennytm commented Mar 15, 2020

Uh oh!

tangenta left a comment

Choose a reason for hiding this comment

Uh oh!

tiancaiamao commented Mar 16, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

kennytm commented Feb 17, 2020 •

edited

Loading

codecov bot commented Feb 17, 2020 •

edited

Loading