Skip to content

backupccl: various user defined types fixes and tests#52033

Merged
craig[bot] merged 1 commit intocockroachdb:masterfrom
rohany:more-backup-restore
Jul 30, 2020
Merged

backupccl: various user defined types fixes and tests#52033
craig[bot] merged 1 commit intocockroachdb:masterfrom
rohany:more-backup-restore

Conversation

@rohany
Copy link
Copy Markdown
Contributor

@rohany rohany commented Jul 28, 2020

This commit fixes some bugs introduced with the ability to change and
drop types, and adds some more tests. In particular it

  • Ensures that backup+restore maintain type backreferences to tables
    that use them, as well as ensuring that backups that use existing
    types get those backreferences installed on the existing descriptors.
  • Adds a test to restore to an existing type that is a logical
    "superset" of an existing type.
  • Adds test to ensure created and dropped types are reflected in
    incremenetal backups and restore of those backups.

Fixes #50321.
Fixes #51787.

Release note: None

@rohany rohany requested review from a team, dt and pbardea July 28, 2020 20:56
@cockroach-teamcity
Copy link
Copy Markdown
Member

This change is Reviewable

@rohany rohany force-pushed the more-backup-restore branch from 222a2d3 to c693caa Compare July 28, 2020 22:06
Copy link
Copy Markdown
Contributor

@pbardea pbardea left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM - just left a few comments in the tests.

Reviewed 5 of 8 files at r1.
Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @dt, @pbardea, and @rohany)


pkg/ccl/backupccl/backup_test.go, line 1300 at r1 (raw file):

	defer log.Scope(t).Close(t)

	// TODO (rohany): Add tests for backup/restore with revision history and

Should we keep the revision history part of this TODO here until those tests are added as well?


pkg/ccl/backupccl/backup_test.go, line 1528 at r1 (raw file):

			sqlDB.Exec(t, `INSERT INTO d2.t2 VALUES (ARRAY['hi'::d2.greeting])`)

			// d.t and d.t2 should both have back references to d2.greeting.

The tables that have the backreference here are d2.t and d2.t2, right?


pkg/ccl/backupccl/backup_test.go, line 1582 at r1 (raw file):

			sqlDB.Exec(t, `RESTORE TABLE d.t FROM 'nodelocal://0/test/' WITH into_db = 'd7'`)
			sqlDB.Exec(t, `INSERT INTO d7.t VALUES ('greetings')`)
			sqlDB.CheckQueryResults(t, `SELECT * FROM d7.t ORDER BY x`, [][]string{{"hello"}, {"greetings"}, {"howdy"}})

Might also be worth adding the backreference check here as well?


pkg/ccl/backupccl/restore_job.go, line 1053 at r1 (raw file):

				for _, id := range typeIDs {
					// If the used type is part of the set of descriptors being restored,
					// then don't do anything.

nit: can we change "don't do anything" to something like "there's no need to add a backreference, since it already should exist".

This commit fixes some bugs introduced with the ability to change and
drop types, and adds some more tests. In particular it
* Ensures that backup+restore maintain type backreferences to tables
  that use them, as well as ensuring that backups that use existing
  types get those backreferences installed on the existing descriptors.
* Adds a test to restore to an existing type that is a logical
  "superset" of an existing type.
* Adds test to ensure created and dropped types are reflected in
  incremenetal backups and restore of those backups.

Fixes cockroachdb#50321.
Fixes cockroachdb#51787.

Release note: None
@rohany rohany force-pushed the more-backup-restore branch from c693caa to fc3c078 Compare July 30, 2020 14:24
Copy link
Copy Markdown
Contributor Author

@rohany rohany left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @dt and @pbardea)


pkg/ccl/backupccl/backup_test.go, line 1300 at r1 (raw file):

Previously, pbardea (Paul Bardea) wrote…

Should we keep the revision history part of this TODO here until those tests are added as well?

woops, got a little trigger happy


pkg/ccl/backupccl/backup_test.go, line 1528 at r1 (raw file):

Previously, pbardea (Paul Bardea) wrote…

The tables that have the backreference here are d2.t and d2.t2, right?

Yup, updated.


pkg/ccl/backupccl/backup_test.go, line 1582 at r1 (raw file):

Previously, pbardea (Paul Bardea) wrote…

Might also be worth adding the backreference check here as well?

Done


pkg/ccl/backupccl/restore_job.go, line 1053 at r1 (raw file):

Previously, pbardea (Paul Bardea) wrote…

nit: can we change "don't do anything" to something like "there's no need to add a backreference, since it already should exist".

Done

@rohany
Copy link
Copy Markdown
Contributor Author

rohany commented Jul 30, 2020

TFTR!

bors r+

@craig
Copy link
Copy Markdown
Contributor

craig bot commented Jul 30, 2020

Build succeeded:

@craig craig bot merged commit 53b0f0c into cockroachdb:master Jul 30, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

backupccl: restores referencing existing types need to track backreferences backup: test changes on user defined types and incremental backup

3 participants