backup: re-enable fast incremental BACKUP via TBI

dt · dt · commit 94bef6574ad4 · 2020-03-21T02:56:33.000Z
Within the current BACKUP/RESTORE feature offering, the way to reduce RPO is to backup more often. For this to work for more frequent durations, the cost of backing up what changed must scale with the size what changed, as opposed to the total data size. This is exactly what the time-bound iterator optimization is supposed to deliver -- by recording the span of timestamps that appear in any given SSTable and then installing a filter to only open sstables that contain relevant times, we can ignore irrelevant data very cheaply. However over 2017 and 2018, we encountered a few tricky correctness issues in the interaction of the time-bound iterator and our MVCC logic and handling intents, and generally lost confidence in this code, ultimately disabling its use in the scans used by BACKUP. However #45785 resolved those concerns by ensuring that every key emitted by the IncrementalIterator was actually read by a normal, non-TBI iterator even when TBI is used. Thus it is now believed to be safe to re-enable TBI for incremental backups. Release note (enterprise change): Incremental BACKUP can quickly skip of unchanged data making frequent incremental BACKUPs 10-100x faster depending on data-size and frequency. Release justification: low-risk and high-impact.
diff --git a/pkg/ccl/backupccl/backup_planning.go b/pkg/ccl/backupccl/backup_planning.go
@@ -67,7 +67,7 @@ var fullClusterSystemTables = []string{
 var useTBI = settings.RegisterBoolSetting(
 	"kv.bulk_io_write.experimental_incremental_export_enabled",
 	"use experimental time-bound file filter when exporting in BACKUP",
-	false,
+	true,
 )
 
 var backupOptionExpectValues = map[string]sql.KVStringOptValidate{

Original file line number	Diff line number	Diff line change
`@@ -67,7 +67,7 @@ var fullClusterSystemTables = []string{`
`67`	`67`	`var useTBI = settings.RegisterBoolSetting(`
`68`	`68`	`"kv.bulk_io_write.experimental_incremental_export_enabled",`
`69`	`69`	`"use experimental time-bound file filter when exporting in BACKUP",`
`70`		`- false,`
	`70`	`+ true,`
`71`	`71`	`)`
`72`	`72`
`73`	`73`	`var backupOptionExpectValues = map[string]sql.KVStringOptValidate{`