sql: Add exclude_data_from_backup storage option to ALTER TABLE#75295
Conversation
|
This change is |
30cbe8c to
b02120e
Compare
b02120e to
547e02c
Compare
SGTM! |
547e02c to
d6c53a9
Compare
pkg/sql/paramparse/paramobserver.go
Outdated
| return errors.New("cannot set data in a table with inbound foreign key constraints to be excluded from backup") | ||
| } | ||
|
|
||
| s, err := GetSingleBool(key, datum) |
There was a problem hiding this comment.
do you want to allow 'on' as well, as we do with outer variables?
if so, you may want to pull in 3bc72cf and use boolFromDatum
There was a problem hiding this comment.
Hmm not sure exclude_data_from_backup = on makes sense so leaving this as is for now
There was a problem hiding this comment.
it also allows 'true' as a string, and is standard for session vars (e.g. set experimental_enable_temp_tables = 'on')
There was a problem hiding this comment.
(although at the end of the day up to you)
There was a problem hiding this comment.
ah makes sense, adding it in. Thanks for the pointers.
There was a problem hiding this comment.
done! sprinkled 'on' and 'true' in the logictest too.
933dd52 to
738b23d
Compare
This change adds a storage parameter `exclude_data_from_backup` to mark a table's row data as excluded from a backup. `ALTER TABLE foo SET (exclude_data_from_backup = true);` The above ALTER TABLE stmt sets the `ExcludeDataFromBackup` field on a table descriptor ExcludeDataFromBackup specifies if the table's row data can be excluded from a backup targeting this table. This in turn means that the protected timestamp record written during the backup will exclude this table as a target, thereby not holding up GC on this table for the runtime of the backup. This is particularly useful for table's with high-churn that can then be configured with a low GC-TTL. It is important to note that we will still backup the table descriptor, and all other objects that reference this table egs: jobs, and so restoring such a table will result in an empty table on the restoring cluster. This change does not teach any part of the system to use this information from the table descriptor, that will come in follow up PRs. Informs: cockroachdb#73536 Release note: None
738b23d to
b557f36
Compare
|
TFTR! bors r=otan |
|
Build succeeded: |
|
cc: @kathancox I just updated this Release Note now that the feature is complete and being shipped in 22.1. Happy to file a docs issue with details if you prefer that? |
|
Hey @adityamaru, if you could open a docs issue that would be great. It would be good to start this in the next week! |
This change adds a storage parameter
exclude_data_from_backupto mark a table's row data as excluded from a backup.
ALTER TABLE foo SET (exclude_data_from_backup = true);The above ALTER TABLE stmt sets the
ExcludeDataFromBackupfieldon a table descriptor
ExcludeDataFromBackup specifies if the table's row data can be excluded
from a backup targeting this table. This in turn means that the protected
timestamp record written during the backup will exclude this table as a
target, thereby not holding up GC on this table for the runtime of the
backup. This is particularly useful for table's with high-churn that can
then be configured with a low GC-TTL.
It is important to note that we will still backup the table descriptor, and
all other objects that reference this table egs: jobs, and so restoring
such a table will result in an empty table on the restoring cluster.
This change does not teach any part of the system to use this information
from the table descriptor, that will come in follow up PRs.
Informs: #73536
Release note (sql change): Adds new
ALTER TABLE <table> SET (exclude_data_from_backup = true);syntax. This allows a user to exclude row data for a particular table during a backup. On restoring this backup, the table will be created in the restoring cluster but will be empty. This flag also prevents a cluster or database backup from holding up the garbage collection time to live (GC TTL) on the key span representing this table. This is particularly useful for high churn tables that wish to keep a much shorter GC TTL to prevent the buildup of garbage.