This is a tracking issue that captures an idea proposed by @dt.

Some users have encountered challenges with tables that contain high-churn, ephemeral data, where they want to GC history in that table quickly, but cannot simply configure it to do so because it is contained within a database, tenant or cluster that is backed up incrementally, which requires that history be preserved until that backup can back it up.

In many such cases, the user does not actually explicitly desire data in these tables to be backed up at all, and are only doing so because it is part of the larger database, tenant or cluster being backed up. Thus one possible solution would be to provide a mechanism for a table like this to opt out of inclusion in backups, freeing it to configure more aggressive GC without affecting the ability of its containing database to backup. Simply excluding the table entirely could be challenging -- it may still be referenced in jobs, logs, views, etc. However excluding its row data would allow the backup to succeed even if its that table's span has been GCed, and would result in it just being empty when restored.

Approach:

One proposed approach would be to mark a table as ephemeral. The simplest solution would be to store this information on the TableDescriptor but this does not work because of the way tenants are backed up. A BACKUP TENANT from the system tenant simply backs up a single-tenant span, and does not look inside the tenant to be able to exclude certain tables. Thus, this information needs to be written to the ZoneConfig, and will subsequently rely on the transport infrastructure that was introduced to support zcfgs in a multi-tenant environment.

The tasks can then largely be broken up into:

• Add epehemeral bit to ZoneConfig for a table, and the corresponding SpanConfig created as a result of reconciliation.

• Teach ExportRequest to return an empty ExportResponse when attempting to export a replica that has an assocaited SpanConfig marked as ephemeral.

• Teach GC to ignore any protected timestamp records that might be written to a replica that is marked as ephemeral. This way, even if a backup were to protect a set of schema objects that included the ephemeral table, it would allow the low GC TTL to continue garbage collecting the data.

• Add a migration to mark system tables that opt-out from a cluster backup as ephemeral so that they are never backed up or protected.

• Add SQL to set/unset a table as ephemeral. This might work out of the box with ALTER ZONE CONFIGURATION.

• Think about how to make the fact that a table is ephemeral observable to the user during a backup. To avoid surprises where they expect a table to be a part of the backup, but it is silently ignored. This might be solved via documentation, but it is worth calling out explicitly.

Epic: CRDB-10306

Jira issue: CRDB-11627