-
Notifications
You must be signed in to change notification settings - Fork 4.1k
sql/schemachanger: backfills should not use protected timestamp for their entire life span #140629
Copy link
Copy link
Closed
Labels
C-enhancementSolution expected to add code/behavior + preserve backward-compat (pg compat issues are exception)Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception)T-sql-foundationsSQL Foundations Team (formerly SQL Schema + SQL Sessions)SQL Foundations Team (formerly SQL Schema + SQL Sessions)branch-masterFailures and bugs on the master branch.Failures and bugs on the master branch.branch-release-24.1Used to mark GA and release blockers, technical advisories, and bugs for 24.1Used to mark GA and release blockers, technical advisories, and bugs for 24.1branch-release-24.3Used to mark GA and release blockers, technical advisories, and bugs for 24.3Used to mark GA and release blockers, technical advisories, and bugs for 24.3branch-release-25.1v24.1.18v24.3.12v25.1.6
Description
Currently, when an index backfill is occurring the primary index will have a protected timestamp on it until the table is completely backfilled. Sadly this can be problematic at larger data scales, since we could end up holding a protected timestamp for large data scales if the backfill spans days, which can have impact on foreground work and space usage.
We can instead opt for a model where after the initial scan we can move the timestamp forward after some number of checkpoints. Additionally, we can also install them as a last resort if the GC interval is short when the time nears completing these chunks of work.
This would be similar to the suggestion found here: #98311 (comment)
Jira issue: CRDB-47273
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
C-enhancementSolution expected to add code/behavior + preserve backward-compat (pg compat issues are exception)Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception)T-sql-foundationsSQL Foundations Team (formerly SQL Schema + SQL Sessions)SQL Foundations Team (formerly SQL Schema + SQL Sessions)branch-masterFailures and bugs on the master branch.Failures and bugs on the master branch.branch-release-24.1Used to mark GA and release blockers, technical advisories, and bugs for 24.1Used to mark GA and release blockers, technical advisories, and bugs for 24.1branch-release-24.3Used to mark GA and release blockers, technical advisories, and bugs for 24.3Used to mark GA and release blockers, technical advisories, and bugs for 24.3branch-release-25.1v24.1.18v24.3.12v25.1.6