-
Notifications
You must be signed in to change notification settings - Fork 4.1k
jobs: migrate jobs to write their Payload and Progress to the jobs_info table #97762
Description
In #84866 we introduced a system.jobs_info table that is to be used to store job-specific state. This table allows a job to access and update information about its execution without adversely impacting the execution of other jobs. This is explained in detail in #82638.
As a first step, we want to teach jobs to write their Payload and Progress in their current form but to the jobs_info table instead of the jobs table. This migration will roughly involve:
-
Double writes for the Payload and Progress of jobs to both the jobs_info and jobs table.
-
A migration to backfill job_info with the Payload and Progress of all job records in the system.jobs table.
-
A change to read methods in the registry to pull the Payload and Progress from the job_info table instead of the jobs table.
In the future, jobs may want to decompose their Payload and Progress protos further and write different fields as their own rows in the jobs_info table. This is outside the scope of this issue.
Jira issue: CRDB-24874
Epic CRDB-8964