Skip to content

jobs: migrate jobs to write their Payload and Progress to the jobs_info table #97762

@adityamaru

Description

@adityamaru

In #84866 we introduced a system.jobs_info table that is to be used to store job-specific state. This table allows a job to access and update information about its execution without adversely impacting the execution of other jobs. This is explained in detail in #82638.

As a first step, we want to teach jobs to write their Payload and Progress in their current form but to the jobs_info table instead of the jobs table. This migration will roughly involve:

  1. Double writes for the Payload and Progress of jobs to both the jobs_info and jobs table.

  2. A migration to backfill job_info with the Payload and Progress of all job records in the system.jobs table.

  3. A change to read methods in the registry to pull the Payload and Progress from the job_info table instead of the jobs table.

In the future, jobs may want to decompose their Payload and Progress protos further and write different fields as their own rows in the jobs_info table. This is outside the scope of this issue.

Jira issue: CRDB-24874

Epic CRDB-8964

Metadata

Metadata

Assignees

Labels

A-disaster-recoveryA-jobsC-enhancementSolution expected to add code/behavior + preserve backward-compat (pg compat issues are exception)GA-blockerT-jobsbranch-release-23.1Used to mark GA and release blockers, technical advisories, and bugs for 23.1

Type

No type

Projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions