Skip to content

jobs: needs to work without liveness and nodeID #47892

@tbg

Description

@tbg

The jobs framework currently uses node liveness to function. For multi-tenancy phase 2, jobs will run on SQL tenant processes which do not have access to the liveness table and which additionally don't have a node ID. This is problematic because jobs uses both to reason about the liveness of jobs via the lease.

As a redeeming feature, phase 2 will have only a single SQL tenant process running on behalf of a given tenant, meaning that job liveness is simple - jobs are always live and jobs that are not running on the tenant should always be resumed.

Past phase 2, we are likely looking at multiple SQL tenant processes running at the same time, and we'll need a discovery/liveness mechanism between them. However, multiple mechanisms will need such a mechanism (for example DistSQL), so it won't have to be developed just for jobs.

Metadata

Metadata

Assignees

Labels

A-multitenancyRelated to multi-tenancyX-anchored-telemetryThe issue number is anchored by telemetry references.

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions