Skip to content

kvserver: consider increasing closed timestamp interval defaults #108656

@erikgrinaker

Description

@erikgrinaker

Currently, we publish rangefeed closed timestamp updates every 200ms. This is controlled by two settings:

  • kv.rangefeed.closed_timestamp_refresh_interval = 0 (defaults to kv.closed_timestamp.side_transport_interval)
  • kv.closed_timestamp.side_transport_interval = '200ms'

For the purposes of rangefeeds, this is overly aggressive, and can lead to excessive load with large numbers of ranges (see #96395). The target duration kv.closed_timestamp.target_duration = '3s' is one order of magnitude larger than the refresh interval, and the default changefeed min_checkpoint_frequency = '30s' is two orders of magnitude larger.

We should increase the default of either kv.rangefeed.closed_timestamp_refresh_interval or kv.closed_timestamp.side_transport_interval to e.g. 1 second, which is still sufficient for rangefeeds, and can significantly reduce CPU usage on nodes with many replicas.

Preferably kv.closed_timestamp.side_transport_interval, since users may already have set this higher (e.g. to 30 seconds), and defaulting kv.rangefeed.closed_timestamp_refresh_interval to e.g. 1 second instead of the side transport interval can cause significant load increases when the effective default changes from 30 seconds to 1 second.

Jira issue: CRDB-30568

Epic CRDB-26372

Metadata

Metadata

Assignees

Labels

A-kv-closed-timestampsRelating to closed timestampsC-performancePerf of queries or internals. Solution not expected to change functional behavior.

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions