-
Notifications
You must be signed in to change notification settings - Fork 4.1k
kvserver: consider increasing closed timestamp interval defaults #108656
Description
Currently, we publish rangefeed closed timestamp updates every 200ms. This is controlled by two settings:
kv.rangefeed.closed_timestamp_refresh_interval = 0(defaults tokv.closed_timestamp.side_transport_interval)kv.closed_timestamp.side_transport_interval = '200ms'
For the purposes of rangefeeds, this is overly aggressive, and can lead to excessive load with large numbers of ranges (see #96395). The target duration kv.closed_timestamp.target_duration = '3s' is one order of magnitude larger than the refresh interval, and the default changefeed min_checkpoint_frequency = '30s' is two orders of magnitude larger.
We should increase the default of either kv.rangefeed.closed_timestamp_refresh_interval or kv.closed_timestamp.side_transport_interval to e.g. 1 second, which is still sufficient for rangefeeds, and can significantly reduce CPU usage on nodes with many replicas.
Preferably kv.closed_timestamp.side_transport_interval, since users may already have set this higher (e.g. to 30 seconds), and defaulting kv.rangefeed.closed_timestamp_refresh_interval to e.g. 1 second instead of the side transport interval can cause significant load increases when the effective default changes from 30 seconds to 1 second.
Jira issue: CRDB-30568
Epic CRDB-26372