Currently, we publish rangefeed closed timestamp updates every 200ms. This is controlled by two settings:
kv.rangefeed.closed_timestamp_refresh_interval = 0 (defaults to kv.closed_timestamp.side_transport_interval)
kv.closed_timestamp.side_transport_interval = '200ms'
For the purposes of rangefeeds, this is overly aggressive, and can lead to excessive load with large numbers of ranges (see #96395). The target duration kv.closed_timestamp.target_duration = '3s' is one order of magnitude larger than the refresh interval, and the default changefeed min_checkpoint_frequency = '30s' is two orders of magnitude larger.
We should increase the default of either kv.rangefeed.closed_timestamp_refresh_interval or kv.closed_timestamp.side_transport_interval to e.g. 1 second, which is still sufficient for rangefeeds, and can significantly reduce CPU usage on nodes with many replicas.
Preferably kv.closed_timestamp.side_transport_interval, since users may already have set this higher (e.g. to 30 seconds), and defaulting kv.rangefeed.closed_timestamp_refresh_interval to e.g. 1 second instead of the side transport interval can cause significant load increases when the effective default changes from 30 seconds to 1 second.
Jira issue: CRDB-30568
Epic CRDB-26372
Currently, we publish rangefeed closed timestamp updates every 200ms. This is controlled by two settings:
kv.rangefeed.closed_timestamp_refresh_interval = 0(defaults tokv.closed_timestamp.side_transport_interval)kv.closed_timestamp.side_transport_interval = '200ms'For the purposes of rangefeeds, this is overly aggressive, and can lead to excessive load with large numbers of ranges (see #96395). The target duration
kv.closed_timestamp.target_duration = '3s'is one order of magnitude larger than the refresh interval, and the default changefeedmin_checkpoint_frequency = '30s'is two orders of magnitude larger.We should increase the default of either
kv.rangefeed.closed_timestamp_refresh_intervalorkv.closed_timestamp.side_transport_intervalto e.g. 1 second, which is still sufficient for rangefeeds, and can significantly reduce CPU usage on nodes with many replicas.Preferably
kv.closed_timestamp.side_transport_interval, since users may already have set this higher (e.g. to 30 seconds), and defaultingkv.rangefeed.closed_timestamp_refresh_intervalto e.g. 1 second instead of the side transport interval can cause significant load increases when the effective default changes from 30 seconds to 1 second.Jira issue: CRDB-30568
Epic CRDB-26372