-
Notifications
You must be signed in to change notification settings - Fork 4.1k
rangefeed: ensure correct use of exclusive start time #82488
Description
The start time of rangefeeds (i.e. RangeFeedRequest.Header.Timestamp) is exclusive; the first possible event (including catchup scans) will be at Timestamp.Next(). This allows using a previous checkpoint timestamp to resume a rangefeed without getting duplicates at the checkpoint timestamp, and shares the semantics of incremental backups (see e.g. storage.ExportOptions.StartTS).
However, as discussed in #82451 (comment), our rangefeed clients often assume that the start time is inclusive. This will lead to dropped messages if they happen to fall exactly on the start time.
We need to audit our client code -- e.g. pkg/kv/kvclient/rangefeed, pkg/ccl/changefeedccl/kvfeed, and CDC in general -- to make sure they use the exclusive timestamp semantics. Several call sites are known to get this wrong, e.g.:
https://github.com/cockroachdb/cockroach/blob/master/pkg/ccl/changefeedccl/kvfeed/kv_feed.go#L254
Alternatively, we can change the API to use an inclusive start time. However, this would entail cross-version compatibility concerns that we may want to avoid.
Jira issue: CRDB-16418
Epic CRDB-9179