Skip to content

rangefeed: ensure correct use of exclusive start time #82488

@erikgrinaker

Description

@erikgrinaker

The start time of rangefeeds (i.e. RangeFeedRequest.Header.Timestamp) is exclusive; the first possible event (including catchup scans) will be at Timestamp.Next(). This allows using a previous checkpoint timestamp to resume a rangefeed without getting duplicates at the checkpoint timestamp, and shares the semantics of incremental backups (see e.g. storage.ExportOptions.StartTS).

However, as discussed in #82451 (comment), our rangefeed clients often assume that the start time is inclusive. This will lead to dropped messages if they happen to fall exactly on the start time.

We need to audit our client code -- e.g. pkg/kv/kvclient/rangefeed, pkg/ccl/changefeedccl/kvfeed, and CDC in general -- to make sure they use the exclusive timestamp semantics. Several call sites are known to get this wrong, e.g.:

https://github.com/cockroachdb/cockroach/blob/master/pkg/ccl/changefeedccl/kvfeed/kv_feed.go#L254

https://github.com/cockroachdb/cockroach/blob/master/pkg/kv/kvclient/kvcoord/dist_sender_rangefeed.go#L496-L496

Alternatively, we can change the API to use an inclusive start time. However, this would entail cross-version compatibility concerns that we may want to avoid.

Jira issue: CRDB-16418

Epic CRDB-9179

Metadata

Metadata

Assignees

Labels

A-cdcChange Data CaptureA-kv-replicationRelating to Raft, consensus, and coordination.C-bugCode not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior.T-cdc

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions