-
Notifications
You must be signed in to change notification settings - Fork 4.1k
cdc,sql: core changefeeds don't work as expected with vectorized=on #55605
Copy link
Copy link
Closed
Labels
A-cdcChange Data CaptureChange Data CaptureA-sql-executionRelating to SQL execution.Relating to SQL execution.C-bugCode not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior.Code not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior.
Description
Describe the problem
When a core changefeed runs, it pushes rows out one a time, when they are available. This seems at odds with the goals of the column materialized used in vectorized execution. The problem seems to be that the execution engine waits for a batch of rows and just hangs. This results in core changefeeds effectively not working with the vectorized execution engine.
To Reproduce
Create a table with a small number of rows.
SET CLUSTER SETTING kv.rangefeed.enabled=true;
CREATE TABLE foo(i INT PRIMARY KEY);
INSERT INTO foo VALUES (1), (2);
SET vectorize=on;
\set display_format=csv
EXPERIMENTAL CHANGEFEED FOR foo WITH resolved='5s'; -- this will just hang.
Expected behavior
The above should work. We need to either set a batch size of 1 or provide some other mechanism to sync the changefeed.
Environment:
- CockroachDB version 20.1.6
Additional context
It seems that this bufferring in the materializer also leads to issues during shutdown and is the reason we see #55408.
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
A-cdcChange Data CaptureChange Data CaptureA-sql-executionRelating to SQL execution.Relating to SQL execution.C-bugCode not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior.Code not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior.