Skip to content

cdc,sql: core changefeeds don't work as expected with vectorized=on #55605

@ajwerner

Description

@ajwerner

Describe the problem

When a core changefeed runs, it pushes rows out one a time, when they are available. This seems at odds with the goals of the column materialized used in vectorized execution. The problem seems to be that the execution engine waits for a batch of rows and just hangs. This results in core changefeeds effectively not working with the vectorized execution engine.

To Reproduce

Create a table with a small number of rows.

SET CLUSTER SETTING kv.rangefeed.enabled=true;
CREATE TABLE foo(i INT PRIMARY KEY);
INSERT INTO foo VALUES (1), (2);
SET vectorize=on;
\set display_format=csv
EXPERIMENTAL CHANGEFEED FOR foo WITH resolved='5s'; -- this will just hang.

Expected behavior

The above should work. We need to either set a batch size of 1 or provide some other mechanism to sync the changefeed.

Environment:

  • CockroachDB version 20.1.6

Additional context

It seems that this bufferring in the materializer also leads to issues during shutdown and is the reason we see #55408.

Metadata

Metadata

Assignees

Labels

A-cdcChange Data CaptureA-sql-executionRelating to SQL execution.C-bugCode not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior.

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions