Skip to content

sql,kv: power ColIndexJoin by COL_BATCH_RESPONSE scan format #94807

@yuzefovich

Description

@yuzefovich

The initial work on the KV projection pushdown will only utilize the newly-introduced COL_BATCH_RESPONSE scan format for simple scans (i.e. TableReaders). We should extend that also to index joins since we already have the vectorized ColIndexJoin (unlike the lookup joins that still go through the row-by-row engine).

The main contribution here would be to teach the Streamer to use the new scan format.

Also an important question that will need to be answered as part of this item is whether we squash coldata.Batches from multiple requests into one or not. This is important since index joins read only a single row, so setting up the whole pushdown pipeline for each request separately can lead to significant overhead. One idea that was mentioned previously was to introduce a new request type (something like BatchedScanRequest) which would contain key spans that need to be scanned and can be returned as a single response. For index joins we don't care to know which particular input row resulted in which response row.

Jira issue: CRDB-23150

Metadata

Metadata

Assignees

No one assigned

    Labels

    C-enhancementSolution expected to add code/behavior + preserve backward-compat (pg compat issues are exception)T-sql-queriesSQL Queries Team

    Type

    No type

    Projects

    Status

    Backlog

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions