Skip to content

coldata: vary coldata.BatchSize for tests #40791

@asubiotto

Description

@asubiotto

This issue tracks changing the coldata.BatchSize constant used to determine batch sizes to use in our vectorized operators to a package-level function that returns the value of a package-level private variable. This variable could be modified by exporting a public mutator that is only available when testing. We could then generate random batch sizes in a given range to test different batch sizes.

The ability to test different batch sizes greatly increases our test coverage given that unit tests usually do not have enough data to cause an operator to return multiple output batches (input batch sizes are easier to control). Doing this tests that operators manage their state correctly (e.g. are nulls and selection vectors being reset properly?).

Tests currently fail when manually lowering coldata.BatchSize so those failures should be investigated and fixed before proceeding with varying operators' batch sizes.

Metadata

Metadata

Assignees

Labels

A-sql-vecSQL vectorized engineC-enhancementSolution expected to add code/behavior + preserve backward-compat (pg compat issues are exception)

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions