-
Notifications
You must be signed in to change notification settings - Fork 4.1k
coldata: vary coldata.BatchSize for tests #40791
Description
This issue tracks changing the coldata.BatchSize constant used to determine batch sizes to use in our vectorized operators to a package-level function that returns the value of a package-level private variable. This variable could be modified by exporting a public mutator that is only available when testing. We could then generate random batch sizes in a given range to test different batch sizes.
The ability to test different batch sizes greatly increases our test coverage given that unit tests usually do not have enough data to cause an operator to return multiple output batches (input batch sizes are easier to control). Doing this tests that operators manage their state correctly (e.g. are nulls and selection vectors being reset properly?).
Tests currently fail when manually lowering coldata.BatchSize so those failures should be investigated and fixed before proceeding with varying operators' batch sizes.