-
Notifications
You must be signed in to change notification settings - Fork 4k
Open
Description
Describe the enhancement requested
PyIceberg uses positional deletes to indicate which rows should be omitted. In the normal table.filter(..) a boolean mask is allowed, but not when constructing a scanner. Therefore we first have to read the data into a table, and then filter it. It would simplify the code (and also speed up) if this can be done directly when reading the data.
Having a __row_index where we can filter would greatly simplify this logic.
Component(s)
Python
wjones127 and legout