-
Notifications
You must be signed in to change notification settings - Fork 105
[Go][Parquet] Does this implementation support page indexes? #33
Description
I would like to create page indexes (per this doc) for a column when writing a parquet file, and then use that index to seek to a particular row. I have a case where the file is sorted by a particular ID, and the queries often want to start with a particular ID and then read all rows thereafter. Without an index, I can find the right row group using statistics, but then have to scan through all values in the column in the row group to find the ID and determine the right row.
I've gone through all of the code and API in the parquet package and sub-packages, and all I can find are columns in Thrift-generated code for this and accessors in the column chunk metadata that return the file offset for where the index is stored. But there seems to be no API to actually read the index and use it. And there is no configuration, on the write side, for whether to create an index or not.
When will this be supported? How active is development on the Go runtime?