-
Notifications
You must be signed in to change notification settings - Fork 10.3k
Description
Proposal
#10421 added a new method ShardedPostings in IndexReader to filter a specific shard using provided shardIndex and shardCount parameter.
The implementation in #10421 uses hashmod to check if a series belong to the given shard. However, there are different ways to do that and hashmod is only one way. It would be good to use a configurable function to customize the sharding logic. This might not be used in Prometheus, but it would be useful for downstream projects.
There are two main implementations of ShardedPostings method:
index.Readerhttps://github.com/prometheus/prometheus/blob/main/tsdb/index/index.go#L1774headIndexReaderhttps://github.com/prometheus/prometheus/blob/main/tsdb/head_read.go
I am still unsure where I can add the logic to allow passing a custom function from downstream projects, especially for index.Reader. Maybe I have to change ShardedPostings method signature to
ShardedPostings(p index.Postings, shardIndex, shardCount uint64, shardFunc func(hash, shardIndex, shardCount uint64) bool) index.Postings
Any suggestion would be appreciated!