Existing benchmark tool is dated. The way it's implemented makes it difficult to introduce new data types or custom workloads into it. Its support for cluster mode is also very limited. I suggest we collect requirements and consider rewriting it from scratch.
Some use cases that I would like to be supported in the future:
- Support for custom data types, such as JSON, vectors, etc.
- Support for mixed workloads, such as read/write/delete in certain order with ability to operate on a given key across multiple operations, potentially with some delay.
- Better cluster mode support (ability to read from replicas), so we can test maximum read throughput of the cluster.
- Support for distributed execution (in order to be able to saturate larger clusters)
- Metrics publishing (support for prometheus), so we can see detailed performance information over time tagged with different workloads, and be able to aggregate the data for distributed runs more easily.
This issue is an attempt to gauge interest in this effort from the valkey community and gather additional use cases that hypothetical rewrite should address.
Existing benchmark tool is dated. The way it's implemented makes it difficult to introduce new data types or custom workloads into it. Its support for cluster mode is also very limited. I suggest we collect requirements and consider rewriting it from scratch.
Some use cases that I would like to be supported in the future:
This issue is an attempt to gauge interest in this effort from the valkey community and gather additional use cases that hypothetical rewrite should address.