This is excellent, thanks!

This PR against the ClickBench repository is similar in spirit as @kitaisreal's Ursa (i.e. a research fork of ClickHouse). If all PRs are being integrated into the main codebase anyways, perhaps we don't need this PR (or can keep it open and continuosly update it for the time being)?

Thanks for the feedback!

I'd actually prefer to have this PR merged into the ClickBench repository for a few reasons:

• A continuously updated tuned baseline: This fork plays a similar role to clickhouse-tuned — a place where performance-oriented patches can be evaluated holistically. It gives us a stable, visible comparison point against upstream ClickHouse, even before all PRs are merged. This helps track net gains across batches of changes.

• Surfacing trade-offs and non-merged work: Not all optimizations may land in upstream immediately — some might be blocked due to generality, compatibility, or maintenance concerns. Keeping this variant in the ClickBench repo allows us to observe and quantify the impact of those changes, even if they're not ultimately accepted upstream. It serves as a real-world benchmark of possible trade-offs.

• Clear visibility of contribution impact: From my contributor's perspective, having this fork merged into ClickBench makes the performance gains more tangible and attributable. It is highly motivating and helps justify the continued investment.

This PR against the ClickBench repository is similar in spirit as @kitaisreal's Ursa (i.e. a research fork of ClickHouse).

Interesting — the string layout modification mentioned there is also implemented in ByConity (as BigString). I’ve encountered a similar need when working on the projection index feature (row-level index), where faster row seeking on string columns is critical. I’ll look into whether we can achieve this in a backward-compatible way.

As long as the results are reproducible, let's merge.

I’ll look into whether we can achieve this in a backward-compatible way.

Yes, it is entirely possible.

• name the new String data type String_v2, rename the old one to String_v1;
• introduce a table-level setting to interpret the String name as either String_v1 or String_v2;
• creation of a table with String will rewrite it to either String_v1 or String_v2;
• loading and ATTACHing a table with String will interpret it as String_v1;
• the native protocol will serialize String_v1 and String_v2 as the old String with conversion;

which applies max_rows_to_group_by for trivial GROUP BY LIMIT queries to avoid unnecessary aggregation work

Thanks, I wanted it for a long time!

Yes, it is entirely possible.

Hmm, I was actually thinking of a different strategy: keep using the same type, but recognize the underlying streams — and if there's a separate size stream, apply a new serde logic accordingly. This behavior would only apply to MergeTree's wide format, which I believe should be sufficient.

Maybe we can try. Although, having to look up an additional file looks hacky.
Another solution is to approach it as different column representations, like ColumnSparse (how do we store that a certain column is in the Sparse format?).

Maybe we can try. Although, having to look up an additional file looks hacky.
Another solution is to approach it as different column representations, like ColumnSparse (how do we store that a certain column is in the Sparse format?).

Sure, a different serde in serialization.json is definitely better than looking for files.

As long as the results are reproducible, let's merge.

I've just merged an additional optimization from my team that addresses the Q23 issue . With this fix, the results should now be fully reproducible without any manual post-processing.

I've updated benchmark.sh to use the following binary:

https://clickhouse-builds.s3.amazonaws.com/PRs/81944/cda07f8aca770d97ea149eec6b477dcfd59d134e/build_amd_release/clickhouse-common-static-25.7.1.1-amd64.tgz

@rschu1ze @nickitat Could you help re-run the benchmarks and update the results on both c6a.metal and c6a.4xlarge, as those were mentioned in the Firebolt PR as the most commonly used environments?

Thanks a lot!

I've implemented the separate size stream string format with .size subcolumn support and corresponding optimizations. However, the performance improvement is only significant in Q27, while other queries show negligible gains or even slight regressions.

Given this, it's not suitable to enable by default, so I’ll leave it disabled for now.

P.S. The results have been updated. @alexey-milovidov — if everything looks good, would you mind helping to merge this PR? Much appreciated!

@rschu1ze, thanks a lot! Just wondering — the entry appears on the leaderboard, but it only shows the name without rendering the actual benchmark results. Do you know what might be causing this?

It's because we added tuned field recently. Here is the fix #421 . cc @rschu1ze

Is there a plan to introduce a PR for "Push TopN threshold to MergeTreeSource"? There are some issues (ClickHouse/ClickHouse#65990, ClickHouse/ClickHouse#85081, I reported ClickHouse/ClickHouse#75098) that I imagine would benefit significantly from that optimization, is that correct?

Is there a plan to introduce a PR ...

There is definitely a plan, but since I already have several PRs pending, it will take some time to prepare and land it.

@amosbird, the build file used in this entry has disappeared. Please update.

@amosbird, the build file used in this entry has disappeared. Please update.

Affected binary:
https://clickhouse-builds.s3.amazonaws.com/PRs/81944/cda07f8aca770d97ea149eec6b477dcfd59d134e/build_amd_release/clickhouse-common-static-25.7.1.1-amd64.tgz

From PR message:

Benchmark results were generated using artifacts built by the official CI pipeline of ClickHouse/ClickHouse#81944, with great help from @nickitat — thank you!

@amosbird Simply rebase ClickHouse/ClickHouse#81944?

Sure. I'll rebase it today.