Skip to content

Projection Index Step 2: Enable multi-projection part filtering and improve projection introspection#80343

Merged
KochetovNicolai merged 16 commits intoClickHouse:masterfrom
amosbird:projection-index-4-pre
Jun 6, 2025
Merged

Projection Index Step 2: Enable multi-projection part filtering and improve projection introspection#80343
KochetovNicolai merged 16 commits intoClickHouse:masterfrom
amosbird:projection-index-4-pre

Conversation

@amosbird
Copy link
Copy Markdown
Collaborator

@amosbird amosbird commented May 16, 2025

Changelog category (leave one):

  • Improvement

Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):

Enable multiple-projection filtering support, allowing to use more than one projection for part-level filtering. This addresses #55525. This is the second step to implement projection index, following #78429.

This PR also adds projection analysis introspection ability: explain projections=1, showing which projections are selected and how they are applied.

This PR also enables PREWHERE support in projections and query condition cache for projections.

Documentation entry for user-facing changes

  • Documentation is written (mandatory for new features)

@clickhouse-gh
Copy link
Copy Markdown
Contributor

clickhouse-gh bot commented May 16, 2025 •
edited
Loading

Workflow [PR], commit [4839aef]

@clickhouse-gh clickhouse-gh bot added the pr-improvement Pull request with some product improvements label May 16, 2025
@amosbird amosbird force-pushed the projection-index-4-pre branch from 888d214 to b828808 Compare May 17, 2025 21:18
@amosbird
Copy link
Copy Markdown
Collaborator Author

amosbird commented May 18, 2025

Integration tests (release, 1/4) — fail: 1

#80317

@amosbird
Copy link
Copy Markdown
Collaborator Author

amosbird commented May 19, 2025

Stateless tests (ubsan) — Server died, fail: 12, passed: 2342, skipped: 32

#80418

@KochetovNicolai KochetovNicolai self-assigned this May 23, 2025
@amosbird amosbird force-pushed the projection-index-4-pre branch 2 times, most recently from 2c59d89 to 86de4ef Compare May 27, 2025 08:46
@amosbird
Copy link
Copy Markdown
Collaborator Author

amosbird commented May 27, 2025

Stateless tests (release, ParallelReplicas, s3 storage)

02458_insert_select_progress_tcp timeout error. unrelated

PR / AST fuzzer (amd_ubsan) (pull_request)Failing after 22m

#80917

@amosbird amosbird force-pushed the projection-index-4-pre branch from 86de4ef to b3cdbfa Compare May 29, 2025 14:26
@amosbird amosbird mentioned this pull request May 30, 2025
Merged
1 task
baibaichen pushed a commit to Kyligence/ClickHouse that referenced this pull request Jun 27, 2025
@zzcclp
[GLUTEN-4454][CH] Support write mergetree data for CH backend (#480)
ed23b8a 
(cherry picked from commit d3f342a)

Fix rebase issue:
- 20250607 ClickHouse#80343
baibaichen pushed a commit to Kyligence/ClickHouse that referenced this pull request Jun 28, 2025
@zzcclp
[GLUTEN-4454][CH] Support write mergetree data for CH backend (#480)
b64f21e 
(cherry picked from commit d3f342a)

Fix rebase issue:
- 20250607 ClickHouse#80343
baibaichen pushed a commit to Kyligence/ClickHouse that referenced this pull request Jun 29, 2025
@zzcclp
[GLUTEN-4454][CH] Support write mergetree data for CH backend (#480)
58c9c81 
(cherry picked from commit d3f342a)

Fix rebase issue:
- 20250607 ClickHouse#80343
kyligence-git pushed a commit to Kyligence/ClickHouse that referenced this pull request Jun 29, 2025
@zzcclp
[GLUTEN-4454][CH] Support write mergetree data for CH backend (#480)
1827242 
(cherry picked from commit d3f342a)

Fix rebase issue:
- 20250607 ClickHouse#80343
baibaichen pushed a commit to Kyligence/ClickHouse that referenced this pull request Jul 3, 2025
@zzcclp
[GLUTEN-4454][CH] Support write mergetree data for CH backend (#480)
537d99a 
(cherry picked from commit d3f342a)

Fix rebase issue:
- 20250607 ClickHouse#80343
baibaichen pushed a commit to Kyligence/ClickHouse that referenced this pull request Jul 4, 2025
@zzcclp
[GLUTEN-4454][CH] Support write mergetree data for CH backend (#480)
4a12684 
(cherry picked from commit d3f342a)

Fix rebase issue:
- 20250607 ClickHouse#80343
baibaichen pushed a commit to Kyligence/ClickHouse that referenced this pull request Jul 5, 2025
@zzcclp
[GLUTEN-4454][CH] Support write mergetree data for CH backend (#480)
3e6330a 
(cherry picked from commit d3f342a)

Fix rebase issue:
- 20250607 ClickHouse#80343
kyligence-git pushed a commit to Kyligence/ClickHouse that referenced this pull request Jul 6, 2025
@zzcclp
[GLUTEN-4454][CH] Support write mergetree data for CH backend (#480)
15a7ccb 
(cherry picked from commit d3f342a)

Fix rebase issue:
- 20250607 ClickHouse#80343
baibaichen pushed a commit to Kyligence/ClickHouse that referenced this pull request Jul 7, 2025
@zzcclp
[GLUTEN-4454][CH] Support write mergetree data for CH backend (#480)
465f197 
(cherry picked from commit d3f342a)

Fix rebase issue:
- 20250607 ClickHouse#80343
kyligence-git pushed a commit to Kyligence/ClickHouse that referenced this pull request Jul 7, 2025
@zzcclp
[GLUTEN-4454][CH] Support write mergetree data for CH backend (#480)
a9ba460 
(cherry picked from commit d3f342a)

Fix rebase issue:
- 20250607 ClickHouse#80343
kyligence-git pushed a commit to Kyligence/ClickHouse that referenced this pull request Jul 8, 2025
@zzcclp
[GLUTEN-4454][CH] Support write mergetree data for CH backend (#480)
00606f1 
(cherry picked from commit d3f342a)

Fix rebase issue:
- 20250607 ClickHouse#80343
kyligence-git pushed a commit to Kyligence/ClickHouse that referenced this pull request Jul 9, 2025
@zzcclp
[GLUTEN-4454][CH] Support write mergetree data for CH backend (#480)
16441eb 
(cherry picked from commit d3f342a)

Fix rebase issue:
- 20250607 ClickHouse#80343
kyligence-git pushed a commit to Kyligence/ClickHouse that referenced this pull request Jul 10, 2025
@zzcclp
[GLUTEN-4454][CH] Support write mergetree data for CH backend (#480)
e67d129 
(cherry picked from commit d3f342a)

Fix rebase issue:
- 20250607 ClickHouse#80343
kyligence-git pushed a commit to Kyligence/ClickHouse that referenced this pull request Jul 11, 2025
@zzcclp
[GLUTEN-4454][CH] Support write mergetree data for CH backend (#480)
b763e7d 
(cherry picked from commit d3f342a)

Fix rebase issue:
- 20250607 ClickHouse#80343
baibaichen pushed a commit to Kyligence/ClickHouse that referenced this pull request Jul 14, 2025
@zzcclp
[GLUTEN-4454][CH] Support write mergetree data for CH backend (#480)
af87df8 
(cherry picked from commit d3f342a)

Fix rebase issue:
- 20250607 ClickHouse#80343
kyligence-git pushed a commit to Kyligence/ClickHouse that referenced this pull request Jul 14, 2025
@zzcclp
[GLUTEN-4454][CH] Support write mergetree data for CH backend (#480)
a32a628 
(cherry picked from commit d3f342a)

Fix rebase issue:
- 20250607 ClickHouse#80343
kyligence-git pushed a commit to Kyligence/ClickHouse that referenced this pull request Jul 15, 2025
@zzcclp
[GLUTEN-4454][CH] Support write mergetree data for CH backend (#480)
0aad7c9 
(cherry picked from commit d3f342a)

Fix rebase issue:
- 20250607 ClickHouse#80343
baibaichen pushed a commit to Kyligence/ClickHouse that referenced this pull request Jul 18, 2025
@zzcclp
[GLUTEN-4454][CH] Support write mergetree data for CH backend (#480)
e83e31c 
(cherry picked from commit d3f342a)

Fix rebase issue:
- 20250607 ClickHouse#80343
baibaichen pushed a commit to Kyligence/ClickHouse that referenced this pull request Jul 18, 2025
@zzcclp
[GLUTEN-4454][CH] Support write mergetree data for CH backend (#480)
0b26a84 
(cherry picked from commit d3f342a)

Fix rebase issue:
- 20250607 ClickHouse#80343
baibaichen pushed a commit to Kyligence/ClickHouse that referenced this pull request Jul 20, 2025
@zzcclp
[GLUTEN-4454][CH] Support write mergetree data for CH backend (#480)
b0f7eaa 
(cherry picked from commit d3f342a)

Fix rebase issue:
- 20250607 ClickHouse#80343
kyligence-git pushed a commit to Kyligence/ClickHouse that referenced this pull request Jul 20, 2025
@zzcclp
[GLUTEN-4454][CH] Support write mergetree data for CH backend (#480)
3fa06e5 
(cherry picked from commit d3f342a)

Fix rebase issue:
- 20250607 ClickHouse#80343
kyligence-git pushed a commit to Kyligence/ClickHouse that referenced this pull request Jul 21, 2025
@zzcclp
[GLUTEN-4454][CH] Support write mergetree data for CH backend (#480)
77945c0 
(cherry picked from commit d3f342a)

Fix rebase issue:
- 20250607 ClickHouse#80343
baibaichen pushed a commit to Kyligence/ClickHouse that referenced this pull request Jul 28, 2025
@zzcclp
[GLUTEN-4454][CH] Support write mergetree data for CH backend (#480)
780401e 
(cherry picked from commit d3f342a)

Fix rebase issue:
- 20250607 ClickHouse#80343
baibaichen pushed a commit to Kyligence/ClickHouse that referenced this pull request Jul 29, 2025
@zzcclp
[GLUTEN-4454][CH] Support write mergetree data for CH backend (#480)
828a7f5 
(cherry picked from commit d3f342a)

Fix rebase issue:
- 20250607 ClickHouse#80343
kyligence-git pushed a commit to Kyligence/ClickHouse that referenced this pull request Jul 29, 2025
@zzcclp
[GLUTEN-4454][CH] Support write mergetree data for CH backend (#480)
6c4935c 
(cherry picked from commit d3f342a)

Fix rebase issue:
- 20250607 ClickHouse#80343
kyligence-git pushed a commit to Kyligence/ClickHouse that referenced this pull request Jul 30, 2025
@zzcclp
[GLUTEN-4454][CH] Support write mergetree data for CH backend (#480)
01e8a6c 
(cherry picked from commit d3f342a)

Fix rebase issue:
- 20250607 ClickHouse#80343
kyligence-git pushed a commit to Kyligence/ClickHouse that referenced this pull request Jul 31, 2025
@zzcclp
[GLUTEN-4454][CH] Support write mergetree data for CH backend (#480)
997b665 
(cherry picked from commit d3f342a)

Fix rebase issue:
- 20250607 ClickHouse#80343
kyligence-git pushed a commit to Kyligence/ClickHouse that referenced this pull request Aug 1, 2025
@zzcclp
[GLUTEN-4454][CH] Support write mergetree data for CH backend (#480)
452c19b 
(cherry picked from commit d3f342a)

Fix rebase issue:
- 20250607 ClickHouse#80343
kyligence-git pushed a commit to Kyligence/ClickHouse that referenced this pull request Aug 2, 2025
@zzcclp
[GLUTEN-4454][CH] Support write mergetree data for CH backend (#480)
fbd5e98 
(cherry picked from commit d3f342a)

Fix rebase issue:
- 20250607 ClickHouse#80343
kyligence-git pushed a commit to Kyligence/ClickHouse that referenced this pull request Aug 3, 2025
@zzcclp
[GLUTEN-4454][CH] Support write mergetree data for CH backend (#480)
22f49ec 
(cherry picked from commit d3f342a)

Fix rebase issue:
- 20250607 ClickHouse#80343
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@KochetovNicolai KochetovNicolai KochetovNicolai approved these changes

Assignees

@KochetovNicolai KochetovNicolai

Labels

pr-improvement Pull request with some product improvements pr-synced-to-cloud The PR is synced to the cloud repo

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@amosbird @KochetovNicolai @robot-ch-test-poll1