Skip to content

Read only required VIEW columns on the shards#83036

Merged
novikd merged 4 commits intomasterfrom
analyzer-remove-unused-projections-on-shards
Jul 4, 2025
Merged

Read only required VIEW columns on the shards#83036
novikd merged 4 commits intomasterfrom
analyzer-remove-unused-projections-on-shards

Conversation

@novikd
Copy link
Copy Markdown
Member

@novikd novikd commented Jul 1, 2025

Changelog category (leave one):

  • Bug Fix (user-visible misbehavior in an official stable release)

Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):

Fix performance degradation with the enabled analyzer when secondary queries always read all columns from the VIEWs. Fixes #81718.

Documentation entry for user-facing changes

  • Documentation is written (mandatory for new features)

@clickhouse-gh
Copy link
Copy Markdown
Contributor

clickhouse-gh bot commented Jul 1, 2025 •
edited
Loading

Workflow [PR], commit [c5c54e9]

Summary: ❌

job_name test_name status info comment
Stateless tests (amd_binary, old analyzer, s3 storage, DatabaseReplicated, 2/2) failure
Logical error thrown (see clickhouse-server.log or logical_errors.txt) FAIL

@clickhouse-gh clickhouse-gh bot added the pr-bugfix Pull request with bugfix, not backported by default label Jul 1, 2025
@KochetovNicolai KochetovNicolai self-assigned this Jul 2, 2025
KochetovNicolai
KochetovNicolai reviewed Jul 2, 2025
View reviewed changes
src/Analyzer/QueryTreePassManager.cpp
{
// Run only QueryAnalysisPass and GroupingFunctionsResolvePass passes.
run(query_tree_node, 3);
// Run only query tree passes that doesn't affect output header:
Copy link
Copy Markdown
Member

@KochetovNicolai KochetovNicolai Jul 2, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we can even have a special case for MV

@novikd
Copy link
Copy Markdown
Member Author

novikd commented Jul 4, 2025

Stateless tests (amd_binary, old analyzer, s3 storage, DatabaseReplicated, 2/2) — Failed: 0, Passed: 3899, Ski

#74734

@novikd novikd added the pr-must-backport Pull request should be backported intentionally. Use this label with great care! label Jul 4, 2025
@novikd novikd added this pull request to the merge queue Jul 4, 2025
Merged via the queue into master with commit 58886e9 Jul 4, 2025
119 of 122 checks passed
@novikd novikd deleted the analyzer-remove-unused-projections-on-shards branch July 4, 2025 08:47
@robot-ch-test-poll2 robot-ch-test-poll2 added the pr-synced-to-cloud The PR is synced to the cloud repo label Jul 4, 2025
@robot-clickhouse-ci-1 robot-clickhouse-ci-1 added pr-backports-created-cloud deprecated label, NOOP pr-must-backport-synced The `*-must-backport` labels are synced into the cloud Sync PR labels Jul 4, 2025
@robot-ch-test-poll1 robot-ch-test-poll1 mentioned this pull request Jul 4, 2025
Merged
robot-ch-test-poll1 added a commit that referenced this pull request Jul 4, 2025
@robot-ch-test-poll1
Merge pull request #83232 from ClickHouse/cherrypick/24.8/83036
e64292c 
Cherry pick #83036 to 24.8: Read only required VIEW columns on the shards
robot-clickhouse added a commit that referenced this pull request Jul 4, 2025
@robot-clickhouse
Backport #83036 to 24.8: Read only required VIEW columns on the shards
4fe8090
This was referenced Jul 4, 2025
Merged
Merged
robot-ch-test-poll1 added a commit that referenced this pull request Jul 4, 2025
@robot-ch-test-poll1
Merge pull request #83234 from ClickHouse/cherrypick/25.3/83036
5677370 
Cherry pick #83036 to 25.3: Read only required VIEW columns on the shards
robot-clickhouse added a commit that referenced this pull request Jul 4, 2025
@robot-clickhouse
Backport #83036 to 25.3: Read only required VIEW columns on the shards
4326db8
This was referenced Jul 4, 2025
Merged
Merged
robot-ch-test-poll1 added a commit that referenced this pull request Jul 4, 2025
@robot-ch-test-poll1
Merge pull request #83237 from ClickHouse/cherrypick/25.4/83036
e81d265 
Cherry pick #83036 to 25.4: Read only required VIEW columns on the shards
robot-clickhouse added a commit that referenced this pull request Jul 4, 2025
@robot-clickhouse
Backport #83036 to 25.4: Read only required VIEW columns on the shards
c78c6ce
@robot-ch-test-poll1 robot-ch-test-poll1 mentioned this pull request Jul 4, 2025
Merged
@robot-ch-test-poll1 robot-ch-test-poll1 mentioned this pull request Jul 4, 2025
Merged
robot-ch-test-poll1 added a commit that referenced this pull request Jul 4, 2025
@robot-ch-test-poll1
Merge pull request #83239 from ClickHouse/cherrypick/25.5/83036
365a1cf 
Cherry pick #83036 to 25.5: Read only required VIEW columns on the shards
robot-clickhouse added a commit that referenced this pull request Jul 4, 2025
@robot-clickhouse
Backport #83036 to 25.5: Read only required VIEW columns on the shards
5f44381
This was referenced Jul 4, 2025
Merged
Merged
robot-ch-test-poll1 added a commit that referenced this pull request Jul 4, 2025
@robot-ch-test-poll1
Merge pull request #83241 from ClickHouse/cherrypick/25.6/83036
450bbe6 
Cherry pick #83036 to 25.6: Read only required VIEW columns on the shards
robot-clickhouse added a commit that referenced this pull request Jul 4, 2025
@robot-clickhouse
Backport #83036 to 25.6: Read only required VIEW columns on the shards
95bf236
@robot-ch-test-poll1 robot-ch-test-poll1 mentioned this pull request Jul 4, 2025
Merged
@robot-ch-test-poll3 robot-ch-test-poll3 added the pr-backports-created Backport PRs are successfully created, it won't be processed by CI script anymore label Jul 4, 2025
clickhouse-gh bot added a commit that referenced this pull request Jul 4, 2025
@clickhouse-gh
Merge pull request #83242 from ClickHouse/backport/25.6/83036
03d17fa 
Backport #83036 to 25.6: Read only required VIEW columns on the shards
alexey-milovidov added a commit that referenced this pull request Jul 4, 2025
@alexey-milovidov
Merge pull request #83240 from ClickHouse/backport/25.5/83036
88edf52 
Backport #83036 to 25.5: Read only required VIEW columns on the shards
alexey-milovidov added a commit that referenced this pull request Jul 4, 2025
@alexey-milovidov
Merge pull request #83238 from ClickHouse/backport/25.4/83036
dfbbf2a 
Backport #83036 to 25.4: Read only required VIEW columns on the shards
novikd added a commit that referenced this pull request Jul 9, 2025
@novikd
Merge pull request #83233 from ClickHouse/backport/24.8/83036
c1b02bb 
Backport #83036 to 24.8: Read only required VIEW columns on the shards
novikd added a commit that referenced this pull request Aug 13, 2025
@novikd
Merge pull request #83235 from ClickHouse/backport/25.3/83036
73c64bc 
Backport #83036 to 25.3: Read only required VIEW columns on the shards
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@KochetovNicolai KochetovNicolai KochetovNicolai approved these changes

Assignees

@KochetovNicolai KochetovNicolai

Labels

pr-backports-created Backport PRs are successfully created, it won't be processed by CI script anymore pr-backports-created-cloud deprecated label, NOOP pr-bugfix Pull request with bugfix, not backported by default pr-must-backport Pull request should be backported intentionally. Use this label with great care! pr-must-backport-synced The `*-must-backport` labels are synced into the cloud Sync PR pr-synced-to-cloud The PR is synced to the cloud repo

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Query to remote table with enable_analyzer=1 try to read all table from remote or Distributed, instead of executing query remotely

5 participants

@novikd @KochetovNicolai @robot-ch-test-poll2 @robot-ch-test-poll3 @robot-clickhouse-ci-1