Skip to content

Do not load inactive parts at startup#42181

Merged
CurtizJ merged 35 commits intoClickHouse:masterfrom
CurtizJ:optimize-loading-parts
Jan 16, 2023
Merged

Do not load inactive parts at startup#42181
CurtizJ merged 35 commits intoClickHouse:masterfrom
CurtizJ:optimize-loading-parts

Conversation

@CurtizJ
Copy link
Copy Markdown
Member

@CurtizJ CurtizJ commented Oct 8, 2022

Changelog category (leave one):

  • Performance Improvement

Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):

Do not load inactive parts at startup of MergeTree tables.

Resolves #4699.

@robot-ch-test-poll robot-ch-test-poll added the pr-performance Pull request with some performance improvements label Oct 8, 2022
@CurtizJ
Copy link
Copy Markdown
Member Author

CurtizJ commented Oct 8, 2022

Most likely it won't work with transactions now. Will fix it later.

alexey-milovidov
alexey-milovidov reviewed Oct 8, 2022
View reviewed changes
@tavplubix tavplubix self-assigned this Oct 10, 2022
@CurtizJ CurtizJ force-pushed the optimize-loading-parts branch from c932f45 to 92ac938 Compare November 21, 2022 17:02
@CurtizJ CurtizJ force-pushed the optimize-loading-parts branch from 7e8b388 to 3da799f Compare November 22, 2022 16:10
@CurtizJ CurtizJ force-pushed the optimize-loading-parts branch from 3da799f to 75c8076 Compare November 22, 2022 16:11
@CurtizJ CurtizJ force-pushed the optimize-loading-parts branch from 5a5a955 to f4d91cd Compare November 24, 2022 03:31
@CurtizJ
Copy link
Copy Markdown
Member Author

CurtizJ commented Nov 24, 2022

One of hung queries should be fixed by #43458.

@CurtizJ
Copy link
Copy Markdown
Member Author

CurtizJ commented Nov 25, 2022

I had to remove test since it doesn't work well because of #37664.

@CurtizJ CurtizJ marked this pull request as ready for review December 15, 2022 14:34
@CurtizJ CurtizJ requested a review from tavplubix December 20, 2022 22:32
tavplubix
tavplubix reviewed Dec 22, 2022
View reviewed changes
@qoega
Copy link
Copy Markdown
Member

qoega commented Dec 29, 2022

Sanitizer assert (in stderr.log) looks related
https://s3.amazonaws.com/clickhouse-test-reports/42181/8d2788445befc456c39cc13aac1c4250be5a014d/stress_test__msan_/stderr.log

@CurtizJ CurtizJ force-pushed the optimize-loading-parts branch from f1cab4e to 1e5d6e4 Compare January 9, 2023 13:08
@CurtizJ CurtizJ requested a review from tavplubix January 11, 2023 16:26
@tavplubix tavplubix mentioned this pull request Jan 12, 2023
Closed
@tavplubix
Copy link
Copy Markdown
Member

tavplubix commented Jan 12, 2023

Integration tests (tsan) [2/6] failure is related, see #44929 (comment)

@CurtizJ
Copy link
Copy Markdown
Member Author

CurtizJ commented Jan 16, 2023

All checks are green.

@CurtizJ CurtizJ merged commit 6863cd1 into ClickHouse:master Jan 16, 2023
@aalexfvk aalexfvk mentioned this pull request Mar 31, 2023
Merged
@myeongjjun myeongjjun mentioned this pull request Jan 23, 2026
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@alexey-milovidov alexey-milovidov alexey-milovidov left review comments

@tavplubix tavplubix tavplubix approved these changes

Assignees

@tavplubix tavplubix

Labels

pr-performance Pull request with some performance improvements

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Speed up loading of data parts at startup.

6 participants

@CurtizJ @qoega @tavplubix @alexey-milovidov @robot-ch-test-poll @alesapin