Skip to content

Introduce system.background_schedule_pool{,_log} to improve introspection of background jobs#91157

Merged
azat merged 7 commits intoClickHouse:masterfrom
azat:bg-sched-pool-introspection
Dec 1, 2025
Merged

Introduce system.background_schedule_pool{,_log} to improve introspection of background jobs#91157
azat merged 7 commits intoClickHouse:masterfrom
azat:bg-sched-pool-introspection

Conversation

@azat
Copy link
Copy Markdown
Member

@azat azat commented Nov 29, 2025

Changelog category (leave one):

  • Improvement

Changelog entry (a user-readable short description of the changes that goes into CHANGELOG.md):

Introduce system.background_schedule_pool{,_log} to improve introspection of background jobs.

Various tables (and not only tables) can run periodic background tasks (that will be reflected no in system.background_schedule_pool{,_log}) which previously lack of the introspection tools except for system.text_log. Now they all will be available in system.background_schedule_pool and you can see how much time does it takes now. And also system.background_schedule_pool_log added, that includes all executed tasks with some meta information (duration, exceptions).

And I've seen cases when this tasks can work incorrectly, i.e. taking too much time, and it was hard to debug before. This PR should improve this.

@clickhouse-gh
Copy link
Copy Markdown
Contributor

clickhouse-gh bot commented Nov 29, 2025

Workflow [PR], commit [7ce9cc4b]

Summary:

job_name test_name status info comment
Stateless tests (arm_asan, targeted) failure
02174_cte_scalar_cache_mv FAIL cidb
BuzzHouse (amd_debug) failure
Logical error: 'Inconsistent AST formatting: the query: FAIL cidb, issue
BuzzHouse (amd_tsan) failure
Segmentation fault (STID: None) FAIL cidb
BuzzHouse (amd_ubsan) failure
UndefinedBehaviorSanitizer: undefined behavior (STID: 4443-51fd) FAIL cidb
Integration tests (amd_asan, flaky) error

@clickhouse-gh clickhouse-gh bot added the pr-improvement Pull request with some product improvements label Nov 29, 2025
@azat azat requested review from CheSema and removed request for CheSema November 29, 2025 15:17
@azat azat force-pushed the bg-sched-pool-introspection branch from 7db4a8b to a3a9a15 Compare November 29, 2025 15:24
azat added a commit to azat/chdig that referenced this pull request Nov 29, 2025
azat added a commit to azat/chdig that referenced this pull request Nov 29, 2025
azat added a commit to azat/chdig that referenced this pull request Nov 29, 2025
azat added a commit to azat/chdig that referenced this pull request Nov 29, 2025
- Add missing event_date/event_time filters
- Per table tasks
- Tasks view
- Per-task logs
- Show number of tasks per table

Refs: ClickHouse/ClickHouse#91157
@azat azat enabled auto-merge December 1, 2025 17:53
@azat
Copy link
Copy Markdown
Member Author

azat commented Dec 1, 2025

Failed: Integration tests (amd_asan, flaky)

It will never pass, it tries to run all tests, requires at least #91194

@azat azat added this pull request to the merge queue Dec 1, 2025
Merged via the queue into ClickHouse:master with commit 45c9aad Dec 1, 2025
244 of 256 checks passed
@azat azat deleted the bg-sched-pool-introspection branch December 1, 2025 18:28
@robot-clickhouse-ci-2 robot-clickhouse-ci-2 added the pr-synced-to-cloud The PR is synced to the cloud repo label Dec 1, 2025
@rschu1ze
Copy link
Copy Markdown
Member

rschu1ze commented Dec 2, 2025

@azat We'll also need a markdown documentation file for the two new system tables each here.

@azat
Copy link
Copy Markdown
Member Author

azat commented Dec 18, 2025

We'll also need a markdown documentation file for the two new system tables each here.

It is pretty internal, but since you asked here it is - #92587

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

pr-improvement Pull request with some product improvements pr-synced-to-cloud The PR is synced to the cloud repo

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants