Fix sparse columns aggregation for any overflow mode by korowa · Pull Request #88440 · ClickHouse/ClickHouse

korowa · 2025-10-13T09:36:17Z

Closes #88417
Closes #87615

(both issues have this method in stacktraces)

Similar to regular add method, there should be a check that place value is not null.

Changelog category (leave one):

Bug Fix (user-visible misbehavior in an official stable release)

Changelog entry (a user-readable short description of the changes that goes into CHANGELOG.md):

Fix aggregation of sparse columns when group_by_overflow_mode is set to any

Documentation entry for user-facing changes

Documentation is written (mandatory for new features)

Similar to regular `add` method, there should be a check that `place` value is not null.

clickhouse-gh · 2025-10-13T11:15:54Z

Workflow [PR], commit [6ab65e0]

Summary: ❌

job_name	test_name	status	info	comment
Integration tests (amd_asan, old analyzer, 4/6)		failure
	test_insert_into_distributed/test.py::test_inserts_low_cardinality	FAIL

antaljanosbenjamin

Nicely done 👍

korowa · 2025-10-14T08:41:30Z

Integration tests (amd_asan, old analyzer, 4/6)

Test failure is not related -- the problem with the test (and not only this one) is that node may fail to start DDLWorker and exit with the following error

stacktrace

2025.10.13 13:58:28.875575 [ 680 ] {} <Error> DDLWorker: ZooKeeper error: Code: 999. Coordination::Exception: Transaction failed (Node exists): Op #0, path: /clickhouse/task_queue/replicas/localhost:9000/active. (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below):

0. ./contrib/llvm-project/libcxx/include/__exception/exception.h:113: Poco::Exception::Exception(String const&, int) @ 0x00000000499908f1
1. ./ci/tmp/build/./src/Common/Exception.cpp:129: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x0000000020eaae73
2. DB::Exception::Exception(String&&, int, String, bool) @ 0x000000000df816f7
3. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000df8082a
4. ./src/Common/Exception.h:141: Coordination::Exception::Exception<Coordination::Error&, unsigned long&>(Coordination::Error, FormatStringHelperImpl<std::type_identity<Coordination::Error&>::type, std::type_identity<unsigned long&>::type>, Coordination::Error&, unsigned long&) @ 0x000000003d98eee4
5. ./ci/tmp/build/./src/Common/ZooKeeper/ZooKeeper.cpp:1603: zkutil::KeeperMultiException::KeeperMultiException(Coordination::Error, unsigned long, std::vector<std::shared_ptr<Coordination::Request>, std::allocator<std::shared_ptr<Coordination::Request>>> const&, std::vector<std::shared_ptr<Coordination::Response>, std::allocator<std::shared_ptr<Coordination::Response>>> const&) @ 0x000000003d976fd6
6. ./ci/tmp/build/./src/Common/ZooKeeper/ZooKeeper.cpp:1627: zkutil::KeeperMultiException::check(Coordination::Error, std::vector<std::shared_ptr<Coordination::Request>, std::allocator<std::shared_ptr<Coordination::Request>>> const&, std::vector<std::shared_ptr<Coordination::Response>, std::allocator<std::shared_ptr<Coordination::Response>>> const&) @ 0x000000003d9606c8
7. ./ci/tmp/build/./src/Common/ZooKeeper/ZooKeeper.cpp:748: zkutil::ZooKeeper::multi(std::vector<std::shared_ptr<Coordination::Request>, std::allocator<std::shared_ptr<Coordination::Request>>> const&, bool) @ 0x000000003d9604c4
8. ./ci/tmp/build/./src/Interpreters/DDLWorker.cpp:1394: DB::DDLWorker::markReplicasActive(bool) @ 0x00000000311cff4b
9. ./ci/tmp/build/./src/Interpreters/DDLWorker.cpp:1145: DB::DDLWorker::initializeMainThread() @ 0x00000000311c841c
10. ./ci/tmp/build/./src/Interpreters/DDLWorker.cpp:1212: DB::DDLWorker::runMainThread() @ 0x000000003119af2b
11. ./contrib/llvm-project/libcxx/include/__type_traits/invoke.h:117: ThreadFromGlobalPoolImpl<true, true>::ThreadFromGlobalPoolImpl<void (DB::DDLWorker::*)(), DB::DDLWorker*>(void (DB::DDLWorker::*&&)(), DB::DDLWorker*&&)::'lambda'()::operator()() @ 0x00000000311d4ee3
12. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x00000000211c62ae
13. ./contrib/llvm-project/libcxx/include/__type_traits/invoke.h:117: void* std::__thread_proxy[abi:ne190107]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void (ThreadPoolImpl<std::thread>::ThreadFromThreadPool::*)(), ThreadPoolImpl<std::thread>::ThreadFromThreadPool*>>(void*) @ 0x00000000211d4e50
14. asan_thread_start(void*) @ 0x000000000df2be67
15. ? @ 0x0000000000094ac3
16. ? @ 0x0000000000126850
 (version 25.10.1.2375). Failed to start DDLWorker.
2025.10.13 13:58:28.875747 [ 680 ] {} <Fatal> : Logical error: 'false'.
2025.10.13 13:58:28.888049 [ 680 ] {} <Fatal> : Stack trace (when copying this message, always include the lines below):

0. ./ci/tmp/build/./src/Common/StackTrace.cpp:395: StackTrace::StackTrace() @ 0x000000002103b527
1. ./ci/tmp/build/./src/Common/Exception.cpp:57: DB::abortOnFailedAssertion(String const&) @ 0x0000000020ea9b6e
2. ./ci/tmp/build/./src/Interpreters/DDLWorker.cpp:1155: DB::DDLWorker::initializeMainThread() @ 0x00000000311c8ae7
3. ./ci/tmp/build/./src/Interpreters/DDLWorker.cpp:1212: DB::DDLWorker::runMainThread() @ 0x000000003119af2b
4. ./contrib/llvm-project/libcxx/include/__type_traits/invoke.h:117: ThreadFromGlobalPoolImpl<true, true>::ThreadFromGlobalPoolImpl<void (DB::DDLWorker::*)(), DB::DDLWorker*>(void (DB::DDLWorker::*&&)(), DB::DDLWorker*&&)::'lambda'()::operator()() @ 0x00000000311d4ee3
5. ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ? @ 0x00000000211c62ae
6. ./contrib/llvm-project/libcxx/include/__type_traits/invoke.h:117: void* std::__thread_proxy[abi:ne190107]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void (ThreadPoolImpl<std::thread>::ThreadFromThreadPool::*)(), ThreadPoolImpl<std::thread>::ThreadFromThreadPool*>>(void*) @ 0x00000000211d4e50
7. asan_thread_start(void*) @ 0x000000000df2be67
8. ? @ 0x0000000000094ac3
9. ? @ 0x0000000000126850

2025.10.13 13:58:28.888394 [ 12 ] {} <Fatal> BaseDaemon: ########## Short fault info ############
2025.10.13 13:58:28.888467 [ 12 ] {} <Fatal> BaseDaemon: (version 25.10.1.2375, build id: 4EC2FB5B04495E61A0E21429C27C616BE8D05E65, git hash: e5a6e2fde05a3493dd61b9d165c5b91077091117, architecture: x86_64) (from thread 680) Received signal 6
2025.10.13 13:58:28.888494 [ 12 ] {} <Fatal> BaseDaemon: Signal description: Aborted
2025.10.13 13:58:28.888510 [ 12 ] {} <Fatal> BaseDaemon:
2025.10.13 13:58:28.888582 [ 12 ] {} <Fatal> BaseDaemon: Stack trace: 0x00007f5c759a89fd 0x00007f5c75954476 0x00007f5c7593a7f3 0x00005582d2a09abf 0x00005582d2a0ab90 0x00005582e2d29ae7 0x00005582e2cfbf2b 0x00005582e2d35ee3 0x00005582d2d272ae 0x00005582d2d35e50 0x00005582bfa8ce67 0x00007f5c759a6ac3 0x00007f5c75a38850
2025.10.13 13:58:28.888599 [ 12 ] {} <Fatal> BaseDaemon: ########################################
2025.10.13 13:58:28.888674 [ 12 ] {} <Fatal> BaseDaemon: (version 25.10.1.2375, build id: 4EC2FB5B04495E61A0E21429C27C616BE8D05E65, git hash: e5a6e2fde05a3493dd61b9d165c5b91077091117) (from thread 680) (no query) Received signal Aborted (6)
2025.10.13 13:58:28.888688 [ 12 ] {} <Fatal> BaseDaemon:
2025.10.13 13:58:28.888698 [ 12 ] {} <Fatal> BaseDaemon: Stack trace: 0x00007f5c759a89fd 0x00007f5c75954476 0x00007f5c7593a7f3 0x00005582d2a09abf 0x00005582d2a0ab90 0x00005582e2d29ae7 0x00005582e2cfbf2b 0x00005582e2d35ee3 0x00005582d2d272ae 0x00005582d2d35e50 0x00005582bfa8ce67 0x00007f5c759a6ac3 0x00007f5c75a38850
2025.10.13 13:58:28.888749 [ 12 ] {} <Fatal> BaseDaemon: 3. ? @ 0x00000000000969fd
2025.10.13 13:58:28.888769 [ 12 ] {} <Fatal> BaseDaemon: 4. ? @ 0x0000000000042476
2025.10.13 13:58:28.888786 [ 12 ] {} <Fatal> BaseDaemon: 5. ? @ 0x00000000000287f3
2025.10.13 13:58:28.955420 [ 12 ] {} <Fatal> BaseDaemon: 6. ./ci/tmp/build/./src/Common/Exception.cpp:52: DB::abortOnFailedAssertion(String const&, void* const*, unsigned long, unsigned long) @ 0x0000000020ea8abf
2025.10.13 13:58:29.016653 [ 12 ] {} <Fatal> BaseDaemon: 7. ./ci/tmp/build/./src/Common/Exception.cpp:58: ? @ 0x0000000020ea9b90
2025.10.13 13:58:29.206146 [ 12 ] {} <Fatal> BaseDaemon: 8. ./ci/tmp/build/./src/Interpreters/DDLWorker.cpp:1155: DB::DDLWorker::initializeMainThread() @ 0x00000000311c8ae7
2025.10.13 13:58:29.331832 [ 12 ] {} <Fatal> BaseDaemon: 9. ./ci/tmp/build/./src/Interpreters/DDLWorker.cpp:1212: DB::DDLWorker::runMainThread() @ 0x000000003119af2b
2025.10.13 13:58:29.448825 [ 12 ] {} <Fatal> BaseDaemon: 10.0. inlined from ./contrib/llvm-project/libcxx/include/__type_traits/invoke.h:117: decltype(*std::declval<DB::DDLWorker*&>().*std::declval<void (DB::DDLWorker::*&)()>()()) std::__invoke[abi:ne190107]<void (DB::DDLWorker::*&)(), DB::DDLWorker*&, void>(void (DB::DDLWorker::*&)(), DB::DDLWorker*&)
2025.10.13 13:58:29.448872 [ 12 ] {} <Fatal> BaseDaemon: 10.1. inlined from ./contrib/llvm-project/libcxx/include/tuple:1354: decltype(auto) std::__apply_tuple_impl[abi:ne190107]<void (DB::DDLWorker::*&)(), std::tuple<DB::DDLWorker*>&, 0ul>(void (DB::DDLWorker::*&)(), std::tuple<DB::DDLWorker*>&, std::__tuple_indices<0ul>)
2025.10.13 13:58:29.448890 [ 12 ] {} <Fatal> BaseDaemon: 10.2. inlined from ./contrib/llvm-project/libcxx/include/tuple:1358: decltype(auto) std::apply[abi:ne190107]<void (DB::DDLWorker::*&)(), std::tuple<DB::DDLWorker*>&>(void (DB::DDLWorker::*&)(), std::tuple<DB::DDLWorker*>&)
2025.10.13 13:58:29.448907 [ 12 ] {} <Fatal> BaseDaemon: 10. ./src/Common/ThreadPool.h:312: ThreadFromGlobalPoolImpl<true, true>::ThreadFromGlobalPoolImpl<void (DB::DDLWorker::*)(), DB::DDLWorker*>(void (DB::DDLWorker::*&&)(), DB::DDLWorker*&&)::'lambda'()::operator()() @ 0x00000000311d4ee3
2025.10.13 13:58:29.492931 [ 12 ] {} <Fatal> BaseDaemon: 11.0. inlined from ./contrib/llvm-project/libcxx/include/__functional/function.h:716: ?
2025.10.13 13:58:29.492954 [ 12 ] {} <Fatal> BaseDaemon: 11.1. inlined from ./contrib/llvm-project/libcxx/include/__functional/function.h:989: ?
2025.10.13 13:58:29.492988 [ 12 ] {} <Fatal> BaseDaemon: 11. ./ci/tmp/build/./src/Common/ThreadPool.cpp:812: ThreadPoolImpl<std::thread>::ThreadFromThreadPool::worker() @ 0x00000000211c62ae
2025.10.13 13:58:29.586785 [ 12 ] {} <Fatal> BaseDaemon: 12.0. inlined from ./contrib/llvm-project/libcxx/include/__type_traits/invoke.h:117: decltype(*std::declval<ThreadPoolImpl<std::thread>::ThreadFromThreadPool*>().*std::declval<void (ThreadPoolImpl<std::thread>::ThreadFromThreadPool::*)()>()()) std::__invoke[abi:ne190107]<void (ThreadPoolImpl<std::thread>::ThreadFromThreadPool::*)(), ThreadPoolImpl<std::thread>::ThreadFromThreadPool*, void>(void (ThreadPoolImpl<std::thread>::ThreadFromThreadPool::*&&)(), ThreadPoolImpl<std::thread>::ThreadFromThreadPool*&&)
2025.10.13 13:58:29.586835 [ 12 ] {} <Fatal> BaseDaemon: 12.1. inlined from ./contrib/llvm-project/libcxx/include/__thread/thread.h:192: void std::__thread_execute[abi:ne190107]<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void (ThreadPoolImpl<std::thread>::ThreadFromThreadPool::*)(), ThreadPoolImpl<std::thread>::ThreadFromThreadPool*, 2ul>(std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void (ThreadPoolImpl<std::thread>::ThreadFromThreadPool::*)(), ThreadPoolImpl<std::thread>::ThreadFromThreadPool*>&, std::__tuple_indices<2ul>)
2025.10.13 13:58:29.586860 [ 12 ] {} <Fatal> BaseDaemon: 12. ./contrib/llvm-project/libcxx/include/__thread/thread.h:201: void* std::__thread_proxy[abi:ne190107]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void (ThreadPoolImpl<std::thread>::ThreadFromThreadPool::*)(), ThreadPoolImpl<std::thread>::ThreadFromThreadPool*>>(void*) @ 0x00000000211d4e50
2025.10.13 13:58:29.676213 [ 12 ] {} <Fatal> BaseDaemon: 13. asan_thread_start(void*) @ 0x000000000df2be67
2025.10.13 13:58:29.676259 [ 12 ] {} <Fatal> BaseDaemon: 14. ? @ 0x0000000000094ac3
2025.10.13 13:58:29.676267 [ 12 ] {} <Fatal> BaseDaemon: 15. ? @ 0x0000000000126850
2025.10.13 13:58:29.676278 [ 12 ] {} <Fatal> BaseDaemon: Integrity check of the executable skipped because the reference checksum could not be read.

which (if I'm not mistaken) should be fixed by #88472

mode Similarly as with IAggregateFunction.h in [1], this should be done in AggregateFunctionSum.h as well [1]: ClickHouse#88440

Fix sparse columns aggregation for any overflow mode

24.8.14 Backport of ClickHouse#88440 - Fix sparse columns aggregation for any overflow mode

Fix sparse columns aggregation for any overflow mode

fix sparse columns aggregation for any overflow mode

6ab65e0

Similar to regular `add` method, there should be a check that `place` value is not null.

antaljanosbenjamin added the can be tested Allows running workflows for external contributors label Oct 13, 2025

antaljanosbenjamin self-assigned this Oct 13, 2025

clickhouse-gh bot added the pr-bugfix Pull request with bugfix, not backported by default label Oct 13, 2025

antaljanosbenjamin approved these changes Oct 13, 2025

View reviewed changes

antaljanosbenjamin added this pull request to the merge queue Oct 15, 2025

Merged via the queue into ClickHouse:master with commit a5391e9 Oct 15, 2025
121 of 123 checks passed

robot-ch-test-poll2 added the pr-synced-to-cloud The PR is synced to the cloud repo label Oct 15, 2025

mkmkme mentioned this pull request Jan 27, 2026

Fix sparse column aggregation with sum() and timeseries #95301

Merged

mkmkme pushed a commit to Altinity/ClickHouse that referenced this pull request Feb 10, 2026

Merge pull request ClickHouse#88440 from korowa/fix-gby-overflow-sparse

3b86faf

Fix sparse columns aggregation for any overflow mode

mkmkme mentioned this pull request Feb 10, 2026

24.8.14 Backport of #88440 - Fix sparse columns aggregation for any overflow mode Altinity/ClickHouse#1389

Merged

25 tasks

mkmkme pushed a commit to Altinity/ClickHouse that referenced this pull request Feb 12, 2026

Merge pull request ClickHouse#88440 from korowa/fix-gby-overflow-sparse

3d2606c

Fix sparse columns aggregation for any overflow mode

mkmkme mentioned this pull request Feb 12, 2026

25.8.16 Backports of #88440, #95301 - Fix sparse columns aggregation for any overflow mode Altinity/ClickHouse#1396

Open

25 tasks

zvonand added a commit to Altinity/ClickHouse that referenced this pull request Feb 25, 2026

Merge pull request #1389 from Altinity/backports/24.8.14/88440

26ad35c

24.8.14 Backport of ClickHouse#88440 - Fix sparse columns aggregation for any overflow mode

zvonand pushed a commit to Altinity/ClickHouse that referenced this pull request Mar 17, 2026

Merge pull request ClickHouse#88440 from korowa/fix-gby-overflow-sparse

375ac54

Fix sparse columns aggregation for any overflow mode

zvonand mentioned this pull request Mar 17, 2026

24.8.14 Stable Pre-Release PR Altinity/ClickHouse#1540

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix sparse columns aggregation for any overflow mode#88440

Fix sparse columns aggregation for any overflow mode#88440
antaljanosbenjamin merged 1 commit intoClickHouse:masterfrom
korowa:fix-gby-overflow-sparse

korowa commented Oct 13, 2025

Uh oh!

clickhouse-gh bot commented Oct 13, 2025 •

edited

Loading

Uh oh!

antaljanosbenjamin left a comment

Uh oh!

korowa commented Oct 14, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

korowa commented Oct 13, 2025

Changelog category (leave one):

Changelog entry (a user-readable short description of the changes that goes into CHANGELOG.md):

Documentation entry for user-facing changes

Uh oh!

clickhouse-gh bot commented Oct 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

antaljanosbenjamin left a comment

Choose a reason for hiding this comment

Uh oh!

korowa commented Oct 14, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

clickhouse-gh bot commented Oct 13, 2025 •

edited

Loading