Skip to content

A simple GROUP BY query failed with distributed_aggregation_memory_efficient enabled and group by key items greater than 65535 #76934

@ucasfl

Description

@ucasfl

Company or project name

No response

Describe the unexpected behaviour

OK:

SELECT
    cityHash64(item_id) % 65535 AS itemid,
    sum(1)
FROM ice_wxg_euler.dws_finderlive_uin_ecom_pcoc_detail_hi AS a
WHERE (a.day_ = toDate('2025-02-27')) AND (a.hour_ = toDateTime('2025-02-27 16:00:00')) AND (trimBoth(commentScene) = 'temp_2') AND ((stayTime > 5000) OR (duration > 5000)) AND ((stayTime + duration) > 0) AND (rankscore_str != '')
GROUP BY itemid
SETTINGS distributed_aggregation_memory_efficient = 1
FORMAT `Null`

Query id: a372983e-0261-437e-85ce-231769087ee2

Ok.

0 rows in set. Elapsed: 20.824 sec. Processed 388.99 million rows, 54.84 GB (18.68 million rows/s., 2.63 GB/s.)

Not OK:

SELECT
    cityHash64(item_id) % 65536 AS itemid,
    sum(1)
FROM ice_wxg_euler.dws_finderlive_uin_ecom_pcoc_detail_hi AS a
WHERE (a.day_ = toDate('2025-02-27')) AND (a.hour_ = toDateTime('2025-02-27 16:00:00')) AND (trimBoth(commentScene) = 'temp_2') AND ((stayTime > 5000) OR (duration > 5000)) AND ((stayTime + duration) > 0) AND (rankscore_str != '')
GROUP BY itemid
SETTINGS distributed_aggregation_memory_efficient = 1
FORMAT `Null`

Query id: 53eca855-f4ff-4f2e-a4dd-b6e814b00028

↓ Progress: 388.99 million rows, 54.84 GB (17.49 million rows/s., 2.46 GB/s.) 
0 rows in set. Elapsed: 22.247 sec. Processed 388.99 million rows, 54.84 GB (17.49 million rows/s., 2.46 GB/s.)

Received exception from server (version 23.8.5):
Code: 49. DB::Exception: Received from mmdcchsvrnewtestsz1:28328. DB::Exception: SortingAggregatedTransform already got bucket with number 217. (LOGICAL_ERROR)

How to reproduce

None

Expected behavior

No response

Error message and/or stacktrace

No response

Additional context

No response

Metadata

Metadata

Assignees

Labels

bugConfirmed user-visible misbehaviour in official releaseclose in a month if not activeThis will be closed in case of no information

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions