Skip to content

ParallelInputsProcessor does not work well with extreme data skew #4856

@SeaRise

Description

@SeaRise

Enhancement

https://github.com/pingcap/tiflash/blob/master/dbms/src/DataStreams/ParallelInputsProcessor.h#L287-L289
https://github.com/pingcap/tiflash/blob/master/dbms/src/DataStreams/ParallelInputsProcessor.h#L314-L315

If the number of threads is 10 and the number of streams is 10.
Only one stream A can be read 100 times and the other streams can only be read once.
In the end, only one thread will execute stream A, instead of 10 threads executing together.

Metadata

Metadata

Assignees

Labels

type/enhancementThe issue or PR belongs to an enhancement.

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions