Skip to content

Conversation

@morningman
Copy link
Contributor

Internal commit 95787f8be1fd0ff215708fb0f49997b632876586

@morningman morningman merged commit 5de798f into apache:master Mar 23, 2018
BiteTheDDDDt pushed a commit to BiteTheDDDDt/incubator-doris that referenced this pull request Nov 16, 2021
starocean999 pushed a commit to starocean999/incubator-doris that referenced this pull request Jul 26, 2023
# Checklist

* 内部Jira:
* 修复PR:
  * master:
  * 1.2-lts:
  * 1.2-lts: 
* Doris-master:
  * [ ] 已修复
  * [ ] 未修复
  * [ ] 无问题
* Doris-1.2-lts:
  * [ ] 已修复
  * [ ] 未修复
  * [ ] 无问题
* Doris-1.1-lts:
  * [ ] 已修复
  * [ ] 未修复
  * [ ] 无问题
yiguolei pushed a commit that referenced this pull request Sep 28, 2023
Current multi-window plan generation has problem on the project sequence, for example:

+--LogicalWindow ( windowExpressions=[avg(sum_sales#115) WindowSpec(...) AS `avg_monthly_sales`#116, rank() WindowSpec(...) AS `rn`#117], ...)
and correspond physical plan is:

+--PhysicalWindow[6572]@16 ( windowFrameGroup=(Funcs=[avg(sum_sales#115) WindowSpec(...) AS `avg_monthly_sales`#116], ... )
    +--PhysicalWindow[6568]@29 ( windowFrameGroup=(Funcs=[rank() WindowSpec(...) AS `rn`#117], ...] )
If the final plan is generated as following:

MultiCastDataSinks
STREAM DATA SINK
  EXCHANGE ID: 20
  HASH_PARTITIONED: rn[#208], i_brand[#202], cc_name[#203], i_category[#201]
Before we eventually resolve the multi-window issue, we add a projection as following and force a mapping but this will not cover all potential problems.

MultiCastDataSinks
STREAM DATA SINK
  EXCHANGE ID: 20
  HASH_PARTITIONED: rn[#219], i_brand[#213], cc_name[#214], i_category[#212]
  PROJECTIONS: i_category[#184], i_brand[#185], cc_name[#186], d_year[#187], d_moy[#188], sum_sales[#189], avg_monthly_sales[#191], rn[#190]
  PROJECTION TUPLE: 20
vinlee19 pushed a commit to vinlee19/doris that referenced this pull request Oct 7, 2023
…24912)

Current multi-window plan generation has problem on the project sequence, for example:

+--LogicalWindow ( windowExpressions=[avg(sum_sales#115) WindowSpec(...) AS `avg_monthly_sales`apache#116, rank() WindowSpec(...) AS `rn`apache#117], ...)
and correspond physical plan is:

+--PhysicalWindow[6572]@16 ( windowFrameGroup=(Funcs=[avg(sum_sales#115) WindowSpec(...) AS `avg_monthly_sales`apache#116], ... )
    +--PhysicalWindow[6568]@29 ( windowFrameGroup=(Funcs=[rank() WindowSpec(...) AS `rn`apache#117], ...] )
If the final plan is generated as following:

MultiCastDataSinks
STREAM DATA SINK
  EXCHANGE ID: 20
  HASH_PARTITIONED: rn[apache#208], i_brand[apache#202], cc_name[apache#203], i_category[apache#201]
Before we eventually resolve the multi-window issue, we add a projection as following and force a mapping but this will not cover all potential problems.

MultiCastDataSinks
STREAM DATA SINK
  EXCHANGE ID: 20
  HASH_PARTITIONED: rn[apache#219], i_brand[apache#213], cc_name[apache#214], i_category[apache#212]
  PROJECTIONS: i_category[apache#184], i_brand[apache#185], cc_name[apache#186], d_year[apache#187], d_moy[apache#188], sum_sales[apache#189], avg_monthly_sales[apache#191], rn[apache#190]
  PROJECTION TUPLE: 20
bobhan1 added a commit to bobhan1/doris that referenced this pull request Oct 28, 2024
gavinchou pushed a commit that referenced this pull request Nov 6, 2025
Currently, tablet report logic uses a ForkJoinPool to process tablet
information, but often encounters unexplained hangs in the ForkJoinPool.
The printed stack trace doesn't reveal where the hang occurs, making it
difficult to troubleshoot the issue.

use forkjoin pool, report stuck stack such as 
```
"report-thread" #187 daemon prio=5 os_prio=0 cpu=97864.95ms elapsed=2469428.50s tid=0x00007ff1cb5c8530 nid=0xef2 waiting on condition  [0x00007fef462e1000]
   java.lang.Thread.State: WAITING (parking)
        at jdk.internal.misc.Unsafe.park(java.base@17.0.15/Native Method)
        - parking to wait for  <0x00000006dc3fae00> (a java.util.concurrent.ForkJoinTask$AdaptedRunnableAction)
        at java.util.concurrent.locks.LockSupport.park(java.base@17.0.15/LockSupport.java:341)
        at java.util.concurrent.ForkJoinTask.awaitDone(java.base@17.0.15/ForkJoinTask.java:468)
        at java.util.concurrent.ForkJoinTask.join(java.base@17.0.15/ForkJoinTask.java:670)
        at org.apache.doris.catalog.TabletInvertedIndex.tabletReport(TabletInvertedIndex.java:370)
        at org.apache.doris.master.ReportHandler.tabletReport(ReportHandler.java:509)
        at org.apache.doris.master.ReportHandler$ReportTask.exec(ReportHandler.java:339)
        at org.apache.doris.master.ReportHandler.runOneCycle(ReportHandler.java:1466)
        at org.apache.doris.common.util.Daemon.run(Daemon.java:119)

   Locked ownable synchronizers:
        - None
```
Can't find where the problem is in this stack

When the tablet report is stuck, the TabletInvertedIndex holds a read
lock, leading to a deadlock.

This pr uses a normal thread pool to replace forkjoinpool
github-actions bot pushed a commit that referenced this pull request Nov 11, 2025
Currently, tablet report logic uses a ForkJoinPool to process tablet
information, but often encounters unexplained hangs in the ForkJoinPool.
The printed stack trace doesn't reveal where the hang occurs, making it
difficult to troubleshoot the issue.

use forkjoin pool, report stuck stack such as 
```
"report-thread" #187 daemon prio=5 os_prio=0 cpu=97864.95ms elapsed=2469428.50s tid=0x00007ff1cb5c8530 nid=0xef2 waiting on condition  [0x00007fef462e1000]
   java.lang.Thread.State: WAITING (parking)
        at jdk.internal.misc.Unsafe.park(java.base@17.0.15/Native Method)
        - parking to wait for  <0x00000006dc3fae00> (a java.util.concurrent.ForkJoinTask$AdaptedRunnableAction)
        at java.util.concurrent.locks.LockSupport.park(java.base@17.0.15/LockSupport.java:341)
        at java.util.concurrent.ForkJoinTask.awaitDone(java.base@17.0.15/ForkJoinTask.java:468)
        at java.util.concurrent.ForkJoinTask.join(java.base@17.0.15/ForkJoinTask.java:670)
        at org.apache.doris.catalog.TabletInvertedIndex.tabletReport(TabletInvertedIndex.java:370)
        at org.apache.doris.master.ReportHandler.tabletReport(ReportHandler.java:509)
        at org.apache.doris.master.ReportHandler$ReportTask.exec(ReportHandler.java:339)
        at org.apache.doris.master.ReportHandler.runOneCycle(ReportHandler.java:1466)
        at org.apache.doris.common.util.Daemon.run(Daemon.java:119)

   Locked ownable synchronizers:
        - None
```
Can't find where the problem is in this stack

When the tablet report is stuck, the TabletInvertedIndex holds a read
lock, leading to a deadlock.

This pr uses a normal thread pool to replace forkjoinpool
github-actions bot pushed a commit that referenced this pull request Nov 11, 2025
Currently, tablet report logic uses a ForkJoinPool to process tablet
information, but often encounters unexplained hangs in the ForkJoinPool.
The printed stack trace doesn't reveal where the hang occurs, making it
difficult to troubleshoot the issue.

use forkjoin pool, report stuck stack such as 
```
"report-thread" #187 daemon prio=5 os_prio=0 cpu=97864.95ms elapsed=2469428.50s tid=0x00007ff1cb5c8530 nid=0xef2 waiting on condition  [0x00007fef462e1000]
   java.lang.Thread.State: WAITING (parking)
        at jdk.internal.misc.Unsafe.park(java.base@17.0.15/Native Method)
        - parking to wait for  <0x00000006dc3fae00> (a java.util.concurrent.ForkJoinTask$AdaptedRunnableAction)
        at java.util.concurrent.locks.LockSupport.park(java.base@17.0.15/LockSupport.java:341)
        at java.util.concurrent.ForkJoinTask.awaitDone(java.base@17.0.15/ForkJoinTask.java:468)
        at java.util.concurrent.ForkJoinTask.join(java.base@17.0.15/ForkJoinTask.java:670)
        at org.apache.doris.catalog.TabletInvertedIndex.tabletReport(TabletInvertedIndex.java:370)
        at org.apache.doris.master.ReportHandler.tabletReport(ReportHandler.java:509)
        at org.apache.doris.master.ReportHandler$ReportTask.exec(ReportHandler.java:339)
        at org.apache.doris.master.ReportHandler.runOneCycle(ReportHandler.java:1466)
        at org.apache.doris.common.util.Daemon.run(Daemon.java:119)

   Locked ownable synchronizers:
        - None
```
Can't find where the problem is in this stack

When the tablet report is stuck, the TabletInvertedIndex holds a read
lock, leading to a deadlock.

This pr uses a normal thread pool to replace forkjoinpool
yiguolei pushed a commit that referenced this pull request Nov 13, 2025
Currently, tablet report logic uses a ForkJoinPool to process tablet
information, but often encounters unexplained hangs in the ForkJoinPool.
The printed stack trace doesn't reveal where the hang occurs, making it
difficult to troubleshoot the issue.

use forkjoin pool, report stuck stack such as 
```
"report-thread" #187 daemon prio=5 os_prio=0 cpu=97864.95ms elapsed=2469428.50s tid=0x00007ff1cb5c8530 nid=0xef2 waiting on condition  [0x00007fef462e1000]
   java.lang.Thread.State: WAITING (parking)
        at jdk.internal.misc.Unsafe.park(java.base@17.0.15/Native Method)
        - parking to wait for  <0x00000006dc3fae00> (a java.util.concurrent.ForkJoinTask$AdaptedRunnableAction)
        at java.util.concurrent.locks.LockSupport.park(java.base@17.0.15/LockSupport.java:341)
        at java.util.concurrent.ForkJoinTask.awaitDone(java.base@17.0.15/ForkJoinTask.java:468)
        at java.util.concurrent.ForkJoinTask.join(java.base@17.0.15/ForkJoinTask.java:670)
        at org.apache.doris.catalog.TabletInvertedIndex.tabletReport(TabletInvertedIndex.java:370)
        at org.apache.doris.master.ReportHandler.tabletReport(ReportHandler.java:509)
        at org.apache.doris.master.ReportHandler$ReportTask.exec(ReportHandler.java:339)
        at org.apache.doris.master.ReportHandler.runOneCycle(ReportHandler.java:1466)
        at org.apache.doris.common.util.Daemon.run(Daemon.java:119)

   Locked ownable synchronizers:
        - None
```
Can't find where the problem is in this stack

When the tablet report is stuck, the TabletInvertedIndex holds a read
lock, leading to a deadlock.

This pr uses a normal thread pool to replace forkjoinpool
wyxxxcat pushed a commit to wyxxxcat/doris that referenced this pull request Nov 18, 2025
…57382)

Currently, tablet report logic uses a ForkJoinPool to process tablet
information, but often encounters unexplained hangs in the ForkJoinPool.
The printed stack trace doesn't reveal where the hang occurs, making it
difficult to troubleshoot the issue.

use forkjoin pool, report stuck stack such as 
```
"report-thread" apache#187 daemon prio=5 os_prio=0 cpu=97864.95ms elapsed=2469428.50s tid=0x00007ff1cb5c8530 nid=0xef2 waiting on condition  [0x00007fef462e1000]
   java.lang.Thread.State: WAITING (parking)
        at jdk.internal.misc.Unsafe.park(java.base@17.0.15/Native Method)
        - parking to wait for  <0x00000006dc3fae00> (a java.util.concurrent.ForkJoinTask$AdaptedRunnableAction)
        at java.util.concurrent.locks.LockSupport.park(java.base@17.0.15/LockSupport.java:341)
        at java.util.concurrent.ForkJoinTask.awaitDone(java.base@17.0.15/ForkJoinTask.java:468)
        at java.util.concurrent.ForkJoinTask.join(java.base@17.0.15/ForkJoinTask.java:670)
        at org.apache.doris.catalog.TabletInvertedIndex.tabletReport(TabletInvertedIndex.java:370)
        at org.apache.doris.master.ReportHandler.tabletReport(ReportHandler.java:509)
        at org.apache.doris.master.ReportHandler$ReportTask.exec(ReportHandler.java:339)
        at org.apache.doris.master.ReportHandler.runOneCycle(ReportHandler.java:1466)
        at org.apache.doris.common.util.Daemon.run(Daemon.java:119)

   Locked ownable synchronizers:
        - None
```
Can't find where the problem is in this stack

When the tablet report is stuck, the TabletInvertedIndex holds a read
lock, leading to a deadlock.

This pr uses a normal thread pool to replace forkjoinpool
deardeng added a commit to deardeng/incubator-doris that referenced this pull request Dec 4, 2025
…57382)

Currently, tablet report logic uses a ForkJoinPool to process tablet
information, but often encounters unexplained hangs in the ForkJoinPool.
The printed stack trace doesn't reveal where the hang occurs, making it
difficult to troubleshoot the issue.

use forkjoin pool, report stuck stack such as
```
"report-thread" apache#187 daemon prio=5 os_prio=0 cpu=97864.95ms elapsed=2469428.50s tid=0x00007ff1cb5c8530 nid=0xef2 waiting on condition  [0x00007fef462e1000]
   java.lang.Thread.State: WAITING (parking)
        at jdk.internal.misc.Unsafe.park(java.base@17.0.15/Native Method)
        - parking to wait for  <0x00000006dc3fae00> (a java.util.concurrent.ForkJoinTask$AdaptedRunnableAction)
        at java.util.concurrent.locks.LockSupport.park(java.base@17.0.15/LockSupport.java:341)
        at java.util.concurrent.ForkJoinTask.awaitDone(java.base@17.0.15/ForkJoinTask.java:468)
        at java.util.concurrent.ForkJoinTask.join(java.base@17.0.15/ForkJoinTask.java:670)
        at org.apache.doris.catalog.TabletInvertedIndex.tabletReport(TabletInvertedIndex.java:370)
        at org.apache.doris.master.ReportHandler.tabletReport(ReportHandler.java:509)
        at org.apache.doris.master.ReportHandler$ReportTask.exec(ReportHandler.java:339)
        at org.apache.doris.master.ReportHandler.runOneCycle(ReportHandler.java:1466)
        at org.apache.doris.common.util.Daemon.run(Daemon.java:119)

   Locked ownable synchronizers:
        - None
```
Can't find where the problem is in this stack

When the tablet report is stuck, the TabletInvertedIndex holds a read
lock, leading to a deadlock.

This pr uses a normal thread pool to replace forkjoinpool
hello-stephen pushed a commit that referenced this pull request Dec 4, 2025
…#58714)

Currently, tablet report logic uses a ForkJoinPool to process tablet
information, but often encounters unexplained hangs in the ForkJoinPool.
The printed stack trace doesn't reveal where the hang occurs, making it
difficult to troubleshoot the issue.

use forkjoin pool, report stuck stack such as
```
"report-thread" #187 daemon prio=5 os_prio=0 cpu=97864.95ms elapsed=2469428.50s tid=0x00007ff1cb5c8530 nid=0xef2 waiting on condition  [0x00007fef462e1000]
   java.lang.Thread.State: WAITING (parking)
        at jdk.internal.misc.Unsafe.park(java.base@17.0.15/Native Method)
        - parking to wait for  <0x00000006dc3fae00> (a java.util.concurrent.ForkJoinTask$AdaptedRunnableAction)
        at java.util.concurrent.locks.LockSupport.park(java.base@17.0.15/LockSupport.java:341)
        at java.util.concurrent.ForkJoinTask.awaitDone(java.base@17.0.15/ForkJoinTask.java:468)
        at java.util.concurrent.ForkJoinTask.join(java.base@17.0.15/ForkJoinTask.java:670)
        at org.apache.doris.catalog.TabletInvertedIndex.tabletReport(TabletInvertedIndex.java:370)
        at org.apache.doris.master.ReportHandler.tabletReport(ReportHandler.java:509)
        at org.apache.doris.master.ReportHandler$ReportTask.exec(ReportHandler.java:339)
        at org.apache.doris.master.ReportHandler.runOneCycle(ReportHandler.java:1466)
        at org.apache.doris.common.util.Daemon.run(Daemon.java:119)

   Locked ownable synchronizers:
        - None
```
Can't find where the problem is in this stack

When the tablet report is stuck, the TabletInvertedIndex holds a read
lock, leading to a deadlock.

This pr uses a normal thread pool to replace forkjoinpool

### What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

### Release note

None

### Check List (For Author)

- Test <!-- At least one of them must be included. -->
    - [ ] Regression test
    - [ ] Unit Test
    - [ ] Manual test (add detailed scripts or steps below)
    - [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
        - [ ] Previous test can cover this change.
        - [ ] No code files have been changed.
        - [ ] Other reason <!-- Add your reason?  -->

- Behavior changed:
    - [ ] No.
    - [ ] Yes. <!-- Explain the behavior change -->

- Does this need documentation?
    - [ ] No.
- [ ] Yes. <!-- Add document PR link here. eg:
apache/doris-website#1214 -->

### Check List (For Reviewer who merge this PR)

- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR
should merge into -->
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants