-
Notifications
You must be signed in to change notification settings - Fork 3.7k
[fix](pipeline) Make all upstream tasks runnable if all tasks finishe… #41292
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
Thank you for your contribution to Apache Doris. Since 2024-03-18, the Document has been moved to doris-website. |
|
run buildall |
|
TeamCity be ut coverage result: |
|
run buildall |
|
TeamCity be ut coverage result: |
|
run buildall |
3 similar comments
|
run buildall |
|
run buildall |
|
run buildall |
|
TeamCity be ut coverage result: |
|
run buildall |
|
TeamCity be ut coverage result: |
|
run buildall |
1 similar comment
|
run buildall |
|
TeamCity be ut coverage result: |
c6c8fc5 to
71361c6
Compare
apache#41292) Consider 3 pipelines in this fragment (... -> join -> shuffle) : pipeline 0 : `... -> local exchange sink` pipeline 1 : `... -> join build (INNER JOIN)` pipeline 2 : `local exchange source -> join probe (INNER JOIN) -> data stream sender ` Assume the JoinBuild returned 0 rows, join probe can finish directly once join build finished and do not need to wait for the `local exchange sink` finished. In this case, if pipeline 0 is blocked by a dependency for a long time, pipeline 2 should notify pipeline 0 to finish.
apache#41292) Consider 3 pipelines in this fragment (... -> join -> shuffle) : pipeline 0 : `... -> local exchange sink` pipeline 1 : `... -> join build (INNER JOIN)` pipeline 2 : `local exchange source -> join probe (INNER JOIN) -> data stream sender ` Assume the JoinBuild returned 0 rows, join probe can finish directly once join build finished and do not need to wait for the `local exchange sink` finished. In this case, if pipeline 0 is blocked by a dependency for a long time, pipeline 2 should notify pipeline 0 to finish.
apache#41292) Consider 3 pipelines in this fragment (... -> join -> shuffle) : pipeline 0 : `... -> local exchange sink` pipeline 1 : `... -> join build (INNER JOIN)` pipeline 2 : `local exchange source -> join probe (INNER JOIN) -> data stream sender ` Assume the JoinBuild returned 0 rows, join probe can finish directly once join build finished and do not need to wait for the `local exchange sink` finished. In this case, if pipeline 0 is blocked by a dependency for a long time, pipeline 2 should notify pipeline 0 to finish.
…ly (apache#41667) send ignored rf when hash join build closed early to avoid runtime filter sync/merge error Follow-up : apache#41292
…pache#41751) Sometimes eos is true, the finish dependency is still not ready. Therefore, we need to use wake_up_by_downstream to determine whether it was closed early. Otherwise, it will enter the normal rf build process in this case and generate related errors. Follow-up : apache#41292
…pache#41751) ## Proposed changes Sometimes eos is true, the finish dependency is still not ready. Therefore, we need to use wake_up_by_downstream to determine whether it was closed early. Otherwise, it will enter the normal rf build process in this case and generate related errors. Follow-up : apache#41292
apache#41292) Consider 3 pipelines in this fragment (... -> join -> shuffle) : pipeline 0 : `... -> local exchange sink` pipeline 1 : `... -> join build (INNER JOIN)` pipeline 2 : `local exchange source -> join probe (INNER JOIN) -> data stream sender ` Assume the JoinBuild returned 0 rows, join probe can finish directly once join build finished and do not need to wait for the `local exchange sink` finished. In this case, if pipeline 0 is blocked by a dependency for a long time, pipeline 2 should notify pipeline 0 to finish.
…ly (apache#41667) ## Proposed changes send ignored rf when hash join build closed early to avoid runtime filter sync/merge error Follow-up : apache#41292
…pache#41751) ## Proposed changes Sometimes eos is true, the finish dependency is still not ready. Therefore, we need to use wake_up_by_downstream to determine whether it was closed early. Otherwise, it will enter the normal rf build process in this case and generate related errors. Follow-up : apache#41292
…d in this pipeline
Proposed changes
Consider 3 pipelines in this fragment (... -> join -> shuffle) :
pipeline 0 :
... -> local exchange sinkpipeline 1 :
... -> join build (INNER JOIN)pipeline 2 :
local exchange source -> join probe (INNER JOIN) -> data stream senderAssume the JoinBuild returned 0 rows, join probe can finish directly once join build finished and do not need to wait for the
local exchange sinkfinished. In this case, if pipeline 0 is blocked by a dependency for a long time, pipeline 2 should notify pipeline 0 to finish.