Skip to content

[feat] Add batch rerun and stop for process instance#10013

Merged
caishunfeng merged 6 commits intoapache:devfrom
zhongjiajie:f-pi-batch-rerun-del-stop
May 16, 2022
Merged

[feat] Add batch rerun and stop for process instance#10013
caishunfeng merged 6 commits intoapache:devfrom
zhongjiajie:f-pi-batch-rerun-del-stop

Conversation

@zhongjiajie
Copy link
Copy Markdown
Member

@zhongjiajie zhongjiajie commented May 12, 2022

expand entry point executors/execute and make it support batch operator

@zhongjiajie zhongjiajie added enhancement New feature or request backend labels May 12, 2022
@zhongjiajie zhongjiajie self-assigned this May 12, 2022
@zhongjiajie zhongjiajie marked this pull request as draft May 12, 2022 06:48
@codecov-commenter
Copy link
Copy Markdown

codecov-commenter commented May 12, 2022

Codecov Report

Merging #10013 (6c19691) into dev (bc1c15b) will decrease coverage by 0.04%.
The diff coverage is 15.00%.

@@             Coverage Diff              @@
##                dev   #10013      +/-   ##
============================================
- Coverage     40.81%   40.77%   -0.05%     
  Complexity     4604     4604              
============================================
  Files           834      834              
  Lines         33832    33875      +43     
  Branches       3745     3756      +11     
============================================
+ Hits          13810    13811       +1     
- Misses        18701    18742      +41     
- Partials       1321     1322       +1
Impacted Files Coverage Δ
...inscheduler/api/controller/ExecutorController.java 28.30% <5.55%> (-11.70%) ⬇️
...uler/api/controller/ProcessInstanceController.java 72.09% <100.00%> (ø)
.../org/apache/dolphinscheduler/api/enums/Status.java 100.00% <100.00%> (ø)
...e/dolphinscheduler/remote/NettyRemotingClient.java 52.81% <0.00%> (-0.71%) ⬇️
...er/server/master/runner/WorkflowExecuteThread.java 8.04% <0.00%> (-0.21%) ⬇️
...nscheduler/service/process/ProcessServiceImpl.java 30.85% <0.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update bc1c15b...6c19691. Read the comment docs.

@sonarqubecloud
Copy link
Copy Markdown

sonarqubecloud bot commented May 12, 2022

SonarCloud Quality Gate failed.    Quality Gate failed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 2 Code Smells

10.7% 10.7% Coverage
0.0% 0.0% Duplication

@zhongjiajie zhongjiajie marked this pull request as ready for review May 13, 2022 01:47
caishunfeng
caishunfeng approved these changes May 16, 2022
View reviewed changes
Copy link
Copy Markdown
Contributor

@caishunfeng caishunfeng left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Tianqi-Dotes
Tianqi-Dotes approved these changes May 16, 2022
View reviewed changes
Copy link
Copy Markdown
Member

@Tianqi-Dotes Tianqi-Dotes left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@caishunfeng caishunfeng merged commit 3e471f8 into apache:dev May 16, 2022
@zhongjiajie zhongjiajie deleted the f-pi-batch-rerun-del-stop branch May 16, 2022 03:36
rockfang pushed a commit to rockfang/dolphinscheduler that referenced this pull request May 18, 2022
@zhongjiajie @rockfang
[feat] Add batch rerun and stop for process instance (apache#10013)
9137f1a 
* [feat] Add batch rerun and stop for process instance

* fix execute type pass

* error message changed

* correct error message

* add note to i18n ignore

* enhance the note of process instance ids
vagetablechicken added a commit to vagetablechicken/dolphinscheduler that referenced this pull request May 21, 2022
@vagetablechicken
Squashed commit of the following:
7b36f80 
commit 8046180
Author: 王心雨 <wangxinyu@4paradigm.com>
Date:   Thu May 19 12:34:07 2022 +0800

    去掉错误提交的代码

commit 5e9e9a9
Author: 王心雨 <wangxinyu@4paradigm.com>
Date:   Thu May 19 12:23:44 2022 +0800

    加入openmldb task

commit b1885c7
Author: caishunfeng <caishunfeng2021@gmail.com>
Date:   Mon May 16 15:41:45 2022 +0800

    [Bug] fix run on docker and k8s (apache#10026)

    * fix docker-compose init schema

    * recovery depend on zk

    * update doc and dockerfile

    * fix run on k8s

    * udpate doc

    * add DOCKER flag & update doc

    * remove repeat DOCKER env

commit d643e1c
Author: Dannila <94423827+Dannila@users.noreply.github.com>
Date:   Mon May 16 15:06:21 2022 +0800

    [Fix-10039] Flink run command when perfecting Python jobs (apache#10042)

    * [fix] flink task

    * [fix] flink task

commit 3e471f8
Author: Jiajie Zhong <zhongjiajie955@hotmail.com>
Date:   Mon May 16 11:01:32 2022 +0800

    [feat] Add batch rerun and stop for process instance (apache#10013)

    * [feat] Add batch rerun and stop for process instance

    * fix execute type pass

    * error message changed

    * correct error message

    * add note to i18n ignore

    * enhance the note of process instance ids

commit f666c64
Author: chuxing <923622908@qq.com>
Date:   Mon May 16 09:53:45 2022 +0800

    [doc] Correct docs of development-environment-setup (apache#9995)

commit 359cbe2
Author: zixi0825 <sunzhaohe0825@gmail.com>
Date:   Sun May 15 17:46:31 2022 +0800

    [dataquality] Fix task commnd null bug (apache#9974)

commit 2423fa5
Author: xiangzihao <460888207@qq.com>
Date:   Sun May 15 13:04:48 2022 +0800

    [Feature-10034][CI] Add postgresql cluster test in CI (apache#10035)

    * add postgresql cluster test

    * add postgresql cluster test

    * add postgresql cluster test

    * add postgresql cluster test

commit df04c4a
Author: chuxing <923622908@qq.com>
Date:   Sun May 15 10:24:31 2022 +0800

    [fix-9991][worker]fix statement is closed before resultSet.getMetaData() (apache#10014)

commit baf654c
Author: xiangzihao <460888207@qq.com>
Date:   Sat May 14 12:30:57 2022 +0800

    [Feature-9474] [CI] Add cluster test script verify on shell script (apache#9997)

    * cluster test

    * fix init db failed

    * fix init db failed

    * fix init db failed

    * fix init db failed

    * fix init db failed

    * add github actions

    * add github actions

    * add github actions

    * add github actions

    * add github actions

    * add github actions

    * add sudo

    * add github actions

    * add github actions

    * add github actions

    * add github actions

    * add github actions

    * add github actions

    * add github actions

    * add github actions

    * add github actions

    * add github actions

    * add github actions

    * add github actions

    * add github actions

    * add github actions

    * add github actions

    * remove cluster-test workflows

    * add github actions

    * add github actions

    * refactor test to docker mode

    * refactor test to docker mode

    * refactor test to docker mode

    * refactor test to docker mode

    * remove create schema logic

    * remove create schema logic

    * remove create schema logic

    * fix runnint cluster test error

    * fix runnint cluster test error

    * fix runnint cluster test error

    * fix runnint cluster test error

    * fix runnint cluster test error

    * fix runnint cluster test error

    * fix runnint cluster test error

    * fix runnint cluster test error

    * fix runnint cluster test error

    * fix runnint cluster test error

    * fix runnint cluster test error

    * fix runnint cluster test error

    * fix runnint cluster test error

    * fix runnint cluster test error

    * fix runnint cluster test error

    * fix runnint cluster test error

    * fix runnint cluster test error

    * fix runnint cluster test error

    * fix runnint cluster test error

    * fix runnint cluster test error

    * fix runnint cluster test error

    * fix runnint cluster test error

    * fix runnint cluster test error

    * fix runnint cluster test error

    * fix runnint cluster test error

    * fix runnint cluster test error

    * add github actions

    * add github actions

    * add github actions

    * add cluster test timeout

    * add cluster start test loop check

    * add cluster start test loop check

    * optimize cluster start test loop check

commit ee2c516
Author: 旺阳 <wang@lqwang.net>
Date:   Fri May 13 10:39:05 2022 +0800

    [doc] Correct kubernetes (apache#9985)

    Co-authored-by: qingwli <qingwli@cisco.com>

commit d17379d
Author: rockfang <657297417@qq.com>
Date:   Fri May 13 10:16:54 2022 +0800

    [Fix][UI] Fix errorOutputPath column in dataquality page (apache#10015)

commit 8036936
Author: Eric Gao <ericgao.apache@gmail.com>
Date:   Thu May 12 19:55:49 2022 +0800

    [feat][task plugin] Add Jupyter task plugin (apache#9872)

    Co-authored-by: Amy0104 <97265214+Amy0104@users.noreply.github.com>

commit 0fe7548
Author: QuakeWang <45645138+QuakeWang@users.noreply.github.com>
Date:   Thu May 12 18:23:43 2022 +0800

    [doc] Add example and notice about task type Dependent (apache#10001)

    Co-authored-by: Jiajie Zhong <zhongjiajie955@gmail.com>

commit 18bfe63
Author: Amy0104 <97265214+Amy0104@users.noreply.github.com>
Date:   Thu May 12 16:54:03 2022 +0800

    [Fix][UI] Support only one file upload on the file manage page. (apache#10007)

commit dbdbfea
Author: WangJPLeo <103574007+WangJPLeo@users.noreply.github.com>
Date:   Thu May 12 16:31:53 2022 +0800

    [Fix-9975] The selected task instance was recreated when the Master service fail… (apache#9976)

    * The selected task instance was recreated when the Master service failed over.

    * Returns the expression result directly.

    * Use Recovery to determine whether to use the old task instance.

commit 00f1029
Author: Amy0104 <97265214+Amy0104@users.noreply.github.com>
Date:   Thu May 12 15:27:55 2022 +0800

    [Fix][UI] Fix the task name validator error. (apache#10008)

commit bce5a28
Author: Kerwin <37063904+zhuangchong@users.noreply.github.com>
Date:   Thu May 12 14:42:39 2022 +0800

    [doc] Add the description about execute type in SQL task (apache#9987)

commit 53cd700
Author: rockfang <657297417@qq.com>
Date:   Thu May 12 14:11:52 2022 +0800

    [Fix-10002] Fix some bugs in datasource list (apache#10005)

    * fix: fix ellipsis bug in table column

    * fix ellipsis bug in table column

    * fix: json-highlight component support nested jsonstring

    * fix: make datasource description show

commit 8d17fd2
Author: Jiajie Zhong <zhongjiajie955@hotmail.com>
Date:   Thu May 12 11:30:01 2022 +0800

    [ci] Dead link check all markdown file (apache#10004)

    there are 162 markdown files in dir `docs`
    and all file in project is 175 files, so I
    think check all file will not add too much
    load for our ci, but it could discovered
    the dead link in time to avoid some thing
    like apache#9998

commit 99b1c40
Author: songjianet <1778651752@qq.com>
Date:   Wed May 11 22:15:36 2022 +0800

    [Docs] Fix docker link. (apache#9998)

commit d4aeee1
Author: Tq <tianqitobethefirst@gmail.com>
Date:   Wed May 11 18:37:03 2022 +0800

    [Bug] [MASTER-9811]fix cmd param to overwrite global param when executing complement (apache#9952)

    * fix cmd param to overwrite global param when executing complement

    * fix cmd param to overwrite global param when executing complement

commit bc1c15b
Author: qianli2022 <97489722+qianli2022@users.noreply.github.com>
Date:   Wed May 11 16:40:30 2022 +0800

    [Feature-8252][doc] K8s and namespace manager docs and web page update (apache#9881)

    * test

    * test

    * doc

    * fix image error

    * fix

    * Update docs/docs/en/guide/security.md

    * Update docs/docs/en/guide/security.md

    * Update docs/docs/en/guide/security.md

    * Update docs/docs/zh/guide/security.md

    * fix doc

    Co-authored-by: qianl4 <qianl4@cicso.com>
    Co-authored-by: William Tong <weitong@cisco.com>
    Co-authored-by: Jiajie Zhong <zhongjiajie955@gmail.com>
Tianqi-Dotes pushed a commit to Tianqi-Dotes/dolphinscheduler that referenced this pull request Jun 16, 2022
@zhongjiajie
[feat] Add batch rerun and stop for process instance (apache#10013)
d2f472a 
* [feat] Add batch rerun and stop for process instance

* fix execute type pass

* error message changed

* correct error message

* add note to i18n ignore

* enhance the note of process instance ids

(cherry picked from commit 3e471f8)
@SbloodyS SbloodyS added this to the 3.1.0 milestone Aug 31, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@caishunfeng caishunfeng caishunfeng approved these changes

@Tianqi-Dotes Tianqi-Dotes Tianqi-Dotes approved these changes

Assignees

@zhongjiajie zhongjiajie

Labels

backend enhancement New feature or request

Projects

None yet

Milestone

3.1.0

Development

Successfully merging this pull request may close these issues.

5 participants

@zhongjiajie @codecov-commenter @caishunfeng @Tianqi-Dotes @SbloodyS