[Feature][plugin] Add k8s task in task plugin by zhaohehuhu · Pull Request #9425 · apache/dolphinscheduler

zhaohehuhu · 2022-04-11T02:34:44Z

Purpose of the pull request

Introduce k8s task and support running k8s task on kubernetes

close #9310

Brief change log

Add k8s task module in task plugin

Verify this pull request

This pull request is code cleanup without any test coverage.

(or)

This pull request is already covered by existing tests, such as (please describe tests).

(or)

This change added tests and can be verified as follows:

caishunfeng · 2022-04-11T03:27:49Z

I had approve and run the CI

...i/src/main/java/org/apache/dolphinscheduler/plugin/task/api/k8s/AbstractK8sTaskExecutor.java

.../src/main/java/org/apache/dolphinscheduler/plugin/task/api/parameters/k8s/K8sParameters.java

caishunfeng

Please add some test (unit test or e2e) and doc for this pr, thanks.

...i/src/main/java/org/apache/dolphinscheduler/plugin/task/api/k8s/AbstractK8sTaskExecutor.java

...task-api/src/main/java/org/apache/dolphinscheduler/plugin/task/api/utils/ParameterUtils.java

...eduler-task-api/src/main/java/org/apache/dolphinscheduler/plugin/task/api/TaskConstants.java

dolphinscheduler-ui/src/js/conf/home/pages/dag/_source/formModel/tasks/k8s.vue

caishunfeng · 2022-04-17T12:01:16Z

this pr close #9310

zhongjiajie · 2022-04-21T04:20:00Z

this pr close #9310

@caishunfeng the keyword close only work in PR describe #9425 (comment)

caishunfeng · 2022-04-25T02:24:14Z

I had approve to run CI

codecov-commenter · 2022-04-25T02:39:38Z

Codecov Report

Merging #9425 (7d19ee4) into dev (d8b56b8) will decrease coverage by 0.01%.
The diff coverage is 36.33%.

❗ Current head 7d19ee4 differs from pull request most recent head 9ca4b7b. Consider uploading reports for the commit 9ca4b7b to get more accurate results

@@             Coverage Diff              @@
##                dev    #9425      +/-   ##
============================================
- Coverage     40.85%   40.84%   -0.02%     
- Complexity     4630     4681      +51     
============================================
  Files           839      849      +10     
  Lines         34007    34353     +346     
  Branches       3770     3796      +26     
============================================
+ Hits          13895    14030     +135     
- Misses        18782    18983     +201     
- Partials       1330     1340      +10

Impacted Files	Coverage Δ
...nscheduler/api/service/impl/TenantServiceImpl.java	`72.47% <0.00%> (-2.06%)`	⬇️
.../org/apache/dolphinscheduler/common/Constants.java	`80.95% <ø> (ø)`
...g/apache/dolphinscheduler/dao/entity/Resource.java	`45.88% <0.00%> (-1.68%)`	⬇️
...er/server/builder/TaskExecutionContextBuilder.java	`68.88% <0.00%> (-3.21%)`	⬇️
...er/server/master/runner/WorkflowExecuteThread.java	`8.01% <0.00%> (-0.02%)`	⬇️
...r/server/master/runner/task/BaseTaskProcessor.java	`13.30% <0.00%> (-0.62%)`	⬇️
...ver/master/runner/task/ConditionTaskProcessor.java	`5.33% <0.00%> (+0.07%)`	⬆️
...nscheduler/service/process/ProcessServiceImpl.java	`30.70% <0.00%> (-0.15%)`	⬇️
...duler/plugin/task/api/K8sTaskExecutionContext.java	`0.00% <0.00%> (ø)`
...olphinscheduler/plugin/task/api/TaskConstants.java	`0.00% <ø> (ø)`
... and 21 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d8b56b8...9ca4b7b. Read the comment docs.

caishunfeng · 2022-05-13T05:28:07Z

I had rerun the fail CI job.

caishunfeng · 2022-05-13T06:24:50Z

PTAL @kezhenxu94 @zhongjiajie @zhuangchong @devosend

docs/docs/en/guide/task/k8s.md

docs/docs/zh/guide/task/k8s.md

dolphinscheduler-ui/src/views/projects/task/components/node/fields/use-k8s.ts

dolphinscheduler-ui/src/views/projects/task/components/node/format-data.ts

dolphinscheduler-ui/src/views/projects/task/components/node/fields/use-namespace.ts

devosend

The part of front-end LGTM

caishunfeng

LGTM, it's great feature, thans for your contribution.

zhongjiajie

LGTM, waiting for CI

sonarqubecloud · 2022-05-18T09:04:55Z

Kudos, SonarCloud Quality Gate passed!

0 Bugs
0 Vulnerabilities
0 Security Hotspots
5 Code Smells

34.8% Coverage
0.0% Duplication

* [Feature][plugin] Add k8s task in task plugin * [Feature][plugin] fix dos and code problems * [Feature][plugin] refactor some code based on sonar * [Feature][UI] front-end for k8s task plugin * [Feature][plugin] delete some front files * [Feature][plugin] update document * Update docs/docs/zh/guide/task/k8s.md * Update docs/docs/en/guide/task/k8s.md * Update docs/docs/en/guide/task/k8s.md * Update docs/docs/en/guide/task/k8s.md * [Feature][UI] front-end change from review comments * [Feature][UI] replace get namespace list api * [Feature][plugin] change file name * Add kubernetes to zh task list Co-authored-by: hezhao2 <hezhao2@cisco.com> Co-authored-by: William Tong <weitong@cisco.com> Co-authored-by: Jiajie Zhong <zhongjiajie955@gmail.com>

davidzollo · 2022-06-21T14:17:19Z

good job, thanks for your first contribution

caishunfeng requested review from caishunfeng and zhuangchong April 11, 2022 03:27

caishunfeng reviewed Apr 11, 2022

View reviewed changes

...i/src/main/java/org/apache/dolphinscheduler/plugin/task/api/k8s/AbstractK8sTaskExecutor.java Show resolved Hide resolved

.../src/main/java/org/apache/dolphinscheduler/plugin/task/api/parameters/k8s/K8sParameters.java Outdated Show resolved Hide resolved

caishunfeng reviewed Apr 11, 2022

View reviewed changes

dolphinscheduler-ui/src/js/conf/home/pages/dag/_source/formModel/tasks/k8s.vue Outdated Show resolved Hide resolved

zhongjiajie assigned zhaohehuhu Apr 21, 2022

zhongjiajie added the first time contributor First-time contributor label Apr 21, 2022

zhaohehuhu force-pushed the Feature-9310 branch from c696d0c to 688390d Compare April 24, 2022 15:06

caishunfeng self-requested a review April 25, 2022 02:23

zhaohehuhu requested review from Tianqi-Dotes, kezhenxu94 and zhongjiajie as code owners May 10, 2022 06:57

zhaohehuhu force-pushed the Feature-9310 branch 4 times, most recently from 74de75e to 007ae86 Compare May 13, 2022 03:38

zhongjiajie reviewed May 13, 2022

View reviewed changes

devosend reviewed May 16, 2022

View reviewed changes

dolphinscheduler-ui/src/views/projects/task/components/node/fields/use-namespace.ts Outdated Show resolved Hide resolved

zhaohehuhu and others added 4 commits May 17, 2022 12:40

[Feature][plugin] Add k8s task in task plugin

0a94772

[Feature][plugin] fix dos and code problems

71b6613

[Feature][plugin] refactor some code based on sonar

108583b

[Feature][UI] front-end for k8s task plugin

8e53c6d

zhaohehuhu and others added 8 commits May 17, 2022 12:40

[Feature][plugin] delete some front files

24e4a84

[Feature][plugin] update document

99c18de

Update docs/docs/zh/guide/task/k8s.md

f65ca95

Update docs/docs/en/guide/task/k8s.md

d3ef55e

Update docs/docs/en/guide/task/k8s.md

6199754

Update docs/docs/en/guide/task/k8s.md

a02ff92

[Feature][UI] front-end change from review comments

abf52f0

[Feature][UI] replace get namespace list api

58cd95d

zhaohehuhu force-pushed the Feature-9310 branch from f224b2e to 58cd95d Compare May 17, 2022 05:21

[Feature][plugin] change file name

49a8823

zhaohehuhu force-pushed the Feature-9310 branch from 241852a to 49a8823 Compare May 17, 2022 06:58

caishunfeng requested a review from zhongjiajie May 18, 2022 08:23

devosend previously approved these changes May 18, 2022

View reviewed changes

caishunfeng previously approved these changes May 18, 2022

View reviewed changes

Add kubernetes to zh task list

9ca4b7b

zhongjiajie dismissed stale reviews from caishunfeng and devosend via 9ca4b7b May 18, 2022 08:38

zhongjiajie approved these changes May 18, 2022

View reviewed changes

caishunfeng merged commit 5bb1eb0 into apache:dev May 18, 2022

qingwli mentioned this pull request Jun 13, 2022

[Feature][plug-in] Add Spark-K8s task in Spark task plug-in #9550

Closed

3 tasks

devosend added this to the 3.1.0-alpha milestone Jun 16, 2022

zhaohehuhu deleted the Feature-9310 branch December 26, 2024 06:21

Conversation

zhaohehuhu commented Apr 11, 2022 • edited by zhongjiajie Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Purpose of the pull request

Brief change log

Verify this pull request

Uh oh!

caishunfeng commented Apr 11, 2022

Uh oh!

Uh oh!

Uh oh!

caishunfeng left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

caishunfeng commented Apr 17, 2022

Uh oh!

zhongjiajie commented Apr 21, 2022

Uh oh!

caishunfeng commented Apr 25, 2022

Uh oh!

codecov-commenter commented Apr 25, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

caishunfeng commented May 13, 2022

Uh oh!

caishunfeng commented May 13, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

devosend left a comment

Choose a reason for hiding this comment

Uh oh!

caishunfeng left a comment

Choose a reason for hiding this comment

Uh oh!

zhongjiajie left a comment

Choose a reason for hiding this comment

Uh oh!

sonarqubecloud bot commented May 18, 2022

Uh oh!

davidzollo commented Jun 21, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

zhaohehuhu commented Apr 11, 2022 •

edited by zhongjiajie

Loading

codecov-commenter commented Apr 25, 2022 •

edited

Loading

caishunfeng commented May 13, 2022 •

edited

Loading