[Bug] fix run on docker and k8s by caishunfeng · Pull Request #10026 · apache/dolphinscheduler

caishunfeng · 2022-05-13T09:37:39Z

Purpose of the pull request

this pr close #10027
relative pr: #9997 #9733

Brief change log

Verify this pull request

Manually verified the change by testing locally.

deploy/docker/docker-compose.yml

docs/docs/en/guide/start/docker.md

deploy/docker/docker-compose.yml

docs/docs/en/guide/start/docker.md

deploy/docker/docker-compose.yml

codecov-commenter · 2022-05-15T09:50:06Z

Codecov Report

Merging #10026 (65ae8b4) into dev (baf654c) will decrease coverage by 0.02%.
The diff coverage is n/a.

@@             Coverage Diff              @@
##                dev   #10026      +/-   ##
============================================
- Coverage     40.89%   40.86%   -0.03%     
  Complexity     4631     4631              
============================================
  Files           839      839              
  Lines         33983    34005      +22     
  Branches       3764     3769       +5     
============================================
- Hits          13896    13895       -1     
- Misses        18756    18782      +26     
+ Partials       1331     1328       -3

Impacted Files	Coverage Δ
...inscheduler/api/controller/ExecutorController.java	`28.30% <0.00%> (-11.70%)`	⬇️
...org/apache/dolphinscheduler/remote/utils/Host.java	`37.77% <0.00%> (-2.23%)`	⬇️
...e/dolphinscheduler/remote/NettyRemotingClient.java	`52.11% <0.00%> (-1.41%)`	⬇️
...r/plugin/task/sqoop/parameter/SqoopParameters.java	`53.33% <0.00%> (-1.34%)`	⬇️
.../org/apache/dolphinscheduler/api/enums/Status.java	`100.00% <0.00%> (ø)`
...lphinscheduler/plugin/task/dq/DataQualityTask.java	`0.00% <0.00%> (ø)`
...dolphinscheduler/remote/future/ResponseFuture.java	`83.60% <0.00%> (+1.63%)`	⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update baf654c...65ae8b4. Read the comment docs.

zhongjiajie · 2022-05-15T12:10:38Z

I restart the failed CI

deploy/kubernetes/dolphinscheduler/templates/job-dolphinscheduler-schema-initializer.yaml

SbloodyS

LGTM

kezhenxu94 · 2022-05-16T04:51:07Z

deploy/docker/.env

 #
 HUB=ghcr.io/apache/dolphinscheduler
 TAG=latest
+DOCKER=TRUE


This is already set in all Dockerfile

dolphinscheduler/dolphinscheduler-tools/src/main/docker/Dockerfile

Lines 20 to 21 in 67cc260

ENV DOCKER true

ENV TZ Asia/Shanghai

This is already set in all Dockerfile

dolphinscheduler/dolphinscheduler-tools/src/main/docker/Dockerfile

Lines 20 to 21 in 67cc260

ENV DOCKER true

ENV TZ Asia/Shanghai

I had updated and thanks for your reminder, YYDS.

sonarqubecloud · 2022-05-16T06:51:57Z

Kudos, SonarCloud Quality Gate passed!

0 Bugs
0 Vulnerabilities
0 Security Hotspots
0 Code Smells

No Coverage information
No Duplication information

* fix docker-compose init schema * recovery depend on zk * update doc and dockerfile * fix run on k8s * udpate doc * add DOCKER flag & update doc * remove repeat DOCKER env (cherry picked from commit b1885c7)

commit 8046180 Author: 王心雨 <wangxinyu@4paradigm.com> Date: Thu May 19 12:34:07 2022 +0800 去掉错误提交的代码 commit 5e9e9a9 Author: 王心雨 <wangxinyu@4paradigm.com> Date: Thu May 19 12:23:44 2022 +0800 加入openmldb task commit b1885c7 Author: caishunfeng <caishunfeng2021@gmail.com> Date: Mon May 16 15:41:45 2022 +0800 [Bug] fix run on docker and k8s (apache#10026) * fix docker-compose init schema * recovery depend on zk * update doc and dockerfile * fix run on k8s * udpate doc * add DOCKER flag & update doc * remove repeat DOCKER env commit d643e1c Author: Dannila <94423827+Dannila@users.noreply.github.com> Date: Mon May 16 15:06:21 2022 +0800 [Fix-10039] Flink run command when perfecting Python jobs (apache#10042) * [fix] flink task * [fix] flink task commit 3e471f8 Author: Jiajie Zhong <zhongjiajie955@hotmail.com> Date: Mon May 16 11:01:32 2022 +0800 [feat] Add batch rerun and stop for process instance (apache#10013) * [feat] Add batch rerun and stop for process instance * fix execute type pass * error message changed * correct error message * add note to i18n ignore * enhance the note of process instance ids commit f666c64 Author: chuxing <923622908@qq.com> Date: Mon May 16 09:53:45 2022 +0800 [doc] Correct docs of development-environment-setup (apache#9995) commit 359cbe2 Author: zixi0825 <sunzhaohe0825@gmail.com> Date: Sun May 15 17:46:31 2022 +0800 [dataquality] Fix task commnd null bug (apache#9974) commit 2423fa5 Author: xiangzihao <460888207@qq.com> Date: Sun May 15 13:04:48 2022 +0800 [Feature-10034][CI] Add postgresql cluster test in CI (apache#10035) * add postgresql cluster test * add postgresql cluster test * add postgresql cluster test * add postgresql cluster test commit df04c4a Author: chuxing <923622908@qq.com> Date: Sun May 15 10:24:31 2022 +0800 [fix-9991][worker]fix statement is closed before resultSet.getMetaData() (apache#10014) commit baf654c Author: xiangzihao <460888207@qq.com> Date: Sat May 14 12:30:57 2022 +0800 [Feature-9474] [CI] Add cluster test script verify on shell script (apache#9997) * cluster test * fix init db failed * fix init db failed * fix init db failed * fix init db failed * fix init db failed * add github actions * add github actions * add github actions * add github actions * add github actions * add github actions * add sudo * add github actions * add github actions * add github actions * add github actions * add github actions * add github actions * add github actions * add github actions * add github actions * add github actions * add github actions * add github actions * add github actions * add github actions * add github actions * remove cluster-test workflows * add github actions * add github actions * refactor test to docker mode * refactor test to docker mode * refactor test to docker mode * refactor test to docker mode * remove create schema logic * remove create schema logic * remove create schema logic * fix runnint cluster test error * fix runnint cluster test error * fix runnint cluster test error * fix runnint cluster test error * fix runnint cluster test error * fix runnint cluster test error * fix runnint cluster test error * fix runnint cluster test error * fix runnint cluster test error * fix runnint cluster test error * fix runnint cluster test error * fix runnint cluster test error * fix runnint cluster test error * fix runnint cluster test error * fix runnint cluster test error * fix runnint cluster test error * fix runnint cluster test error * fix runnint cluster test error * fix runnint cluster test error * fix runnint cluster test error * fix runnint cluster test error * fix runnint cluster test error * fix runnint cluster test error * fix runnint cluster test error * fix runnint cluster test error * fix runnint cluster test error * add github actions * add github actions * add github actions * add cluster test timeout * add cluster start test loop check * add cluster start test loop check * optimize cluster start test loop check commit ee2c516 Author: 旺阳 <wang@lqwang.net> Date: Fri May 13 10:39:05 2022 +0800 [doc] Correct kubernetes (apache#9985) Co-authored-by: qingwli <qingwli@cisco.com> commit d17379d Author: rockfang <657297417@qq.com> Date: Fri May 13 10:16:54 2022 +0800 [Fix][UI] Fix errorOutputPath column in dataquality page (apache#10015) commit 8036936 Author: Eric Gao <ericgao.apache@gmail.com> Date: Thu May 12 19:55:49 2022 +0800 [feat][task plugin] Add Jupyter task plugin (apache#9872) Co-authored-by: Amy0104 <97265214+Amy0104@users.noreply.github.com> commit 0fe7548 Author: QuakeWang <45645138+QuakeWang@users.noreply.github.com> Date: Thu May 12 18:23:43 2022 +0800 [doc] Add example and notice about task type Dependent (apache#10001) Co-authored-by: Jiajie Zhong <zhongjiajie955@gmail.com> commit 18bfe63 Author: Amy0104 <97265214+Amy0104@users.noreply.github.com> Date: Thu May 12 16:54:03 2022 +0800 [Fix][UI] Support only one file upload on the file manage page. (apache#10007) commit dbdbfea Author: WangJPLeo <103574007+WangJPLeo@users.noreply.github.com> Date: Thu May 12 16:31:53 2022 +0800 [Fix-9975] The selected task instance was recreated when the Master service fail… (apache#9976) * The selected task instance was recreated when the Master service failed over. * Returns the expression result directly. * Use Recovery to determine whether to use the old task instance. commit 00f1029 Author: Amy0104 <97265214+Amy0104@users.noreply.github.com> Date: Thu May 12 15:27:55 2022 +0800 [Fix][UI] Fix the task name validator error. (apache#10008) commit bce5a28 Author: Kerwin <37063904+zhuangchong@users.noreply.github.com> Date: Thu May 12 14:42:39 2022 +0800 [doc] Add the description about execute type in SQL task (apache#9987) commit 53cd700 Author: rockfang <657297417@qq.com> Date: Thu May 12 14:11:52 2022 +0800 [Fix-10002] Fix some bugs in datasource list (apache#10005) * fix: fix ellipsis bug in table column * fix ellipsis bug in table column * fix: json-highlight component support nested jsonstring * fix: make datasource description show commit 8d17fd2 Author: Jiajie Zhong <zhongjiajie955@hotmail.com> Date: Thu May 12 11:30:01 2022 +0800 [ci] Dead link check all markdown file (apache#10004) there are 162 markdown files in dir `docs` and all file in project is 175 files, so I think check all file will not add too much load for our ci, but it could discovered the dead link in time to avoid some thing like apache#9998 commit 99b1c40 Author: songjianet <1778651752@qq.com> Date: Wed May 11 22:15:36 2022 +0800 [Docs] Fix docker link. (apache#9998) commit d4aeee1 Author: Tq <tianqitobethefirst@gmail.com> Date: Wed May 11 18:37:03 2022 +0800 [Bug] [MASTER-9811]fix cmd param to overwrite global param when executing complement (apache#9952) * fix cmd param to overwrite global param when executing complement * fix cmd param to overwrite global param when executing complement commit bc1c15b Author: qianli2022 <97489722+qianli2022@users.noreply.github.com> Date: Wed May 11 16:40:30 2022 +0800 [Feature-8252][doc] K8s and namespace manager docs and web page update (apache#9881) * test * test * doc * fix image error * fix * Update docs/docs/en/guide/security.md * Update docs/docs/en/guide/security.md * Update docs/docs/en/guide/security.md * Update docs/docs/zh/guide/security.md * fix doc Co-authored-by: qianl4 <qianl4@cicso.com> Co-authored-by: William Tong <weitong@cisco.com> Co-authored-by: Jiajie Zhong <zhongjiajie955@gmail.com>

caishunfeng requested review from Tianqi-Dotes and zhongjiajie as code owners May 13, 2022 09:37

SbloodyS reviewed May 13, 2022

View reviewed changes

deploy/docker/docker-compose.yml Show resolved Hide resolved

songjianet assigned caishunfeng May 13, 2022

songjianet added bug Something isn't working backend labels May 13, 2022

zhongjiajie reviewed May 13, 2022

View reviewed changes

docs/docs/en/guide/start/docker.md Show resolved Hide resolved

deploy/docker/docker-compose.yml Outdated Show resolved Hide resolved

docs/docs/en/guide/start/docker.md Outdated Show resolved Hide resolved

docs/docs/en/guide/start/docker.md Outdated Show resolved Hide resolved

SbloodyS reviewed May 14, 2022

View reviewed changes

deploy/docker/docker-compose.yml Outdated Show resolved Hide resolved

SbloodyS reviewed May 14, 2022

View reviewed changes

deploy/docker/docker-compose.yml Outdated Show resolved Hide resolved

caishunfeng added 5 commits May 14, 2022 22:56

fix docker-compose init schema

ad8fd1e

recovery depend on zk

755bf15

update doc and dockerfile

c3cf630

fix run on k8s

530fd59

udpate doc

3ed9a8a

caishunfeng force-pushed the docker-compose-run branch from fecd95c to 530fd59 Compare May 15, 2022 09:32

caishunfeng changed the title ~~[Bug] fix docker-compose init schema~~ [Bug] fix run on docker-compose and k8s May 15, 2022

caishunfeng changed the title ~~[Bug] fix run on docker-compose and k8s~~ [Bug] fix run on docker and k8s May 15, 2022

caishunfeng requested review from SbloodyS, kezhenxu94 and zhongjiajie May 15, 2022 09:54

SbloodyS reviewed May 16, 2022

View reviewed changes

deploy/kubernetes/dolphinscheduler/templates/job-dolphinscheduler-schema-initializer.yaml Show resolved Hide resolved

add DOCKER flag & update doc

7bfdfd1

SbloodyS previously approved these changes May 16, 2022

View reviewed changes

kezhenxu94 reviewed May 16, 2022

View reviewed changes

remove repeat DOCKER env

65ae8b4

caishunfeng dismissed SbloodyS’s stale review via 65ae8b4 May 16, 2022 06:30

kezhenxu94 approved these changes May 16, 2022

View reviewed changes

caishunfeng merged commit b1885c7 into apache:dev May 16, 2022

qingwli mentioned this pull request May 18, 2022

[Bug] [Docker] Docker Doc Error && Alert Plugin WebexTeams Chinese Translation Error #10083

Closed

3 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Bug] fix run on docker and k8s#10026

[Bug] fix run on docker and k8s#10026
caishunfeng merged 7 commits intoapache:devfrom
caishunfeng:docker-compose-run

caishunfeng commented May 13, 2022 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

codecov-commenter commented May 15, 2022 •

edited

Loading

Uh oh!

zhongjiajie commented May 15, 2022

Uh oh!

Uh oh!

SbloodyS left a comment

Uh oh!

kezhenxu94 May 16, 2022

Uh oh!

caishunfeng May 16, 2022

Uh oh!

sonarqubecloud bot commented May 16, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

Conversation

caishunfeng commented May 13, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Purpose of the pull request

Brief change log

Verify this pull request

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

codecov-commenter commented May 15, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

zhongjiajie commented May 15, 2022

Uh oh!

Uh oh!

SbloodyS left a comment

Choose a reason for hiding this comment

Uh oh!

kezhenxu94 May 16, 2022

Choose a reason for hiding this comment

Uh oh!

caishunfeng May 16, 2022

Choose a reason for hiding this comment

Uh oh!

sonarqubecloud bot commented May 16, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

caishunfeng commented May 13, 2022 •

edited

Loading

codecov-commenter commented May 15, 2022 •

edited

Loading