[Fix-#3956][Master] When running a task, the resource file is lost by BoYiZhang · Pull Request #3957 · apache/dolphinscheduler

BoYiZhang · 2020-10-20T11:27:54Z

When running a shell task, the resource file is lost, which results in an error
this close #3956

TaskParams
Taskparams does not have a resourcelist. When converting, the content of resourcelist will be lost

Not only shell tasks, param parameters in other task types may be overridden

The code is as follows:
VarPoolUtils#setTaskNodeLocalParams

/**
     * setTaskNodeLocalParams
     * @param taskNode taskNode
     * @param prop LocalParamName
     * @param value LocalParamValue
     */
    public static void setTaskNodeLocalParams(TaskNode taskNode, String prop, Object value) {
        String taskParamsJson = taskNode.getParams();
        TaskParams taskParams = JSONUtils.parseObject(taskParamsJson, TaskParams.class);
        if (taskParams == null) {
            return;
        }
        taskParams.setLocalParamValue(prop, value);
        taskNode.setParams(JSONUtils.toJsonString(taskParams));
    }

Delete invalid field: executorcores Modify verification prompt

update code

…cheduler into dev

dag add close buttion

…cheduler into dev

update code

modify submit workflow button

modify CLICK_ADD_BUTTON

delete print

1

in addition update '- XX: largepagesizeinbytes = 128M' to '- XX: largepagesizeinbytes = 10M'

This reverts commit 3a2cba7

upadtecode

in addition update '- XX: largepagesizeinbytes = 128M' to '- XX: largepagesizeinbytes = 10M'

update code

update code from apache

update data code

update code from ds

update from code

update code

update code from apache

update code from apa

update from apache

update code from apache

…rror

codecov-io · 2020-10-20T12:27:19Z

Codecov Report

Merging #3957 into dev will increase coverage by 0.10%.
The diff coverage is 80.00%.

@@             Coverage Diff              @@
##                dev    #3957      +/-   ##
============================================
+ Coverage     39.83%   39.94%   +0.10%     
- Complexity     2921     2931      +10     
============================================
  Files           460      459       -1     
  Lines         21826    21793      -33     
  Branches       2653     2647       -6     
============================================
+ Hits           8695     8705      +10     
+ Misses        12308    12271      -37     
+ Partials        823      817       -6

Impacted Files	Coverage Δ	Complexity Δ
...he/dolphinscheduler/common/utils/VarPoolUtils.java	`82.05% <80.00%> (+6.49%)`	`7.00 <0.00> (ø)`
...e/dolphinscheduler/remote/NettyRemotingClient.java	`50.00% <0.00%> (-2.78%)`	`9.00% <0.00%> (-2.00%)`
...dolphinscheduler/remote/future/ResponseFuture.java	`79.66% <0.00%> (-1.70%)`	`17.00% <0.00%> (-1.00%)`
...eduler/server/worker/runner/TaskExecuteThread.java	`55.20% <0.00%> (-0.80%)`	`12.00% <0.00%> (-1.00%)`
...er/master/dispatch/host/assign/RandomSelector.java	`83.33% <0.00%> (+5.55%)`	`4.00% <0.00%> (+1.00%)`
...pache/dolphinscheduler/common/utils/JSONUtils.java	`74.28% <0.00%> (+7.14%)`	`18.00% <0.00%> (ø%)`
...apache/dolphinscheduler/common/model/TaskNode.java	`69.76% <0.00%> (+32.55%)`	`39.00% <0.00%> (+23.00%)`

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 4425968...aace551. Read the comment docs.

…rror

sonarqubecloud · 2020-10-21T15:45:38Z

Kudos, SonarCloud Quality Gate passed!

0 Bugs
0 Vulnerabilities (and 0 Security Hotspots to review)
0 Code Smells

81.8% Coverage
0.0% Duplication

...hinscheduler-common/src/main/java/org/apache/dolphinscheduler/common/utils/VarPoolUtils.java

CalvinKirs

+1

BoYiZhang and others added 30 commits June 30, 2020 00:11

fix bug

d62c98e

Delete invalid field: executorcores Modify verification prompt

fix bug

2c059f9

Delete invalid field: executorcores Modify verification prompt

fix bug

bc98ab5

Delete invalid field: executorcores Modify verification prompt

Merge pull request #1 from apache/dev

1fdb8a2

update code

Merge branch 'dev' into dev

5fec36c

Merge branch 'dev' into dev

9d01a10

Merge branch 'dev' into dev

0332693

Merge branch 'dev' into dev

090c32e

dag add close button

038beff

reset last version

50f33a1

reset last version

9193cea

Merge branch 'dev' of https://github.com/BoYiZhang/incubator-dolphins…

e4d76c5

…cheduler into dev

dag add close buttion

4fc8144

dag add close buttion

Merge branch 'dev' of https://github.com/BoYiZhang/incubator-dolphins…

259a644

…cheduler into dev

Merge pull request #1 from apache/dev

7949b67

update code

update CLICK_SAVE_WORKFLOW_BUTTON xpath

4bf187c

Merge remote-tracking branch 'origin/dev' into dev

3ccb355

updae CLICK_SAVE_WORKFLOW_BUTTON xpath

16a4735

updae CLICK_SAVE_WORKFLOW_BUTTON xpath

e6bd0e3

updae CLICK_SAVE_WORKFLOW_BUTTON xpath

3c41fbe

Update CreateWorkflowLocator.java

5a3df2e

modify submit workflow button

Update CreateWorkflowLocator.java

290d22d

Update CreateWorkflowLocator.java

19462dd

modify CLICK_ADD_BUTTON

Update CreateWorkflowLocator.java

295aead

delete print

Update CreateWorkflowLocator.java

89ec911

1

Update CreateWorkflowLocator.java

c7b87fa

1

Setting '-XX:+DisableExplicitGC ' causes netty memory leaks

3a2cba7

in addition update '- XX: largepagesizeinbytes = 128M' to '- XX: largepagesizeinbytes = 10M'

Update dag.vue

15e64ea

Update dag.vue

f22adad

Update dag.vue

c0ba1dc

BoYiZhang and others added 22 commits July 14, 2020 08:59

Merge remote-tracking branch 'origin/dev' into dev

49f3398

Revert "Setting '-XX:+DisableExplicitGC ' causes netty memory leaks"

06404f5

This reverts commit 3a2cba7

Merge pull request #2 from apache/dev

b21aca8

upadtecode

Setting '-XX:+DisableExplicitGC ' causes netty memory leaks

deb19ee

in addition update '- XX: largepagesizeinbytes = 128M' to '- XX: largepagesizeinbytes = 10M'

Merge pull request #3 from apache/dev

a655e1c

update code

Merge branch 'dev' into dev

3823a08

Update dolphinscheduler-daemon.sh

b11714b

Merge pull request #7 from apache/dev

0e11365

update code from apache

Merge pull request #8 from apache/dev

d329914

update code from apache

Merge pull request #9 from apache/dev

a7cbfb1

update code from apache

Merge pull request #12 from apache/dev

c3666f3

update data code

Merge pull request #13 from apache/dev

dffcf1e

update code from ds

Merge pull request #14 from apache/dev

a26724c

update from code

Merge pull request #15 from apache/dev

5ea4022

update code

Merge pull request #16 from apache/dev

5480a65

update code from apache

Merge pull request #19 from apache/dev

622bce6

update code from apa

Merge pull request #20 from apache/dev

6521601

update from apache

Merge pull request #21 from apache/dev

5f817f2

update code from apache

When running a task, the resource file is lost, which results in an e…

fc56581

…rror

When running a task, the resource file is lost, which results in an e…

aace551

…rror

When running a task, the resource file is lost, which results in an e…

ef6cc62

…rror

When running a task, the resource file is lost, which results in an e…

48177b6

…rror

When running a task, the resource file is lost, which results in an e…

7dd717a

…rror

BoYiZhang changed the title ~~[Fix-#3956][Master] When running a task, the resource file is lost, which results in an error~~ [Fix-#3956][Master] When running a task, the resource file is lost Oct 21, 2020

hermeshephaestus approved these changes Oct 24, 2020

View reviewed changes

CalvinKirs requested changes Oct 24, 2020

View reviewed changes

...hinscheduler-common/src/main/java/org/apache/dolphinscheduler/common/utils/VarPoolUtils.java Show resolved Hide resolved

CalvinKirs approved these changes Oct 24, 2020

View reviewed changes

CalvinKirs merged commit 05b248f into apache:dev Oct 24, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Fix-#3956][Master] When running a task, the resource file is lost#3957

[Fix-#3956][Master] When running a task, the resource file is lost#3957
CalvinKirs merged 54 commits intoapache:devfrom
BoYiZhang:pr_3956

BoYiZhang commented Oct 20, 2020 •

edited

Loading

Uh oh!

codecov-io commented Oct 20, 2020

Uh oh!

sonarqubecloud bot commented Oct 21, 2020

Uh oh!

Uh oh!

CalvinKirs left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

Conversation

BoYiZhang commented Oct 20, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codecov-io commented Oct 20, 2020

Codecov Report

Uh oh!

sonarqubecloud bot commented Oct 21, 2020

Uh oh!

Uh oh!

CalvinKirs left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

BoYiZhang commented Oct 20, 2020 •

edited

Loading