Skip to content

[Bug][api] Upgrade from 1.2.0 to 1.3.0,Run the spark task, the error is "spark task params is not valid" #3397

@xingchun-chen

Description

@xingchun-chen

1.Upgrade from 1.2.0 to 1.3.0
2.Run the spark task, the error is "spark task params is not valid"
image

The error message is as follows:

[INFO] 2020-08-04 15:54:20.988 - [taskAppId=TASK-2-31-42]:[72] - spark task params {"mainArgs":"/chenxingchun/words.txt /chenxingchun/out_c","driverMemory":"512M","executorMemory":"2G","programType":"SCALA","mainClass":"com.journey.spark.WordCount","driverCores":1,"deployMode":"cluster","executorCores":2,"mainJar":{"res":"bigdata-1.0-SNAPSHOT.jar"},"numExecutors":2,"localParams":[],"others":"","resourceList":[{"res":"test.sh"},{"res":"udf-demo-SNAPSHOT.jar"}]}
[ERROR] 2020-08-04 15:54:20.989 org.apache.dolphinscheduler.server.worker.runner.TaskExecuteThread:[140] - task scheduler failure
java.lang.RuntimeException: spark task params is not valid
at org.apache.dolphinscheduler.server.worker.task.spark.SparkTask.init(SparkTask.java:77)
at org.apache.dolphinscheduler.server.worker.runner.TaskExecuteThread.run(TaskExecuteThread.java:126)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
[INFO] 2020-08-04 15:54:21.996 org.apache.dolphinscheduler.service.log.LogClientService:[100] - view log path /data1/dolphinscheduler_1.3.2/logs/2/31/42.log

Which version of Dolphin Scheduler:
-[1.3.2-release]

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

No type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions