Skip to content

[feat][python] Support Pytorch task in python api#11975

Merged
zhongjiajie merged 3 commits intoapache:devfrom
jieguangzhou:pyds-pytorch
Sep 16, 2022
Merged

[feat][python] Support Pytorch task in python api#11975
zhongjiajie merged 3 commits intoapache:devfrom
jieguangzhou:pyds-pytorch

Conversation

@jieguangzhou
Copy link
Copy Markdown
Member

Purpose of the pull request

close: #11924

  • add pyds Pytorch task
  • add YAML example
  • add doc

Brief change log

Verify this pull request

This pull request is code cleanup without any test coverage.

(or)

This pull request is already covered by existing tests, such as (please describe tests).

(or)

This change added tests and can be verified as follows:

(or)

@codecov-commenter
Copy link
Copy Markdown

codecov-commenter commented Sep 15, 2022

Codecov Report

Merging #11975 (3660b12) into dev (0deafba) will increase coverage by 0.13%.
The diff coverage is n/a.

@@             Coverage Diff              @@
##                dev   #11975      +/-   ##
============================================
+ Coverage     38.57%   38.71%   +0.13%     
- Complexity     4049     4064      +15     
============================================
  Files           995     1005      +10     
  Lines         36739    36792      +53     
  Branches       4278     4279       +1     
============================================
+ Hits          14172    14243      +71     
+ Misses        20945    20924      -21     
- Partials       1622     1625       +3     
Impacted Files Coverage Δ
...phinscheduler/remote/utils/NamedThreadFactory.java 91.66% <0.00%> (-8.34%) ⬇️
...eduler/server/worker/task/WorkerHeartBeatTask.java 69.38% <0.00%> (-8.17%) ⬇️
...r/plugin/task/sqoop/parameter/SqoopParameters.java 55.12% <0.00%> (-1.29%) ⬇️
...olphinscheduler/plugin/task/pigeon/PigeonTask.java 53.48% <0.00%> (-0.68%) ⬇️
...erver/worker/runner/WorkerTaskExecuteRunnable.java 29.41% <0.00%> (-0.44%) ⬇️
...phinscheduler/plugin/task/emr/EmrAddStepsTask.java 65.27% <0.00%> (-0.44%) ⬇️
...heduler/api/service/impl/SchedulerServiceImpl.java 8.60% <0.00%> (-0.04%) ⬇️
...r/server/master/service/MasterFailoverService.java 51.90% <0.00%> (-0.03%) ⬇️
...r/server/master/service/WorkerFailoverService.java 50.43% <0.00%> (-0.01%) ⬇️
.../org/apache/dolphinscheduler/spi/enums/DbType.java 0.00% <0.00%> (ø)
... and 35 more

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

jieguangzhou and others added 2 commits September 16, 2022 10:11
…ler/tasks/pytorch.py


fix Optional keyword

Co-authored-by: Jiajie Zhong <zhongjiajie955@gmail.com>
@sonarqubecloud
Copy link
Copy Markdown

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 0 Code Smells

No Coverage information No Coverage information
No Duplication information No Duplication information

@SbloodyS SbloodyS added this to the 3.1.0 milestone Sep 16, 2022
@jieguangzhou
Copy link
Copy Markdown
Member Author

@zhongjiajie PTAL, thanks

Copy link
Copy Markdown
Member

@zhongjiajie zhongjiajie left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks

@zhongjiajie zhongjiajie merged commit 5202e5c into apache:dev Sep 16, 2022
@zhongjiajie zhongjiajie changed the title [Feature][PyDolphinScheduler] Support Pytorch task in pyds [feat][python] Support Pytorch task in python api Sep 16, 2022
xdu-chenrj pushed a commit to xdu-chenrj/dolphinscheduler that referenced this pull request Sep 17, 2022
Co-authored-by: Jiajie Zhong <zhongjiajie955@gmail.com>
zhongjiajie added a commit that referenced this pull request Sep 18, 2022
Co-authored-by: Jiajie Zhong <zhongjiajie955@gmail.com>
(cherry picked from commit 5202e5c)
@caishunfeng caishunfeng added the release cherry-pick Mark this issue/PR had cherry-pick for release version label Sep 19, 2022
xdu-chenrj pushed a commit to xdu-chenrj/dolphinscheduler that referenced this pull request Oct 13, 2022
Co-authored-by: Jiajie Zhong <zhongjiajie955@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

document Pyscheduler release cherry-pick Mark this issue/PR had cherry-pick for release version

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature][PyDolphinScheduler] Support Pytorch task in pyds

5 participants