-
Notifications
You must be signed in to change notification settings - Fork 5k
[Feature][PyDolphinScheduler] Support more task plugin in PyDolphinScheduler #11906
Copy link
Copy link
Closed
Labels
backendfeaturenew featurenew featuregood first issuegood first issuegood first issuehelp wantedExtra attention is neededExtra attention is needed
Milestone
Description
Search before asking
- I had searched in the issues and found no similar feature requirement.
Description
DolphinScheduler has recently added a number of task plugins. We have to add more support in pydolphinscheduler.
List of types in DolphinScheduler: https://dolphinscheduler.apache.org/en-us/docs/dev/user_doc/guide/task/java.html
List of types in PydolphinScheduler: https://dolphinscheduler.apache.org/python/dev/tasks/index.html
There are task types in DolphinScheduler but not in PydolphinScheduler:
- Pigeon
- SeaTunnel
- Amazon EMR
- Apache Zepplin [Feature][PyDolphinScheduler] Support Zeppelin task in pyds #11908
- Jupyter [Feature][Module Name] Support Jupyter task in pyds #11907
- Hive CLI [Feature][PyDolphinScheduler] Support HiveCLI task in pyds #11909
- Kubernetes [Feature][PyDolphinScheduler] Support Kubernetes task in pyds #12465
- MLflow [Feature][PyDolphinScheduler] Support MLflow task in pyds #11923
- OpenMLDB [Feature][PyDolphinScheduler] Support OpenMLDB task in pyds #11943
- DVC [Feature][PyDolphinScheduler] Support DVC task in pyds #11922
- Dinky [Feature][PyDolphinScheduler] Support Dlinky task in pyds #11916
- Java [Feature][PyDolphinScheduler] Support JAVA task in pyds #11914
- SageMaker [Feature][PyDolphinScheduler] Support SageMaker task in pyds #11921
- ChunJun [Feature][PyDolphinScheduler] Support ChunJun task in pyds #11915
- Pytorch [Feature][PyDolphinScheduler] Support Pytorch task in pyds #11924
After finished the pydolphinscheduler implementation, Hope to supplement the YAML sample, if possible
Like this: https://dolphinscheduler.apache.org/python/dev/tutorial.html#overview-of-tutorial
Use case
No response
Related issues
No response
Are you willing to submit a PR?
- Yes I am willing to submit a PR!
Code of Conduct
- I agree to follow this project's Code of Conduct
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
backendfeaturenew featurenew featuregood first issuegood first issuegood first issuehelp wantedExtra attention is neededExtra attention is needed