Skip to content

[Feature][Logging] Add support for logging into remote storage #8543

@EricGao888

Description

@EricGao888

Search before asking

  • I had searched in the issues and found no similar feature requirement.

Description

  • Add support for remote logging so that the tasks logs could be preserved in remote storage service like AWS S3 or Alibaba-Cloud OSS.

Use case

  • Currently, it seems dolphin scheduler does not support logging into remote storage such as AWS S3, Alibaba-Cloud OSS, etc.
  • If dolphin scheduler is deployed on a remote cluster like EMR and the cluster is torn down or there's something wrong with the workers, users will probably lose the logs of previous executed tasks.

Related issues

No response

Are you willing to submit a PR?

  • Yes I am willing to submit a PR!

Code of Conduct

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions