Skip to content

[runtime env] Can we avoid merging two runtime envs? #21494

@SongGuyang

Description

@SongGuyang

Search before asking

  • I had searched in the issues and found no similar feature requirement.

Description

In current design, the actor level runtime env can inherit job level runtime env, like:
If the job level runtime env is:

{
   "py_modules": "s3://ab.zip",
    "pip": ["requests", "pendulum==2.1.2"],
    "env_vars":{
        "a":"b",
    }
}

and if the actor level runtime env is:

{
   "py_modules": "s3://cd.zip",
    "env_vars":{
        "c":"d",
    }
}

The final merged runtime env for actor is:

{
   "py_modules": "s3://cd.zip",
    "pip": ["requests", "pendulum==2.1.2"],
    "env_vars":{
        "a":"b",
        "c":"d",
    }
}

More details we can see https://github.com/ray-project/ray/blob/master/src/ray/core_worker/core_worker.cc#L1355 and https://github.com/ray-project/ray/blob/master/src/ray/core_worker/test/core_worker_test.cc#L937.

Do we have some reasons to support this logic?
I think this logic is not clear and brings some complexity. Can we make it easier? Like:

  • If there is no runtime env option when we create actor, inherit the job level runtime env.
  • Otherwise, use the actor runtime env directly and don't do the merging.
  • We also can give a new API to help users getting job level runtime env if needed.

Use case

No response

Related issues

No response

Are you willing to submit a PR?

  • Yes I am willing to submit a PR!

Metadata

Metadata

Assignees

Labels

P2Important issue, but not time-criticalcoreIssues that should be addressed in Ray Corecore-runtime-envIssues related to Ray environment dependenciesenhancementRequest for new feature and/or capabilitypending-cleanupThis issue is pending cleanup. It will be removed in 2 weeks after being assigned.

Type

No type

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions