Skip to content

[Data] Elevate num_cpus/gpus and memory as top-level params in most APIs #54708

@alexeykudinkin

Description

@alexeykudinkin

Description

Right now some APIs (like map/map_batches) allow you to specify num_cpus/gpus/etc, while some others (like read_*) require you to specify ray_remote_args={"num_cpus": ...}.

Instead we'd promote some routine configs like num_cpus/gpus/memory to be top-level params in all of Ray Data APIs

Related to #40673

Use case

No response

Metadata

Metadata

Assignees

Labels

P1Issue that should be fixed within a few weeksdataRay Data-related issuesenhancementRequest for new feature and/or capabilityusability

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions