Skip to content

[Data]: Support making embeddings batch predictions with build_llm_processor #55384

Closed
Closed
[Data]: Support making embeddings batch predictions with build_llm_processor#55384
Assignees
nrghosh
Labels
community-backlogenhancementRequest for new feature and/or capabilityllmperformancetriageNeeds triage (eg: priority, bug/not-bug, and owning component)usability
@martinbomio

Description

@martinbomio
martinbomio
opened on Aug 7, 2025

Description

Ray data supports making batch chat completion batch predictions with vLLM through vLLMEngineProcessorConfig and build_llm_processor. Would be great to have something similar for /v1/embeddings

Use case

Make batch predictions with embeddings models

Metadata

Metadata

Assignees

Labels

community-backlogenhancementRequest for new feature and/or capabilityllmperformancetriageNeeds triage (eg: priority, bug/not-bug, and owning component)usability

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions