Purpose

Add support for OpenVLA model in vLLM. OpenVLA is a vision-language-action model that uses timm-based vision backbones (Prismatic architecture) with LLM backbones for action prediction tasks. This implementation follows the same pattern as DeepSeek-VL2, which also uses timm for vision processing.

This PR adds:

• Model executor implementation (vllm/model_executor/models/openvla.py)
• Configuration class (vllm/transformers_utils/configs/openvla.py)
• Processor class (vllm/transformers_utils/processors/openvla.py)

The implementation supports:

• Single and fused vision backbones using timm ViT models
• Multiple LLM backbones (Llama-2, Mistral, Phi-3)
• Image embedding insertion via prompt updates
• Tensor parallelism support for vision backbone

FIX #14739

Test Plan

1. Basic inference test:

   vllm serve openvla/openvla-7b --trust-remote-code

2. Test with image input:

   • Use OpenAI-compatible API to send requests with image data
   • Verify image embeddings are correctly processed and inserted

3. Compare outputs with HuggingFace implementation:

   • Run inference on same inputs with both vLLM and HF implementations
   • Verify output logits/tokens match

Test Result

[To be filled after testing]