Summary of Changes

Hello @ppraneth, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly enhances the performance of vision tensor processing within the model gateway by implementing a zero-copy access mechanism. By transitioning from deep copies to borrowed slices for contiguous data, it drastically reduces memory allocations and processing latency, ensuring more efficient handling of large multimodal inputs.

Highlights

• Performance Optimization for Vision Tensors: Previously, the pixel_values_flat method performed a linear-time deep copy of large vision tensors (e.g., ~5.4 MB for a 4-image batch) on every call. This caused significant memory pressure and CPU overhead on the hot path, leading to millisecond-scale latencies for a simple data access operation.
• Zero-Copy Access Implementation: The pixel_values_flat method in src/multimodal/vision/image_processor.rs has been updated to return std::borrow::Cow<'_, [f32]> instead of Vec<f32>. It now leverages ndarray::as_slice() to return a borrowed slice (Cow::Borrowed) if the tensor is contiguous, thereby eliminating redundant allocations and copies.
• Significant Performance Gains: Benchmarking validates a ~222,000x speedup in mean latency (from 5.6564 ms to 25.456 ns) and a 100% reduction in memory allocation (from ~5.4 MB/call to 0 MB) for vision tensor access, achieving zero-copy execution.
• Backward Compatibility: Because Cow<[f32]> implements Deref, all existing call sites in vision processors (e.g., qwen2_vl.rs, qwen3_vl.rs, phi4_vision.rs, llama4_vision.rs) and integration tests continue to function without modification.
• New Benchmark Added: A new benchmark, vision_access_bench, has been added to Cargo.toml and implemented in benches/vision_access_bench.rs to specifically measure and validate the performance improvements of vision tensor access.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

@slin1237 I haven’t touched schedule_batch.py in this PR.
The lint failure appears to be related to this file, as the issue is also appears in newly opened PRs

fixed