Motivation

Controls the image resolution and image count per-req in bench_serving, with a runtime-generated dataset: random-image:

python3 -m sglang.bench_serving --backend sglang-oai-chat --dataset-name random-image --num-prompts 50 --port 9080 --random-output-len 1 --random-input-len 1024 --random-image-resolution 1080p --random-image-num-images 3

here:

1. --random-image-resolution controls the resolution of image, with presets or heightxwidth format
2. --random-image-num-images controls number of images per request

credits @yhyang201

Modifications

Accuracy Tests

Benchmarking and Profiling

Checklist

• Format your code according to the Format code with pre-commit.
• Add unit tests according to the Run and add unit tests.
• Update documentation according to Write documentations.
• Provide accuracy and speed benchmark results according to Test the accuracy and Benchmark the speed.