[diffusion] profiling: add bench_serving.py and VBench#15410
Conversation
Summary of ChangesHello @mickqian, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request introduces a dedicated benchmarking tool, Highlights
🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console. Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
There was a problem hiding this comment.
Code Review
This pull request introduces a new benchmarking script, bench_serving.py, for diffusion models, which is a valuable addition for performance testing. The script is well-structured, leveraging asyncio for concurrent requests and providing a clear class-based approach for handling different datasets. My review focuses on improving the script's robustness, maintainability, and correctness. I've identified a critical resource leak due to an unclosed file handle and suggest a fix. Other recommendations include making argument handling more consistent, improving error handling by catching more specific exceptions, and refactoring parts of the code to enhance clarity and reduce duplication. I've also pointed out some minor issues like misplaced imports and the side effect of creating temporary files in the current working directory.
| if self.args.task == "t2v": | ||
| return self._load_t2v_prompts() | ||
| elif self.args.task == "i2v": | ||
| return self._load_i2v_data() | ||
| elif self.args.task in ["ti2v", "ti2i"]: | ||
| return self._load_i2v_data() # Reuse logic for now | ||
| else: | ||
| # Default to T2V if task not specified or unknown | ||
| return self._load_t2v_prompts() |
There was a problem hiding this comment.
The conditional logic for loading data can be simplified. The tasks "i2v", "ti2v", and "ti2i" all use _load_i2v_data, while "t2v" and the default case use _load_t2v_prompts. This can be expressed more concisely to improve readability and maintainability.
if self.args.task in ["i2v", "ti2v", "ti2i"]:
return self._load_i2v_data()
else: # "t2v" or default
return self._load_t2v_prompts()| dummy_image = "dummy_image.jpg" | ||
| if not os.path.exists(dummy_image): | ||
| # Create a blank dummy image for testing | ||
| try: | ||
| from PIL import Image | ||
|
|
||
| img = Image.new("RGB", (100, 100), color="red") | ||
| img.save(dummy_image) | ||
| print(f"Created dummy image at {dummy_image}") | ||
| except ImportError: | ||
| print("PIL not installed, cannot create dummy image.") |
There was a problem hiding this comment.
The script creates a dummy_image.jpg in the current working directory. This can clutter the user's workspace and is not a clean practice. It would be better to create this temporary file in a dedicated temporary directory (using the tempfile module) or within the ~/.cache/sglang directory that is already being used for other cached data.
| self.args = args | ||
| self.api_url = api_url | ||
| self.model = model | ||
| self.num_prompts = args.num_prompts or 100 |
There was a problem hiding this comment.
The use of or 100 causes inconsistent behavior when --num-prompts 0 is specified. For VBenchDataset, it correctly uses all available prompts, but for RandomDataset, it defaults to 100 prompts instead of 0. The default value for num_prompts is already handled by argparse, so the or 100 fallback is unnecessary and can lead to this unexpected behavior.
self.num_prompts = args.num_prompts* 'main' of https://github.com/sgl-project/sglang: (136 commits) fix: unreachable error check in retraction (sgl-project#15433) [sgl-kernel] chore: update deepgemm version (sgl-project#13402) [diffusion] multi-platform: support diffusion on amd and fix encoder loading on MI325 (sgl-project#13760) [amd] Add deterministic all-reduce kernel for AMD (ROCm) (sgl-project#15340) [diffusion] refactor: refactor _build_req_from_sampling to use shallow_asdict (sgl-project#13782) Add customized sampler registration (sgl-project#15423) Update readme (sgl-project#15425) Fix Mindspore model import warning (sgl-project#15287) [Feature] Xiaomi `MiMo-V2-Flash` day0 support (sgl-project#15207) [diffusion] profiling: add bench_serving.py and VBench (sgl-project#15410) [DLLM] Fix dLLM regression (sgl-project#15371) [Deepseek V3.2] Fix Deepseek MTP in V1 mode (sgl-project#15429) chore: update CI_PERMISSIONS (sgl-project#15431) [DLLM] Add CI for diffusion LLMs (sgl-project#14723) Support using different attention backend for draft decoding. (sgl-project#14843) feat(dsv32): better error handling for DeepSeek-v3.2 encoder (sgl-project#14353) tiny fix lint on main (sgl-project#15424) multimodal: precompute hash for MultimodalDataItem (sgl-project#14354) [AMD] Clear pre-built AITER kernels and warmup to prevent segfaults and test timeouts (sgl-project#15318) [Performance] optimize NSA backend metadata computation for multi-step speculative decoding (sgl-project#14781) ...
Motivation
Modifications
Accuracy Tests
Benchmarking and Profiling
Checklist