2025-11-30 07:59:11 | INFO     | lmms_eval.loggers.evaluation_tracker:save_results_aggregated:239 - Output path not provided, skipping saving results aggregated
openai_compatible (model_version=Qwen/Qwen2.5-VL-7B-Instruct), gen_kwargs: (), limit: None, num_fewshot: None, batch_size: 16
| Tasks  |Version|Filter|n-shot| Metric |   |Value |   |Stderr|
|--------|------:|------|-----:|--------|---|-----:|---|------|
|mmmu_val|      0|none  |     0|mmmu_acc|↑  |0.5067|±  |   N/A|

============ Serving Benchmark Result ============
Backend:                                 sglang-oai-chat
Traffic request rate:                    inf       
Max request concurrency:                 32        
Successful requests:                     256       
Benchmark duration (s):                  9.05      
Total input tokens:                      126333    
Total input text tokens:                 23421     
Total input vision tokens:               102912    
Total generated tokens:                  4541      
Total generated tokens (retokenized):    4539      
Request throughput (req/s):              28.30     
Input token throughput (tok/s):          13965.55  
Output token throughput (tok/s):         501.99    
Total token throughput (tok/s):          14467.54  
Concurrency:                             31.45     
----------------End-to-End Latency----------------
Mean E2E Latency (ms):                   1111.34   
Median E2E Latency (ms):                 1111.86   
---------------Time to First Token----------------
Mean TTFT (ms):                          351.08    
Median TTFT (ms):                        288.96    
P99 TTFT (ms):                           1069.64   
-----Time per Output Token (excl. 1st token)------
Mean TPOT (ms):                          42.58     
Median TPOT (ms):                        45.67     
P99 TPOT (ms):                           74.79     
---------------Inter-Token Latency----------------
Mean ITL (ms):                           45.45     
Median ITL (ms):                         11.53     
P95 ITL (ms):                            240.69    
P99 ITL (ms):                            314.75    
Max ITL (ms):                            794.27    
==================================================

============ Serving Benchmark Result ============
Backend:                                 sglang-oai-chat
Traffic request rate:                    inf       
Max request concurrency:                 32        
Successful requests:                     256       
Benchmark duration (s):                  9.20      
Total input tokens:                      126345    
Total input text tokens:                 23433     
Total input vision tokens:               102912    
Total generated tokens:                  4541      
Total generated tokens (retokenized):    4541      
Request throughput (req/s):              27.83     
Input token throughput (tok/s):          13733.83  
Output token throughput (tok/s):         493.61    
Total token throughput (tok/s):          14227.44  
Concurrency:                             31.13     
----------------End-to-End Latency----------------
Mean E2E Latency (ms):                   1118.60   
Median E2E Latency (ms):                 1097.04   
---------------Time to First Token----------------
Mean TTFT (ms):                          362.51    
Median TTFT (ms):                        292.88    
P99 TTFT (ms):                           1055.66   
-----Time per Output Token (excl. 1st token)------
Mean TPOT (ms):                          42.40     
Median TPOT (ms):                        44.79     
P99 TPOT (ms):                           84.21     
---------------Inter-Token Latency----------------
Mean ITL (ms):                           45.17     
Median ITL (ms):                         10.54     
P95 ITL (ms):                            255.83    
P99 ITL (ms):                            361.42    
Max ITL (ms):                            788.53    
==================================================