Motivation

The multimodal diffusion component-accuracy jobs were temporarily wired with explicit workflow-side pytest / torchrun commands so they could pass CI correctly.

That fixed the immediate CI failures, but it left these jobs outside the normal multimodal runner path. The follow-up goal here is to make python/sglang/multimodal_gen/test/run_suite.py the only entrypoint again, while preserving the execution behavior that component-accuracy actually needs:

• 1-GPU accuracy must keep file/process isolation
• 2-GPU accuracy must launch with distributed world size 2
• existing multimodal server/unit suites should remain unchanged

This PR brings component-accuracy back under run_suite.py without broadening the runner design for unrelated suites.

Modifications

1. Add component-accuracy suites to run_suite.py

New suites:

• component-accuracy-1-gpu
• component-accuracy-2-gpu

2. Add a narrow accuracy-only execution branch in run_suite.py

The default multimodal runner path is preserved for existing suites:

• collect test items
• partition by item
• run plain pytest

For the two component-accuracy suites only:

• partition by file
• run each file separately
• launch 2-GPU accuracy with:
  • python -m torch.distributed.run --nproc_per_node=2 -m pytest

This keeps the behavior that was required for correctness:

• 1-GPU accuracy keeps file-level process isolation
• 2-GPU accuracy gets the distributed launcher it needs

3. Move multimodal component-accuracy jobs back to run_suite.py

Updated:

• .github/workflows/pr-test-multimodal-gen.yml

The workflow no longer invokes explicit file-level pytest / torchrun commands for component accuracy.

Instead, both jobs call:

• python3 sglang/multimodal_gen/test/run_suite.py --suite component-accuracy-1-gpu ...
• python3 sglang/multimodal_gen/test/run_suite.py --suite component-accuracy-2-gpu ...

with partitions [0, 1], so the runner remains the single entrypoint and autopartition is preserved.

High-Level Flow

flowchart TD
    A[pr-test-multimodal-gen.yml] --> B[run_suite.py]

    B --> C{Suite type}

    C -->|unit / 1-gpu / 2-gpu| D[Collect pytest items]
    D --> E[Partition by item]
    E --> F[Run plain pytest]

    C -->|component-accuracy-1-gpu| G[Resolve suite files]
    G --> H[Partition by file]
    H --> I[Run one file per pytest process]

    C -->|component-accuracy-2-gpu| J[Resolve suite files]
    J --> K[Partition by file]
    K --> L[Run one file per distributed pytest process]
    L --> M[python -m torch.distributed.run --nproc_per_node=2 -m pytest]

Accuracy Tests

Speed Tests and Profiling

Checklist

• Format your code according to the Format code with pre-commit.
• Add unit tests according to the Run and add unit tests.
• Update documentation according to Write documentations.
• Provide accuracy and speed benchmark results according to Test the accuracy and Benchmark the speed.
• Follow the SGLang code style guidance.

Review and Merge Process

1. Ping Merge Oncalls to start the process. See the PR Merge Process.
2. Get approvals from CODEOWNERS and other reviewers.
3. Trigger CI tests with comments or contact authorized users to do so.
   • Common commands include /tag-and-rerun-ci, /tag-run-ci-label, /rerun-failed-ci
4. After green CI and required approvals, ask Merge Oncalls or people with Write permission to merge the PR.