lib/llm/src/preprocessor.rs (1)

1020-1022: ⚡ Quick win

Move environment variable read to startup.

The DYN_MM_ROUTING_BACKEND environment variable is read and parsed on every call to gather_mm_exact_routing_info (i.e., once per multimodal request). For high-throughput workloads, this is wasteful. Consider reading the env var once in OpenAIPreprocessor::new_with_parts and storing the resolved mode (e.g., mm_routing_backend: MmRoutingBackend enum or bool flag) as a field on OpenAIPreprocessor. This would reduce per-request overhead to a single field read.

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@lib/llm/src/preprocessor.rs` around lines 1020 - 1022, The code currently
reads DYN_MM_ROUTING_BACKEND inside gather_mm_exact_routing_info on every
request (via sglang_pad_value_mode), causing unnecessary per-request env access;
move this logic into OpenAIPreprocessor::new_with_parts by resolving the env
once (e.g., parse into a MmRoutingBackend enum or a bool field named
mm_routing_backend or similar) and store it as a field on OpenAIPreprocessor;
update gather_mm_exact_routing_info to read that field instead of calling
std::env::var, and ensure constructors set the new field when instantiating
OpenAIPreprocessor.

components/src/dynamo/sglang/request_handlers/llm/decode_handler.py (1)

236-253: ⚡ Quick win

Consider validating list element types.

The method returns Optional[List[str]] but doesn't verify that mm_hashes list elements are actually strings. If the list contains non-string items (e.g., [1, 2, 3]), they will be forwarded to SGLang and may cause a runtime error downstream. Per Python guidelines, prefer failing fast.

🛡️ Proposed element-type guard

     mm_hashes = extra_args.get("mm_hashes")
     if not mm_hashes:
         return None
     if not isinstance(mm_hashes, list):
         return None
+    if not all(isinstance(h, str) for h in mm_hashes):
+        return None
     return mm_hashes

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@components/src/dynamo/sglang/request_handlers/llm/decode_handler.py` around
lines 236 - 253, The _extract_mm_hashes function currently returns a list
without checking element types; update _extract_mm_hashes to validate that
mm_hashes is a list of str by using an element-type guard (e.g.,
all(isinstance(h, str) for h in mm_hashes)) and if any element is not a string
fail fast by raising a TypeError (include a clear message referencing mm_hashes
and the offending values) so downstream SGLang callers never receive non-string
hashes.

examples/backends/sglang/launch/agg_multimodal_router.sh (1)

65-68: 💤 Low value

Consider using print_launch_banner for consistency.

The script prints a custom banner, but the bash launch guidelines recommend calling print_launch_banner (from launch_utils.sh) with script description, model, and port to ensure consistent, debuggable startup logs across all launch scripts.

📋 Example usage of print_launch_banner

+print_launch_banner \
+    "Lightseek MM Exact Routing (SGLang)" \
+    "${MODEL}" \
+    "${HTTP_PORT}"
+echo "NUM_WORKERS=${NUM_WORKERS}, BLOCK_SIZE=${BLOCK_SIZE}"
+echo "NAMESPACE=${NAMESPACE}"
-echo "=== Lightseek MM Exact Routing Launch (SGLang) ==="
-echo "MODEL=${MODEL}"
-echo "NUM_WORKERS=${NUM_WORKERS}, BLOCK_SIZE=${BLOCK_SIZE}"
-echo "HTTP_PORT=${HTTP_PORT}, NAMESPACE=${NAMESPACE}"

As per coding guidelines: "Call print_launch_banner with script description, model, and port to ensure consistent, debuggable startup logs."

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@examples/backends/sglang/launch/agg_multimodal_router.sh` around lines 65 -
68, Replace the custom echo banner with a call to print_launch_banner to follow
launch guidelines: call print_launch_banner with a concise script description
(e.g., "Lightseek MM Exact Routing (SGLang)"), pass MODEL and HTTP_PORT as the
required model and port arguments, and keep printing any extra runtime vars like
NUM_WORKERS and BLOCK_SIZE separately if needed; update the section containing
the current echo lines to invoke print_launch_banner (referencing the
print_launch_banner function) and remove the redundant echo banner to maintain
consistent startup logs.

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes