Skip to content

[model-gateway] Migrate Worker trait to model-aware methods#14250

Merged
slin1237 merged 3 commits intomainfrom
oai-router-n/5
Dec 1, 2025
Merged

[model-gateway] Migrate Worker trait to model-aware methods#14250
slin1237 merged 3 commits intomainfrom
oai-router-n/5

Conversation

@slin1237
Copy link
Copy Markdown
Collaborator

@slin1237 slin1237 commented Dec 1, 2025

Summary

Completes the Worker trait migration to model-aware methods, building on the ModelCard/ModelType foundation.

Key changes:

  • Worker trait methods now require a model_id parameter for model-specific config lookup
  • Model configuration (tokenizer_path, reasoning_parser, tool_parser, chat_template) is now stored in ModelCard objects instead of the labels HashMap
  • Worker registration creates proper ModelCard objects from config

Motivation

Previously, model-related configuration was scattered in the labels HashMap as untyped strings. This made it:

  • Hard to discover what fields existed
  • Easy to have typos in field names
  • Impossible to have per-model configuration on multi-model workers

Now model config is properly typed in ModelCard with IDE autocomplete and compile-time checking.

Changes

  1. Worker trait methods - Changed signature from fn tokenizer_path(&self) to fn tokenizer_path(&self, model_id: &str) (same for reasoning_parser, tool_parser, chat_template)

  2. WorkerBuilder - Added models(Vec<ModelCard>) and model(ModelCard) builder methods

  3. CreateWorkerStep - Creates ModelCard from WorkerConfigRequest instead of storing in labels

Test plan

  • All 531 existing tests pass
  • New tests for model-aware method behavior
  • Verified worker_to_info() returns correct model config

@gemini-code-assist
Copy link
Copy Markdown
Contributor

Warning

You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again!

@slin1237 slin1237 added the run-ci label Dec 1, 2025
@slin1237 slin1237 merged commit 57f933f into main Dec 1, 2025
64 checks passed
@slin1237 slin1237 deleted the oai-router-n/5 branch December 1, 2025 22:32
harvenstar pushed a commit to harvenstar/sglang that referenced this pull request Dec 4, 2025
tonyluj pushed a commit to openanolis/sglang that referenced this pull request Dec 5, 2025
yuchengz816-bot pushed a commit to yuchengz816-bot/sglang that referenced this pull request Dec 8, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant