Skip to content

Add support for multiple providers or base_url config in compression #1591

@apoapostolov

Description

@apoapostolov

Issue Type: Feature Request

Current Behavior:
When using a custom provider (like zai) for the compression model, there's no way to specify a custom base URL. The compression model currently only accepts a summary_model name, not a full provider configuration.

Problem Example:

compression:
  enabled: true
  threshold: 0.85
  summary_model: google/gemini-3-flash-preview  # Uses default provider
  summary_provider: auto

But if you want to use a custom provider like zai:

compression:
  enabled: true
  threshold: 0.85
  summary_model: glm-4.7
  summary_provider: zai

This fails if zai doesn't support Google models, and there's no way to specify a custom base_url for the compression model.

Requested Solution:
Either:

  1. Option A: Allow compression model to accept a full provider config (similar to smart_model_routing):
compression:
  enabled: true
  threshold: 0.85
  summary_model:
    provider: zai
    model: glm-4.7
    base_url: https://api.z.ai/api/coding/paas/v4
  1. Option B: Allow compression model to accept a base_url parameter:
compression:
  enabled: true
  threshold: 0.85
  summary_model: glm-4.7
  summary_provider: zai
  summary_base_url: https://api.z.ai/api/coding/paas/v4

Use Case:
Multiple providers exist (zai, DeepSeek, OpenRouter, etc.) with different models and API endpoints. Users should be able to configure custom providers for the compression model without modifying the core code.

Suggested Implementation:

  • Update the compression config schema to accept either a full model config or a base_url parameter
  • Follow the existing pattern used in smart_model_routing

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions