Skip to content

max_tokens config from custom_providers is not passed to AIAgent #20004

@chengoak

Description

@chengoak

Describe the bug

The max_tokens configuration set in custom_providers is not being properly passed to the AIAgent instance. Instead, the gateway always uses model.max_tokens from the global config, or falls back to provider-specific defaults (like 32k for Kimi).

To Reproduce

  1. Set max_tokens in a custom_provider entry, e.g.:
    custom_providers:
    - name: ark
      max_tokens: 131072
      # ...
  2. Remove or don't set model.max_tokens globally
  3. Start a conversation through the gateway
  4. Observe that max_tokens uses the provider default instead of 131072

Expected behavior
The max_tokens value from custom_providers should be used, with fallback to model.max_tokens if provider-specific config is not present.

Fix
This is fixed by PR #19991 which:

  1. Adds max_tokens to _KNOWN_KEYS in config normalization
  2. Passes max_tokens through runtime provider resolution
  3. Updates gateway to read max_tokens from runtime config first, then fallback to model config

Relates to #19991

Metadata

Metadata

Assignees

No one assigned

    Labels

    P2Medium — degraded but workaround existsarea/configConfig system, migrations, profilescomp/gatewayGateway runner, session dispatch, deliverytype/bugSomething isn't working

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions