Skip to content

[Feature/功能]: Support rate limits #858

@PeViscendia

Description

@PeViscendia

Pre-submission Checklist / 提交前检查

  • I have searched the existing issues and this feature has not been requested / 我已搜索现有 issues,此功能尚未被提出
  • I have read the documentation / 我已阅读文档

Problem Statement / 问题描述

If a provider has rate limits for a model and the model replies that it is unavailable until x time using standard 429 response code, the model should not be attempted again until the rate limit time is over.

Proposed Solution / 期望方案

If hitting rate limits, don't try for a minimum of x seconds defined in the config, if a provider is telling you when you can access it again, use that.

Alternatives Considered / 备选方案

No response

Feature Category / 功能分类

Load Balancing / Failover / 负载均衡 / 故障切换

Additional Context / 其他补充信息

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions