Problem or Use Case
When using local LLM providers such as OLLAMA or LM Studio, having a hardcoded limit of 900 seconds is not enough. This defeats the purpose of the local provider, and force us to use small models that have very limited capabilities. Currently, the timeout field for the OPENAI provider is hardcoded to 900 seconds. Add an OPEN_AI_LLM_TIMEOUT environment variable, to set the timeout field for each LLM call.
Proposed Solution
Add an OPEN_AI_LLM_TIMEOUT , in seconds. This is consistent to the other variables such as URL, KEY, model ID.
Alternatives Considered
Add a configuration in the config.yaml
Feature Type
Configuration option
Scope
Small (single file, < 50 lines)
Contribution
Problem or Use Case
When using local LLM providers such as OLLAMA or LM Studio, having a hardcoded limit of 900 seconds is not enough. This defeats the purpose of the local provider, and force us to use small models that have very limited capabilities. Currently, the timeout field for the OPENAI provider is hardcoded to 900 seconds. Add an OPEN_AI_LLM_TIMEOUT environment variable, to set the timeout field for each LLM call.
Proposed Solution
Add an OPEN_AI_LLM_TIMEOUT , in seconds. This is consistent to the other variables such as URL, KEY, model ID.
Alternatives Considered
Add a configuration in the config.yaml
Feature Type
Configuration option
Scope
Small (single file, < 50 lines)
Contribution