You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This seems to only allow setting a global OPENAI_API_BASE and OPENAI_API_KEY in .env, and then VOICE_TOOLS_OPENAI_KEY. If so, it's not flexible enough: OpenAI themselves run a single API, but locally, it's likely that users would be running one inference server for their textgen model, another for TTS, another for ASR, each on different ports. So, there should at least be a default OPENAI_API_BASE/KEY and overrides per usage type: TTS_OPENAI_API_BASE/KEY, for example. Better approaches would be:
Just add all (protocol, base_url, model) triplets. Then allow specifying the model by provider and names. This is akin to how litellm manages protocols, providers/endpoints/base_urls, and models.
Add all protocols / providers / models per Terminal tool #1. Tag them according to capabilities. Weight automatically for preference, according to some heuristic (such as local first, more capable first, or more lightweight but sufficiently capable). Allow user override of tags and weightings. Auto select the best at runtime according to weight.
Per two, but do everything up to heuristic weighting, then allow the user to override per usage type (TTS, ASR, Text gen, per agent, etc.)
This seems to only allow setting a global OPENAI_API_BASE and OPENAI_API_KEY in .env, and then VOICE_TOOLS_OPENAI_KEY. If so, it's not flexible enough: OpenAI themselves run a single API, but locally, it's likely that users would be running one inference server for their textgen model, another for TTS, another for ASR, each on different ports. So, there should at least be a default OPENAI_API_BASE/KEY and overrides per usage type: TTS_OPENAI_API_BASE/KEY, for example. Better approaches would be:
Just add all (protocol, base_url, model) triplets. Then allow specifying the model by provider and names. This is akin to how litellm manages protocols, providers/endpoints/base_urls, and models.
Add all protocols / providers / models per Terminal tool #1. Tag them according to capabilities. Weight automatically for preference, according to some heuristic (such as local first, more capable first, or more lightweight but sufficiently capable). Allow user override of tags and weightings. Auto select the best at runtime according to weight.
Per two, but do everything up to heuristic weighting, then allow the user to override per usage type (TTS, ASR, Text gen, per agent, etc.)