Problem

The STT provider registry (src/stt/providers/index.ts) is hardcoded to 5 built-in providers (OpenAI, Google, Deepgram, Groq, Mistral). Extensions cannot register custom STT providers into the core system.

The voice-call extension already implements its own OpenAIRealtimeSTTProvider but bypasses the core registry entirely — it can't participate in fallback chains, config-driven selection, or auth profile integration.

Current state

STT has a clean provider interface (src/stt/types.ts):

export type SttProvider = {
  id: string;
  transcribeAudio: (req: AudioTranscriptionRequest) => Promise<AudioTranscriptionResult>;
};

Registry already supports overrides (src/stt/providers/index.ts):

buildSttProviderRegistry(overrides?: Record<string, SttProvider>): Map<string, SttProvider>

The infrastructure is nearly there — the registry just isn't exposed to extensions.

Proposed changes

1. Export SttProvider and AudioTranscriptionRequest/Result types from plugin SDK

src/plugin-sdk/index.ts should export the STT provider types so extensions can implement them.

2. Add stt field to plugin manifest

{
  "id": "my-extension",
  "stt": ["my-custom-stt"]
}

3. Plugin STT provider registration

Extensions export a factory function that returns SttProvider instances:

export function createSttProviders(ctx: PluginContext): SttProvider[] {
  return [{ id: "my-custom-stt", transcribeAudio: async (req) => { ... } }];
}

4. Merge plugin providers into core registry

buildSttProviderRegistry already accepts overrides — wire plugin-registered providers into this mechanism. Plugin providers extend (not replace) built-in providers. Priority: config-specified > plugin > built-in.

5. Plugin SDK manifest discovery

Update src/plugins/manifest-registry.ts to parse stt field from plugin manifests. Validate provider IDs at load time.

Files to change

Design decisions

• Factory vs instance: Factory function (deferred init, config injection)
• Fallback chain: Plugin providers participate in fallback (built-in → plugin, by priority)
• Config selection: Plugin-registered providers selectable via tools.media.audio.models[].provider

Related

• feat: unified auth + multimodal AgentRuntime — implementation plan #415 — unified auth + multimodal plan
• gut(media): remove media-understanding and link-understanding pipelines #425 — media-understanding gut (after which STT is the sole transcription path)