Skip to content

Add Mistral AI provider to the Inference effect #413

@aallan

Description

@aallan

Mistral's API is OpenAI-compatible, making it a straightforward fourth provider for Inference.complete. With four providers (three structurally identical OpenAI-compatible endpoints), this is also the right moment to refactor _call_inference_provider() to a table-driven registry rather than bolting on another elif branch and deferring the cleanup to a fifth.

Background

The _call_inference_provider() helper in vera/codegen/api.py currently has three elif branches. The Moonshot branch is the OpenAI branch with a different URL, key, and default model. Mistral is the same shape again:

  • Endpoint: https://api.mistral.ai/v1/chat/completions
  • Auth: Bearer token (identical pattern)
  • Response: choices[0].message.content (identical to OpenAI/Moonshot)
  • Env var: VERA_MISTRAL_API_KEY
  • Default model: mistral-small-latest (consistent with the cheap/fast defaults: Haiku, GPT-4o-mini, kimi-k2-0905-preview)

At four providers — three of which are structurally identical — the elif chain wants to become a table. At five or six it becomes actively painful to extend.

Proposed design

Replace the elif chain and the per-key function parameters with a _ProviderConfig dataclass and a _PROVIDERS registry dict:

@dataclass(frozen=True)
class _ProviderConfig:
    env_key: str        # e.g. "VERA_MISTRAL_API_KEY"
    url: str            # chat completions endpoint
    default_model: str  # cheap/fast default
    auth_style: str     # "anthropic" | "bearer"
    response_style: str # "anthropic" | "openai"

_PROVIDERS: dict[str, _ProviderConfig] = {
    "anthropic": _ProviderConfig(
        env_key="VERA_ANTHROPIC_API_KEY",
        url="https://api.anthropic.com/v1/messages",
        default_model="claude-haiku-4-5-20251001",
        auth_style="anthropic",
        response_style="anthropic",
    ),
    "openai": _ProviderConfig(
        env_key="VERA_OPENAI_API_KEY",
        url="https://api.openai.com/v1/chat/completions",
        default_model="gpt-4o-mini",
        auth_style="bearer",
        response_style="openai",
    ),
    "moonshot": _ProviderConfig(
        env_key="VERA_MOONSHOT_API_KEY",
        url="https://api.moonshot.ai/v1/chat/completions",
        default_model="kimi-k2-0905-preview",
        auth_style="bearer",
        response_style="openai",
    ),
    "mistral": _ProviderConfig(
        env_key="VERA_MISTRAL_API_KEY",
        url="https://api.mistral.ai/v1/chat/completions",
        default_model="mistral-small-latest",
        auth_style="bearer",
        response_style="openai",
    ),
}

_call_inference_provider() becomes a generic dispatcher (~20 lines) that looks up the config and delegates to one of two request builders: _call_anthropic_style() or _call_openai_compat_style().

Auto-detection in execute() simplifies to iterating _PROVIDERS in priority order and checking which env_key is set — no more per-key parameters on _call_inference_provider().

Adding provider five (xAI Grok, tracked in #425) becomes a one-row addition to _PROVIDERS.

Change list

vera/codegen/api.py

  • Add _ProviderConfig dataclass and _PROVIDERS registry dict
  • Replace _call_inference_provider(provider, prompt, model, anthropic_key, openai_key, moonshot_key) with _call_inference_provider(provider, prompt, model, env) that looks up the provider's key from env via _PROVIDERS
  • Extract _call_anthropic_style() and _call_openai_compat_style() helpers
  • Simplify auto-detection loop: iterate _PROVIDERS, pick first with a set key
  • Update "no provider configured" error to enumerate all _PROVIDERS keys

Tests — tests/test_codegen.py

  • test_inference_mistral_auto_detect — copy the Moonshot auto-detect test, change key name and assertion
  • test_mistral_provider — unit test for the Mistral config entry in _PROVIDERS

Documentation sweep

  • spec/09-standard-library.md — one row in the provider table
  • SKILL.md — update the Inference effect provider table
  • CHANGELOG.md — entry noting refactor + Mistral addition
  • README.md — update the env var table / provider list

Out of scope

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestintegrationIntegration with external tools and languages

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions