fix(llm): pass base_url to default provider clients from LLMSettings by shellybotmoyer · Pull Request #643 · plastic-labs/honcho

shellybotmoyer · 2026-05-03T08:06:55Z

Summary

Fixes #641 — when self-hosting Honcho with an OpenAI-compatible provider (OpenRouter, vLLM, Together, Anyscale, etc.), every LLM call fails with 401 because the default AsyncOpenAI client always hits https://api.openai.com/v1 instead of the configured proxy.

Root Cause

LLMSettings has OPENAI_API_KEY but no OPENAI_BASE_URL field (same for Anthropic and Gemini). The default client construction in registry.py never passes base_url:

# Before — always hits api.openai.com
CLIENTS["openai"] = AsyncOpenAI(
    api_key=settings.LLM.OPENAI_API_KEY,
)

The per-service override path (MODEL_CONFIG__OVERRIDES__BASE_URL) works correctly, but operators who set LLM_OPENAI_BASE_URL in .env expect the default client to use it too.

Changes

src/config.py: Add OPENAI_BASE_URL, ANTHROPIC_BASE_URL, GEMINI_BASE_URL to LLMSettings (env vars: LLM_OPENAI_BASE_URL, LLM_ANTHROPIC_BASE_URL, LLM_GEMINI_BASE_URL).
src/llm/registry.py: Pass base_url from settings to all three default client constructors (CLIENTS dict entries and get_*_client() helpers).

Backward Compatibility

All three *_BASE_URL fields default to None, so existing deployments that don't set them are unaffected. The AsyncOpenAI(api_key=..., base_url=None) behavior is identical to AsyncOpenAI(api_key=...) — the SDK uses its default endpoint.

Test Plan

Set LLM_OPENAI_BASE_URL=https://openrouter.ai/api/v1 and LLM_OPENAI_API_KEY=sk-or-v1-... in .env
Start Honcho and call any dialectic/deriver/summary endpoint
Verify requests go to openrouter.ai instead of api.openai.com

Summary by CodeRabbit

New Features
- Configure custom base URLs for multiple LLM providers (Anthropic, OpenAI-compatible services, Gemini) via configuration files or environment variables.
- Runtime connections to LLM providers will honor these configured endpoints, enabling use of self-hosted or alternative provider deployments.

Fixes plastic-labs#641

coderabbitai · 2026-05-03T08:07:09Z

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 2a7362c6-89de-452d-bafc-a851e64e43ab

📥 Commits

Reviewing files that changed from the base of the PR and between cc35240 and 86c4200.

📒 Files selected for processing (1)

src/llm/registry.py

🚧 Files skipped from review as they are similar to previous changes (1)

src/llm/registry.py

Walkthrough

The PR adds optional base URL configuration fields (ANTHROPIC_BASE_URL, OPENAI_BASE_URL, GEMINI_BASE_URL) to LLMSettings, then wires them into default client construction in the provider registry so clients use configured endpoints instead of hardcoded defaults.

Changes

Provider Base URL Configuration

Layer / File(s)	Summary
Configuration Schema `src/config.py`	Three optional base URL fields (`ANTHROPIC_BASE_URL`, `OPENAI_BASE_URL`, `GEMINI_BASE_URL`) added to `LLMSettings` to support custom provider endpoints (e.g., OpenRouter, vLLM).
Client Factory & Registry `src/llm/registry.py`	`get_anthropic_client()`, `get_openai_client()`, and `get_gemini_client()` updated to pass `base_url` from settings. Module-level `CLIENTS` registry entries for each provider now instantiate with their respective configured base URLs (conditional `HttpOptions` for Gemini).

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~8 minutes

Poem

🐰 With URLs now configurable and free,
OpenRouter, vLLM—all providers decree:
No more hardcoded endpoints that mislead,
Each base path now flows where it's led! 🌐✨

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title accurately summarizes the main change: adding base_url parameter passing to default provider clients from LLMSettings.
Linked Issues check	✅ Passed	The PR implements the primary coding objective from issue `#641`: adding BASE_URL fields to LLMSettings and passing them to default provider clients.
Out of Scope Changes check	✅ Passed	All changes are directly scoped to the primary fix: adding BASE_URL configuration fields and passing them to client constructors.
Docstring Coverage	✅ Passed	Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create PR with unit tests

Tip

💬 Introducing Slack Agent: The best way for teams to turn conversations into code.

Slack Agent is built on CodeRabbit's deep understanding of your code, so your team can collaborate across the entire SDLC without losing context.

Generate code and open pull requests
Plan features and break down work
Investigate incidents and troubleshoot customer tickets together
Automate recurring tasks and respond to alerts with triggers
Summarize progress and report instantly

Built for teams:

Shared memory across your entire org—no repeating context
Per-thread sandboxes to safely plan and execute work
Governance built-in—scoped access, auditability, and budget controls

One agent for your entire SDLC. Right inside Slack.

👉 Get started

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

coderabbitai

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@src/llm/registry.py`:
- Around line 111-120: The startup will raise an AttributeError because the code
instantiates genai.client.Client instead of the public genai.Client; update the
instantiation used when settings.LLM.GEMINI_API_KEY is present so
CLIENTS["gemini"] is created with genai.Client(...), preserving the existing
http_options and api_key args (refer to settings.LLM.GEMINI_API_KEY,
settings.LLM.GEMINI_BASE_URL, http_options, and CLIENTS["gemini"] in the block)
to match the SDK and other usages in this file.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

Push a commit to this branch (recommended)
Create a new PR with the fixes

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 2d4b1380-a38a-40d3-ad14-58da59b8a470

📥 Commits

Reviewing files that changed from the base of the PR and between f37338b and cc35240.

📒 Files selected for processing (2)

src/config.py
src/llm/registry.py

Sync fork with upstream through 9f26fdd (v3.0.2 → v3.0.9). No conflicts; local CLAUDE.md (GitNexus block) auto-merged. Notable: base_url threading to LLM clients (plastic-labs#643), configurable embeddings (plastic-labs#678), deriver custom instructions (plastic-labs#609). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

fix(llm): pass base_url to default provider clients from LLMSettings

cc35240

Fixes plastic-labs#641

coderabbitai Bot reviewed May 3, 2026

View reviewed changes

Comment thread src/llm/registry.py

solifidist mentioned this pull request May 8, 2026

Self-hosted Honcho with OpenAI-compatible providers (OpenRouter / vLLM): default AsyncOpenAI client lacks base_url → 401 against api.openai.com #641

Closed

shellybotmoyer mentioned this pull request May 12, 2026

fix(config,llm): add default base_url support for OpenAI, Anthropic, Gemini clients #665

Closed

fix(registry): use public genai.Client instead of genai.client.Client

86c4200

VVoruganti approved these changes May 14, 2026

View reviewed changes

VVoruganti merged commit b3f371b into plastic-labs:main May 14, 2026
4 checks passed

coderabbitai Bot mentioned this pull request May 15, 2026

feat: LLM model fallback on failure — fast failover to secondary provider/model #692

Open

coderabbitai Bot mentioned this pull request May 29, 2026

feat: LLM model fallback on failure — fast failover to secondary provider/model #736

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix(llm): pass base_url to default provider clients from LLMSettings#643

fix(llm): pass base_url to default provider clients from LLMSettings#643
VVoruganti merged 2 commits into
plastic-labs:mainfrom
shellybotmoyer:fix/llm-default-client-base-url

shellybotmoyer commented May 3, 2026 •

edited by coderabbitai Bot

Loading

Uh oh!

coderabbitai Bot commented May 3, 2026 •

edited

Loading

Uh oh!

coderabbitai Bot left a comment

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

shellybotmoyer commented May 3, 2026 • edited by coderabbitai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Root Cause

Changes

Backward Compatibility

Test Plan

Summary by CodeRabbit

Uh oh!

coderabbitai Bot commented May 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Walkthrough

Changes

Estimated code review effort

Poem

Uh oh!

coderabbitai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

shellybotmoyer commented May 3, 2026 •

edited by coderabbitai Bot

Loading

coderabbitai Bot commented May 3, 2026 •

edited

Loading