Conversation
WalkthroughThis PR updates docs and configuration: backend identifier renames/additions (e.g., lmstudio → lm-studio, add openai-compatible), ANTHROPIC_API_BASE_URL → ANTHROPIC_BASE_URL, health-check state and recovery thresholds, and several configuration default values and guidance (merge-based resolution, timeouts, limits). Changes
Sequence Diagram(s)sequenceDiagram
autonumber
%% Components
participant Client as Client
participant Router as Router
participant Health as HealthChecker
participant CB as CircuitBreaker
participant Backend as Backend
rect rgba(200,200,255,0.5)
Client->>Router: Request arrives
Router->>CB: Check backend health state
CB->>Health: Query latest health status
Health-->>CB: State (Unhealthy / Busy / Warming / Healthy)
alt Unhealthy
CB-->>Router: Route? No (Open)
Router->>Client: Return routable error / fallback
else Half-Open
Router->>Backend: Send probe request
Backend-->>Router: Probe response (success/fail)
alt Success (1 required)
Router->>CB: Report success
CB-->>Router: Transition to Closed (healthy)
Router->>Client: Route normal traffic to Backend
else Fail
Router->>CB: Report failure
CB-->>Router: Remain Open
Router->>Client: Return routable error / fallback
end
else Healthy
Router->>Backend: Route request normally
Backend-->>Router: Response
Router-->>Client: Response
end
end
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~25 minutes Possibly related PRs
🚥 Pre-merge checks | ✅ 2 | ❌ 1❌ Failed checks (1 inconclusive)
✅ Passed checks (2 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Actionable comments posted: 3
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
docs/content/integrations/frontend/claude-code.md (1)
368-370:⚠️ Potential issue | 🟡 MinorUpdate
type: lmstudiototype: lm-studioLine 370 uses the stale identifier
lmstudio. The authoritative backend documentation (docs/content/integrations/backend/lmstudio.md) and modern integration examples consistently uselm-studio. This should be updated for consistency.
🤖 Fix all issues with AI agents
In `@docs/content/index.md`:
- Line 98: The docs show a "LiteLLM: Native Support" badge but `litellm` is not
listed in the X-Olla-Backend-Type values and has no implementation; remove the
LiteLLM badge and any occurrences of "LiteLLM" from the docs so the README
matches implemented backends (leave the X-Olla-Backend-Type list as
`ollama/openai/openai-compatible/lm-studio/llamacpp/vllm/sglang/lemonade`), or
alternatively if you prefer to add support, add `litellm` to the
X-Olla-Backend-Type list and implement the backend in code (create the litellm
backend module and register it where backends are discovered); choose one
approach and make the docs and code consistent with that choice.
In `@docs/content/integrations/api-translation/anthropic.md`:
- Around line 559-562: The docs show
ANTHROPIC_BASE_URL="http://localhost:40114/olla/anthropic" but don't explain
that the Claude Code client auto-appends "/v1", causing an inconsistency with
other clients that include "/v1" explicitly; update the anthropic.md content
near the ANTHROPIC_BASE_URL example to add a brief note stating that Claude Code
automatically appends "/v1" to the base URL (so the example without "/v1" is
correct) and call out that OpenCode/Crush CLI use baseURL/base_url that must
include "/v1" to avoid confusion (reference ANTHROPIC_BASE_URL and the Claude
Code client behavior).
In `@docs/content/integrations/frontend/claude-code.md`:
- Around line 74-75: The README/docs and expected-output use ANTHROPIC_BASE_URL
without a trailing /v1, but the test scripts define ANTHROPIC_URL as
${OLLA_URL}/olla/anthropic/v1 causing a mismatch; pick the canonical form
(prefer keeping ANTHROPIC_BASE_URL as ${OLLA_URL}/olla/anthropic) and make tests
match: update the test scripts that set ANTHROPIC_URL and ANTHROPIC_BASE_URL
(symbols: ANTHROPIC_URL, ANTHROPIC_BASE_URL, OLLA_URL) to remove the trailing
/v1 and adjust any expected-output checks accordingly, or alternatively if the
service requires /v1 then update the docs and expected-output to include
/v1—ensure both docs and tests use the same canonical base URL everywhere.
🧹 Nitpick comments (1)
docs/content/getting-started/quickstart.md (1)
17-39: Improved configuration guidance for local development.The new configuration best practice section clearly explains the
config.local.yamlworkflow, helping users avoid committing local changes. The step-by-step commands make it easy to follow.Minor: The fenced code block at line 39 is missing a language identifier (bash). While this doesn't affect functionality, adding it improves syntax highlighting.
🎨 Add language identifier to code block
- ``` + ```bash $ cp config/config.yaml config/config.local.yaml $ vi config/config.local.yaml # local modifications for your setup ```
| | `X-Olla-Endpoint` | Backend endpoint name | | ||
| | `X-Olla-Model` | Model used for the request | | ||
| | `X-Olla-Backend-Type` | Backend type (ollama/openai/lmstudio/llamacpp/vllm/sglang/lemonade/litellm) | | ||
| | `X-Olla-Backend-Type` | Backend type (ollama/openai/openai-compatible/lm-studio/llamacpp/vllm/sglang/lemonade) | |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
# First, let's check the structure and find the docs file
find . -name "index.md" -path "*/docs/content/*" | head -5Repository: thushan/olla
Length of output: 79
🏁 Script executed:
# Read the documentation file around line 98 to see the context
if [ -f "docs/content/index.md" ]; then
sed -n '90,110p' docs/content/index.md
fiRepository: thushan/olla
Length of output: 995
🏁 Script executed:
# Also check around line 18 for the LiteLLM badge reference mentioned
if [ -f "docs/content/index.md" ]; then
sed -n '10,25p' docs/content/index.md
fiRepository: thushan/olla
Length of output: 2072
🏁 Script executed:
# Search for litellm backend type references in Go files
rg -n "litellm" --type=go -iRepository: thushan/olla
Length of output: 38
🏁 Script executed:
# Search for backend type constants or configurations
rg -n "backend.*type|Backend.*Type" --type=go -A2 | head -50Repository: thushan/olla
Length of output: 4117
🏁 Script executed:
# Check git history to see if litellm was removed recently
git log --oneline --all -- docs/content/index.md | head -10Repository: thushan/olla
Length of output: 339
Resolve inconsistency between LiteLLM badge and backend types list.
The documentation shows a "LiteLLM: Native Support" badge, but litellm does not appear in the X-Olla-Backend-Type header values on line 98 (ollama/openai/openai-compatible/lm-studio/llamacpp/vllm/sglang/lemonade). Additionally, litellm is not referenced in the Go codebase. Either add litellm back to the backend types list and implement the support, or remove the LiteLLM badge to align with the actual capabilities.
🤖 Prompt for AI Agents
In `@docs/content/index.md` at line 98, The docs show a "LiteLLM: Native Support"
badge but `litellm` is not listed in the X-Olla-Backend-Type values and has no
implementation; remove the LiteLLM badge and any occurrences of "LiteLLM" from
the docs so the README matches implemented backends (leave the
X-Olla-Backend-Type list as
`ollama/openai/openai-compatible/lm-studio/llamacpp/vllm/sglang/lemonade`), or
alternatively if you prefer to add support, add `litellm` to the
X-Olla-Backend-Type list and implement the backend in code (create the litellm
backend module and register it where backends are discovered); choose one
approach and make the docs and code consistent with that choice.
| export ANTHROPIC_BASE_URL="http://localhost:40114/olla/anthropic" | ||
| export ANTHROPIC_API_KEY="not-required" # Optional for local use | ||
| claude-code | ||
| claude | ||
| ``` |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
# Check the file around the flagged lines and search for /v1 patterns
echo "=== Content around lines 559-562 ==="
sed -n '550,575p' docs/content/integrations/api-translation/anthropic.md
echo -e "\n=== All ANTHROPIC_BASE_URL references ==="
rg -n "ANTHROPIC.*BASE.*URL" docs/content/integrations/api-translation/anthropic.md -A2 -B2
echo -e "\n=== All URL patterns with /v1 in the file ==="
rg -n "olla/anthropic/v1" docs/content/integrations/api-translation/anthropic.md -B2 -A2Repository: thushan/olla
Length of output: 3808
🏁 Script executed:
#!/bin/bash
# Check the Claude Code Integration file referenced in the document
echo "=== Checking for claude-code.md file ==="
fd "claude-code.md" docs/
echo -e "\n=== Reading Claude Code Integration file if found ==="
if [ -f "docs/content/integrations/frontend/claude-code.md" ]; then
# Look for ANTHROPIC_BASE_URL references
rg -n "ANTHROPIC_BASE_URL" docs/content/integrations/frontend/claude-code.md -B3 -A3
else
echo "File not found at expected path"
fiRepository: thushan/olla
Length of output: 1851
Clarify that Claude Code auto-appends /v1 to the base URL.
The environment variable ANTHROPIC_BASE_URL="http://localhost:40114/olla/anthropic" (without /v1) is correct for Claude Code—the client automatically appends /v1 to construct the full API path. However, other clients configured in the same document (OpenCode, Crush CLI) explicitly include /v1 in their baseURL/base_url settings (lines 577, 595). This inconsistency across clients should be documented more clearly to avoid confusion. Consider adding a note explaining that Claude Code differs from other clients in how it constructs the API endpoint.
🤖 Prompt for AI Agents
In `@docs/content/integrations/api-translation/anthropic.md` around lines 559 -
562, The docs show ANTHROPIC_BASE_URL="http://localhost:40114/olla/anthropic"
but don't explain that the Claude Code client auto-appends "/v1", causing an
inconsistency with other clients that include "/v1" explicitly; update the
anthropic.md content near the ANTHROPIC_BASE_URL example to add a brief note
stating that Claude Code automatically appends "/v1" to the base URL (so the
example without "/v1" is correct) and call out that OpenCode/Crush CLI use
baseURL/base_url that must include "/v1" to avoid confusion (reference
ANTHROPIC_BASE_URL and the Claude Code client behavior).
| Set <code>ANTHROPIC_BASE_URL</code> to Olla Anthropic endpoint <br/> | ||
| <pre>export ANTHROPIC_BASE_URL="http://localhost:40114/olla/anthropic"</pre> |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
echo "=== ANTHROPIC_BASE_URL assignments ==="
rg -n 'ANTHROPIC_BASE_URL' -C2
echo ""
echo "=== ANTHROPIC_URL assignments ==="
rg -n 'ANTHROPIC_URL=' -C2Repository: thushan/olla
Length of output: 6649
Inconsistency: ANTHROPIC_BASE_URL with vs. without /v1 across files.
The test scripts in examples/claude-code-ollama/test.sh (line 131) and examples/claude-code-llamacpp/test.sh (line 122) output ANTHROPIC_BASE_URL="${ANTHROPIC_URL}", where ANTHROPIC_URL is defined as ${OLLA_URL}/olla/anthropic/v1 (with /v1 appended). However, all documentation files and the expected output check in this file (line 569) consistently show ANTHROPIC_BASE_URL as http://localhost:40114/olla/anthropic (without /v1).
Users following the test script output will configure Claude Code with a different base URL than those following the documentation. Reconcile which value is correct — whether Claude Code automatically appends /v1 to the base URL or if the test scripts need updating.
🤖 Prompt for AI Agents
In `@docs/content/integrations/frontend/claude-code.md` around lines 74 - 75, The
README/docs and expected-output use ANTHROPIC_BASE_URL without a trailing /v1,
but the test scripts define ANTHROPIC_URL as ${OLLA_URL}/olla/anthropic/v1
causing a mismatch; pick the canonical form (prefer keeping ANTHROPIC_BASE_URL
as ${OLLA_URL}/olla/anthropic) and make tests match: update the test scripts
that set ANTHROPIC_URL and ANTHROPIC_BASE_URL (symbols: ANTHROPIC_URL,
ANTHROPIC_BASE_URL, OLLA_URL) to remove the trailing /v1 and adjust any
expected-output checks accordingly, or alternatively if the service requires /v1
then update the docs and expected-output to include /v1—ensure both docs and
tests use the same canonical base URL everywhere.
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Fix all issues with AI agents
In `@docs/content/configuration/overview.md`:
- Line 69: Update the "See Configuration Best Practices" line in
docs/content/configuration/overview.md so it no longer promises unsupported
content: either remove the phrase "and Docker Compose examples" from that
sentence, or add Docker Compose examples to the target page (the "Configuration
Best Practices" document in configuration.md) — if adding examples, include a
short Docker Compose snippet and a brief explanation under a new "Docker Compose
examples" section in configuration.md and update any table-of-contents or anchor
links as needed to keep the reference accurate.
In `@docs/content/configuration/practices/configuration.md`:
- Line 45: Replace the incorrect abbreviation "Eg." with the standard lowercase
"e.g." in the sentence inside configuration.md (the line containing "You can
override the settings you need (Eg. Endpoint or Proxy Engine) and remove
others"); update the text to read "... (e.g. Endpoint or Proxy Engine) ..." to
follow the correct "exempli gratia" formatting.
🧹 Nitpick comments (1)
docs/content/configuration/practices/configuration.md (1)
111-113: Nit: missing comma after "full control".Suggested fix
-For full control including provider profiles: +For full control, including provider profiles:
| ghcr.io/thushan/olla:latest | ||
| ``` | ||
|
|
||
| See **[Configuration Best Practices](practices/configuration.md)** for detailed Docker deployment patterns, networking options, and Docker Compose examples |
There was a problem hiding this comment.
Reference to "Docker Compose examples" appears unsupported by the linked page.
This line promises "Docker Compose examples" in the linked Configuration Best Practices page, but docs/content/configuration/practices/configuration.md does not contain any Docker Compose content. Either remove the mention or add Docker Compose examples to the target page.
🤖 Prompt for AI Agents
In `@docs/content/configuration/overview.md` at line 69, Update the "See
Configuration Best Practices" line in docs/content/configuration/overview.md so
it no longer promises unsupported content: either remove the phrase "and Docker
Compose examples" from that sentence, or add Docker Compose examples to the
target page (the "Configuration Best Practices" document in configuration.md) —
if adding examples, include a short Docker Compose snippet and a brief
explanation under a new "Docker Compose examples" section in configuration.md
and update any table-of-contents or anchor links as needed to keep the reference
accurate.
| This works because: | ||
|
|
||
| - `config.local.yaml` has higher resolution priority than `config.yaml` | ||
| - You can override the settings you need (Eg. Endpoint or Proxy Engine) and remove others |
There was a problem hiding this comment.
Minor: "Eg." should be "e.g."
The standard abbreviation for exempli gratia uses lowercase with two periods.
-- You can override the settings you need (Eg. Endpoint or Proxy Engine) and remove others
+- You can override the settings you need (e.g. endpoint or proxy engine) and remove others📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| - You can override the settings you need (Eg. Endpoint or Proxy Engine) and remove others | |
| - You can override the settings you need (e.g. endpoint or proxy engine) and remove others |
🧰 Tools
🪛 LanguageTool
[uncategorized] ~45-~45: The abbreviation “e.g.” (= for example) requires two periods.
Context: ...You can override the settings you need (Eg. Endpoint or Proxy Engine) and remove ot...
(E_G)
🤖 Prompt for AI Agents
In `@docs/content/configuration/practices/configuration.md` at line 45, Replace
the incorrect abbreviation "Eg." with the standard lowercase "e.g." in the
sentence inside configuration.md (the line containing "You can override the
settings you need (Eg. Endpoint or Proxy Engine) and remove others"); update the
text to read "... (e.g. Endpoint or Proxy Engine) ..." to follow the correct
"exempli gratia" formatting.
Documentation updates for the latest available release.
Summary by CodeRabbit
New Features
Improvements