Conversation
adds examples for ollama-openwebui stack
WalkthroughThis update revises documentation and example configurations to standardise endpoint definitions, replacing the Changes
Sequence Diagram(s)sequenceDiagram
participant User
participant OpenWebUI
participant Olla
participant Ollama1
participant Ollama2
User->>OpenWebUI: Initiate request (e.g., chat)
OpenWebUI->>Olla: API request (model inference)
Olla->>Ollama1: Forward request (based on load balancing/health)
Ollama1-->>Olla: Model response
Olla-->>OpenWebUI: Aggregated/proxied response
OpenWebUI-->>User: Display result
Estimated code review effort🎯 2 (Simple) | ⏱️ ~10 minutes Note ⚡️ Unit Test Generation is now available in beta!Learn more here, or try it out under "Finishing Touches" below. ✨ Finishing Touches🧪 Generate unit tests
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
SupportNeed help? Create a ticket on our support page for assistance with any issues or questions. Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
Documentation and Community
|
There was a problem hiding this comment.
Actionable comments posted: 4
🔭 Outside diff range comments (2)
docs/user/configuration.md (1)
188-193: Update remainingplatformkeys totypeThere are still a handful of docs examples using the deprecated
platform:key. Please replace each withtype:to keep copy-paste users from hitting confusing, mixed syntax:
docs/user/getting-started.md
• Line 108: changeplatform: "ollama"→type: "ollama"
• Line 191: changeplatform: "ollama"→type: "ollama"
• Line 195: changeplatform: "ollama"→type: "ollama"
• Line 199: changeplatform: "ollama"→type: "ollama"docs/technical/load-balancing.md
• Line 190: changeplatform: "openai"→type: "openai"docs/technical/service-discovery.md
• Line 501: changeplatform: "openai"→type: "openai"docs/technical/service-discovery.md (1)
495-505: YAML example is syntactically invalid & still uses the deprecatedplatformkey
headers:/Authorization:are indented two spaces fewer than the preceding endpoint mapping, so they are outside the endpoint node.- The third endpoint (
groq-cloud) is indented four spaces instead of six, so it is no longer insidestatic.endpointsand breaks the list.- That same endpoint still uses
platform: "openai"instead of the newtypekey introduced above.- # OpenAI Compatible - - name: "groq-cloud" - url: "https://api.groq.com/openai/v1" - platform: "openai" + # OpenAI-compatible + - name: "groq-cloud" + url: "https://api.groq.com/openai/v1" + type: "openai-compatible"Also move
headers:beneath the LM Studio endpoint (six-space indent) so it belongs to that node.Readers copy-pasting the snippet will currently get a YAML parse error.
🧹 Nitpick comments (5)
docs/user/endpoints.md (1)
198-202: Minor naming nit – double-check header/table captionsThe docs now use
type, but a few earlier sentences still talk about “platform”. A quick wording sweep will avoid confusing newcomers skimming the page.docs/user/load-balancers.md (1)
205-236: Same consistency issue as aboveGreat update, but keep an eye on the prose paragraphs that still reference
platformand the flat list. A follow-up sweep will prevent mixed guidance.docs/technical/service-discovery.md (1)
483-494: Quote the duration literals to avoid YAML 1.2 ambiguityValues such as
2s/1sare not valid numeric scalars; most loaders accept them only when quoted.
Recommend:check_interval: "2s" check_timeout: "1s"This keeps the examples portable for users loading the config with strict parsers or non-Go tooling.
examples/ollama-openwebui/olla.yaml (1)
40-55: Minor lint failures – trailing spaces & missing final newlineYAML-lint flagged:
- Trailing spaces on lines 55 & 65
- File lacks a newline at EOF
- CRLF line endings at line 1
None break functionality but they will annoy CI hooks. Quick fix:
- check_timeout: 1s␠ + check_timeout: 1s ... - check_timeout: 1s␠ + check_timeout: 1s +readme.md (1)
117-146: Ensure YAML example keepsheadersunder the endpointThe two example endpoints lack any auth headers, but the text above talks about bearer tokens. Consider adding a commented-out
headers:block inside each endpoint so that copy-paste users include it in the correct place.
📜 Review details
Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (11)
docs/technical/load-balancing.md(1 hunks)docs/technical/service-discovery.md(2 hunks)docs/user/configuration.md(1 hunks)docs/user/endpoints.md(2 hunks)docs/user/getting-started.md(2 hunks)docs/user/load-balancers.md(4 hunks)docs/user/models.md(8 hunks)examples/ollama-openwebui/README.md(1 hunks)examples/ollama-openwebui/compose.yaml(1 hunks)examples/ollama-openwebui/olla.yaml(1 hunks)readme.md(3 hunks)
🧰 Additional context used
🧠 Learnings (3)
📚 Learning: use the priority balancer for production deployments....
Learnt from: CR
PR: thushan/olla#0
File: CLAUDE.md:0-0
Timestamp: 2025-07-27T12:59:29.787Z
Learning: Use the priority balancer for production deployments.
Applied to files:
docs/technical/load-balancing.mddocs/user/load-balancers.mdreadme.md
📚 Learning: applies to {proxy_sherpa.go,proxy_olla.go} : proxy implementations should be in `proxy_sherpa.go` an...
Learnt from: CR
PR: thushan/olla#0
File: CLAUDE.md:0-0
Timestamp: 2025-07-27T12:59:29.787Z
Learning: Applies to {proxy_sherpa.go,proxy_olla.go} : Proxy implementations should be in `proxy_sherpa.go` and `proxy_olla.go`.
Applied to files:
examples/ollama-openwebui/olla.yaml
📚 Learning: applies to config.yaml : the main configuration should be defined in `config.yaml`....
Learnt from: CR
PR: thushan/olla#0
File: CLAUDE.md:0-0
Timestamp: 2025-07-27T12:59:29.787Z
Learning: Applies to config.yaml : The main configuration should be defined in `config.yaml`.
Applied to files:
examples/ollama-openwebui/olla.yamlreadme.md
🪛 YAMLlint (1.37.1)
examples/ollama-openwebui/olla.yaml
[error] 1-1: wrong new line character: expected \n
(new-lines)
[error] 55-55: trailing spaces
(trailing-spaces)
[error] 65-65: trailing spaces
(trailing-spaces)
[error] 98-98: no new line character at the end of file
(new-line-at-end-of-file)
examples/ollama-openwebui/compose.yaml
[error] 1-1: wrong new line character: expected \n
(new-lines)
[warning] 14-14: wrong indentation: expected 6 but found 8
(indentation)
[error] 46-46: no new line character at the end of file
(new-line-at-end-of-file)
🪛 LanguageTool
examples/ollama-openwebui/README.md
[uncategorized] ~30-~30: The abbreviation “e.g.” (= for example) requires two periods.
Context: ...e to override within that folder.
Eg. the
config.yamlis in `/app/config.ya...
(E_G)
[grammar] ~61-~61: The verb ‘Compose’ is plural. Did you mean: “composes”? Did you use a verb instead of a noun?
Context: ...vers)
How It Works
This Docker Compose stack runs:
- Olla - Proxy that lo...
(PLURAL_VERB_AFTER_THIS)
[grammar] ~134-~134: A determiner may be missing.
Context: ...tions
- **Priority**: Routes to highest priority healthy endpoint (recommended ...
(THE_SUPERLATIVE)
---
[grammar] ~136-~136: A determiner may be missing.
Context: ...-connections**: Routes to endpoint with fewest active requests
## Monitoring
###...
(THE_SUPERLATIVE)
</details>
</details>
<details>
<summary>🪛 markdownlint-cli2 (0.17.2)</summary>
<details>
<summary>examples/ollama-openwebui/README.md</summary>
7-7: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
---
53-53: Bare URL used
(MD034, no-bare-urls)
</details>
</details>
</details>
<details>
<summary>🔇 Additional comments (12)</summary><blockquote>
<details>
<summary>docs/user/endpoints.md (1)</summary>
`138-141`: **Consistent terminology – nice!**
`type: "ollama"` aligns with the new schema.
No further issues spotted in the updated JSON block.
</details>
<details>
<summary>docs/user/load-balancers.md (3)</summary>
`28-48`: **Updated schema clashes with legacy snippet further down**
The priority-balancer example here is correct, but the “Home Lab Setup” / “Enterprise Tiered Infrastructure” YAML blocks that follow (Lines 54-70) still use the old flat `endpoints:` list.
Recommend refreshing those blocks or adding a note (“legacy schema”) so readers aren’t bounced between two formats.
---
`178-196`: **LGTM on the development config**
The nested `static.endpoints` structure and health-check fields read well.
No additional tweaks needed.
---
`245-271`: **Nice touch adding health-check cadence fields**
Including `check_interval` / `check_timeout` directly in examples will save users hunting through the config reference – good doc ergonomics.
</details>
<details>
<summary>docs/user/getting-started.md (1)</summary>
`244-262`: **Multi-platform example reads well**
The switch to `type` for heterogeneous back-ends is clear and the health-check fields are consistent. Nice work.
</details>
<details>
<summary>examples/ollama-openwebui/olla.yaml (1)</summary>
`5-10`: **Quote numeric fields that are logically strings**
`host: "0.0.0.0"` is correct, but `port: 40114` is quoted.
If the struct expects an `int`, the quote forces a string and will cause an unmarshalling error. Remove the quotes around the port (and any other purely numeric field) to stay type-safe.
</details>
<details>
<summary>docs/user/models.md (1)</summary>
`355-367`: **Provider type strings are inconsistent**
`"lm-studio"` (examples earlier) vs `"lmstudio"` (here) will cause unifier rules to miss matches. Standardise on one slug (documentation uses `lm-studio`). Update the model status JSON and any custom-rule sections accordingly.
</details>
<details>
<summary>examples/ollama-openwebui/README.md (5)</summary>
`7-23`: **Add language spec & tidy ASCII diagram for lint-cleanliness**
`markdownlint` flags the fence as `MD040` (missing language).
While fixing that, consider aligning the broken “(Port 40114)” line so the diagram renders cleanly in mono-font.
```diff
-```
+```text
┌─────────────┐ ┌──────────┐ ┌─────────────────┐
│ OpenWebUI │───▶│ Olla │───▶│ Ollama Instance │
-│ (Port 3000) │ │(Port │ │ (External) │
-│ │ │ 40114) │ │ │
+│ (Port 3000) │ │ (Port │ │ (External) │
+│ │ │ 40114) │ │ │
└─────────────┘ └──────────┘ └─────────────────┘
[ suggest_nitpick ]
27-31: Use the correct Latin abbreviation “e.g.”
Minor grammar: replace “Eg.” with “e.g.,” and keep the sentence flowing.
-> Eg. the `config.yaml` is in `/app/config.yaml`. Logs are available in `/app/logs/`.
+> e.g., the `config.yaml` file is in `/app/config.yaml`; logs are available under `/app/logs/`.[ suggest_nitpick ]
61-64: Fix verb agreement – “stack includes” vs “stack runs”
“This Docker Compose stack runs:” triggers LanguageTool’s PLURAL_VERB_AFTER_THIS.
Changing to “includes” removes the grammar warning and reads better.
-This Docker Compose stack runs:
+This Docker Compose stack includes:[ suggest_nitpick ]
134-136: Insert missing determiners for superlatives
Both bullet points lack “the”, flagged by LanguageTool.
-**Priority**: Routes to highest priority healthy endpoint (recommended for mixed hardware)
-**Least-connections**: Routes to endpoint with fewest active requests
+**Priority**: Routes to the highest-priority healthy endpoint (recommended for mixed hardware)
+**Least-connections**: Routes to the endpoint with the fewest active requests[ suggest_nitpick ]
52-54: Wrap bare URL to silence MD034
GitHub markdown renders bare URLs, but wrapping avoids the lint error:
-3. **Access OpenWebUI** at http://localhost:3000
+3. **Access OpenWebUI** at <http://localhost:3000>[ suggest_nitpick ]
| type: "static" | ||
| static: | ||
| endpoints: | ||
| # Local development | ||
| - url: "http://localhost:11434" | ||
| name: "local" | ||
| type: "ollama" | ||
| priority: 1 | ||
| model_url: "/api/tags" | ||
| health_check_url: "/" | ||
| check_interval: 2s | ||
| check_timeout: 1s | ||
|
|
||
| # Primary production | ||
| - url: "http://gpu-cluster.internal:11434" | ||
| name: "prod-primary" | ||
| type: "ollama" | ||
| priority: 2 | ||
| model_url: "/api/tags" | ||
| health_check_url: "/" | ||
| check_interval: 2s | ||
| check_timeout: 1s | ||
|
|
||
| # Failover | ||
| - url: "http://gpu-cluster-2.internal:11434" | ||
| name: "prod-secondary" | ||
| type: "ollama" | ||
| priority: 3 | ||
| model_url: "/api/tags" | ||
| health_check_url: "/" | ||
| check_interval: 2s | ||
| check_timeout: 1s | ||
|
|
There was a problem hiding this comment.
YAML structure now nested – but the example is internally inconsistent
Good: the endpoints are under discovery.static.endpoints.
Problem: immediately below (Line 186-190, unchanged) the “cloud backup” endpoint sits outside the static block and still uses platform:. Copy-pasting this example will give invalid YAML.
Consider moving the cloud backup entry inside the static.endpoints list and renaming the key to type:
- # Cloud backup (expensive)
- - name: "openai-backup"
- url: "https://api.openai.com"
- priority: 10
- platform: "openai"
+ # Cloud backup (expensive)
+ - url: "https://api.openai.com"
+ name: "openai-backup"
+ type: "openai"
+ priority: 10Committable suggestion skipped: line range outside the PR's diff.
🤖 Prompt for AI Agents
In docs/technical/load-balancing.md around lines 153 to 185, the YAML example
shows endpoints nested under discovery.static.endpoints, but the subsequent
"cloud backup" endpoint (lines 186-190) is outside the static block and uses the
key platform:, causing invalid YAML if copied. To fix this, move the cloud
backup endpoint inside the static.endpoints list and rename the key from
platform: to type: to maintain consistency and valid YAML structure.
| type: "static" | ||
| static: | ||
| endpoints: | ||
| # Primary: Local GPU cluster | ||
| - url: "http://10.0.1.10:11434" | ||
| name: "gpu-cluster-1" | ||
| type: "ollama" | ||
| priority: 1 | ||
| model_url: "/api/tags" | ||
| health_check_url: "/" | ||
| check_interval: 2s | ||
| check_timeout: 1s | ||
|
|
||
| - url: "http://10.0.1.11:11434" | ||
| name: "gpu-cluster-2" | ||
| type: "ollama" | ||
| priority: 1 | ||
| model_url: "/api/tags" | ||
| health_check_url: "/" | ||
| check_interval: 2s | ||
| check_timeout: 1s | ||
|
|
There was a problem hiding this comment.
🛠️ Refactor suggestion
Good migration – but the “Basic Configuration” section is now stale
The Quick-start “Basic Configuration” earlier in the doc (Lines 96-110) still shows the old schema (platform, flat list). Readers following the guide chronologically will hit two different formats.
Either:
- Update that snippet to match the new nested
typestructure; or - Add a short “legacy/simple schema” note explaining the difference.
Failing to do so will generate support questions.
🤖 Prompt for AI Agents
In docs/user/getting-started.md around lines 96 to 110, the "Basic
Configuration" section uses an outdated schema format that conflicts with the
new nested "type" structure shown later in lines 211 to 232. Update the snippet
in the "Basic Configuration" section to reflect the new nested schema format
with "type" and "static" keys, or alternatively add a note explaining that the
earlier snippet uses a legacy/simple schema and highlight the differences to
prevent confusion.
| - ./.container/olla/logs:/app/logs:ro | ||
| healthcheck: | ||
| test: ["CMD", "wget", "--quiet", "--tries=1", "--spider", "http://localhost:40114/internal/health"] | ||
| timeout: 5s | ||
| interval: 30s | ||
| retries: 3 | ||
| start_period: 10s |
There was a problem hiding this comment.
🛠️ Refactor suggestion
Container cannot write its own logs (bind mount is read-only)
./.container/olla/logs:/app/logs:ro mounts the log directory read-only, so Olla will fail to create/append logs.
Drop the :ro or map to a dedicated host-side log folder:
- - ./.container/olla/logs:/app/logs:ro
+ - ./olla-logs:/app/logsThis avoids silent logging failures.
📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| - ./.container/olla/logs:/app/logs:ro | |
| healthcheck: | |
| test: ["CMD", "wget", "--quiet", "--tries=1", "--spider", "http://localhost:40114/internal/health"] | |
| timeout: 5s | |
| interval: 30s | |
| retries: 3 | |
| start_period: 10s | |
| - ./olla-logs:/app/logs | |
| healthcheck: | |
| test: ["CMD", "wget", "--quiet", "--tries=1", "--spider", "http://localhost:40114/internal/health"] | |
| timeout: 5s | |
| interval: 30s | |
| retries: 3 | |
| start_period: 10s |
🧰 Tools
🪛 YAMLlint (1.37.1)
[warning] 14-14: wrong indentation: expected 6 but found 8
(indentation)
🤖 Prompt for AI Agents
In examples/ollama-openwebui/compose.yaml around lines 12 to 18, the log
directory is mounted as read-only with ':ro', preventing the container from
writing logs. Remove the ':ro' suffix from the volume mount to make it writable,
or alternatively, map the container log directory to a dedicated writable host
folder to ensure the container can create and append logs without failure.
| test: ["CMD", "wget", "--quiet", "--tries=1", "--spider", "http://localhost:40114/internal/health"] | ||
| timeout: 5s | ||
| interval: 30s | ||
| retries: 3 |
There was a problem hiding this comment.
Indentation off by two spaces breaks the healthcheck block
Docker Compose v2 expects test, timeout, etc. at six spaces under healthcheck. Current eight-space indent triggers:
mapping values are not allowed …
Fix:
- test: ["CMD", "wget", "--quiet", "--tries=1", "--spider", "http://localhost:40114/internal/health"]
- timeout: 5s
+ test: ["CMD", "wget", "--quiet", "--tries=1", "--spider", "http://localhost:40114/internal/health"]
+ timeout: 5sCommittable suggestion skipped: line range outside the PR's diff.
🧰 Tools
🪛 YAMLlint (1.37.1)
[warning] 14-14: wrong indentation: expected 6 but found 8
(indentation)
🤖 Prompt for AI Agents
In examples/ollama-openwebui/compose.yaml at lines 14 to 17, the keys under the
healthcheck block are indented with eight spaces instead of the required six
spaces. Adjust the indentation of test, timeout, interval, and retries to be
exactly six spaces under healthcheck to fix the YAML parsing error.
This PR adds documentation updating examples from scout/sherpa to Olla.
Also adds examples for ollama-openwebui stack to help #37.
Summary by CodeRabbit