feat: provider usage metrics, model capabilities, and active health probing#935
feat: provider usage metrics, model capabilities, and active health probing#935
Conversation
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Repository UI Review profile: ASSERTIVE Plan: Pro Run ID: 📒 Files selected for processing (1)
📜 Recent review details⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (8)
🧰 Additional context used📓 Path-based instructions (5)**/*.{py,ts,tsx,md}📄 CodeRabbit inference engine (CLAUDE.md)
Files:
web/src/**/*.{ts,tsx}📄 CodeRabbit inference engine (CLAUDE.md)
Files:
web/src/**/*📄 CodeRabbit inference engine (CLAUDE.md)
Files:
web/{tsconfig.json,src/**/*.{ts,tsx}}📄 CodeRabbit inference engine (web/CLAUDE.md)
Files:
web/src/__tests__/**/*.{test,spec}.{ts,tsx}📄 CodeRabbit inference engine (web/CLAUDE.md)
Files:
🧠 Learnings (4)📓 Common learnings📚 Learning: 2026-03-30T16:37:21.464ZApplied to files:
📚 Learning: 2026-03-15T18:28:13.207ZApplied to files:
📚 Learning: 2026-03-30T16:37:21.464ZApplied to files:
🔇 Additional comments (7)
WalkthroughThe PR adds provider usage metrics and model capability exposure across backend, tests, and frontend. Backend: Suggested labels
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. Comment |
Dependency Review✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.Snapshot WarningsEnsure that dependencies are being submitted on PR branches. Re-running this action after a short time may resolve the issue. See the documentation for more information and troubleshooting advice. Scanned FilesNone |
There was a problem hiding this comment.
Code Review
This pull request implements active health probing for LLM providers and enriches health metrics with 24-hour token usage and cost data. It introduces a ProviderHealthProber background service, an "unknown" health status for inactive providers, and model capability badges in the UI. Backend updates include provider-level usage aggregation in the CostTracker and enriched model metadata in API responses. Critical feedback highlights multiple instances of invalid Python 2 syntax for catching exceptions in providers.py and health_prober.py, which will result in runtime SyntaxErrors in Python 3.
| if driver is not None: | ||
| try: | ||
| caps = await driver.get_model_capabilities(model_config.id) | ||
| except MemoryError, RecursionError: |
There was a problem hiding this comment.
This line uses Python 2 syntax for catching multiple exceptions. In Python 3, multiple exceptions should be enclosed in parentheses.
This is a critical bug that will cause a SyntaxError at runtime.
except (MemoryError, RecursionError):
raise| "total_cost_24h": usage.total_cost, | ||
| }, | ||
| ) | ||
| except MemoryError, RecursionError: |
There was a problem hiding this comment.
This line uses Python 2 syntax for catching multiple exceptions. In Python 3, multiple exceptions should be enclosed in parentheses.
This is a critical bug that will cause a SyntaxError at runtime.
except (MemoryError, RecursionError):
raise| error_msg = "timeout" | ||
| except asyncio.CancelledError: | ||
| raise | ||
| except MemoryError, RecursionError: |
There was a problem hiding this comment.
This line uses Python 2 syntax for catching multiple exceptions. In Python 3, multiple exceptions should be enclosed in parentheses.
This is a critical bug that will cause a SyntaxError at runtime.
| except MemoryError, RecursionError: | |
| except (MemoryError, RecursionError): | |
| raise |
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #935 +/- ##
==========================================
- Coverage 92.20% 92.18% -0.03%
==========================================
Files 598 600 +2
Lines 31720 31915 +195
Branches 3088 3109 +21
==========================================
+ Hits 29248 29420 +172
- Misses 1949 1966 +17
- Partials 523 529 +6 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
There was a problem hiding this comment.
Actionable comments posted: 11
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@src/synthorg/api/controllers/providers.py`:
- Around line 223-224: In the except blocks that currently read "except
MemoryError, RecursionError:" in src/synthorg/api/controllers/providers.py, add
an error log call (e.g., logger.error or processLogger.error) that includes
relevant context (provider id/name, model, request id or enrichment input) and
the exception details (use exc_info=True or include the exception string)
immediately before re-raising; apply the same change to the second occurrence
mentioned (around the other except block). Ensure you reference the same logger
used in this module and keep the re-raise behavior unchanged.
In `@src/synthorg/api/dto_providers.py`:
- Around line 86-98: Replace the untyped kwargs dict + type: ignore by calling
ProviderModelResponse with explicit typed keyword arguments: pass id=config.id,
alias=config.alias, cost_per_1k_input=config.cost_per_1k_input,
cost_per_1k_output=config.cost_per_1k_output, max_context=config.max_context,
estimated_latency_ms=config.estimated_latency_ms, and for the capability flags
use supports_tools=capabilities.supports_tools if capabilities is not None else
False (similarly for supports_vision and supports_streaming) so the constructor
receives correctly typed values and you can drop the "# type: ignore[arg-type]"
on the ProviderModelResponse(...) call.
In `@src/synthorg/api/dto.py`:
- Around line 830-836: The re-export of ProviderModelResponse and
to_provider_model_response is placed at the bottom of dto.py using a noqa E402;
move that import statement to the top with the other imports (or alternatively
declare these names in __all__) so the module-level re-exports are conventional
and don't require E402 suppression; update the import block to include
ProviderModelResponse and to_provider_model_response from
synthorg.api.dto_providers and remove the inline noqa comment.
In `@src/synthorg/budget/tracker.py`:
- Around line 249-251: The provider parameter currently typed as str | None
allows empty strings to silently pass; change the public method signatures that
accept provider (e.g., get_provider_usage and the other public methods around
the later block) to use NotBlankStr | None instead of str | None, import
NotBlankStr where needed, and update any related type hints/signatures
referencing agent_id, task_id, provider to use NotBlankStr for name/identifier
fields so they match CostRecord.provider and fail fast on blank input; apply the
same change to the other methods mentioned around the 292-298 region.
In `@src/synthorg/providers/health_prober.py`:
- Around line 99-109: The constructor (__init__) should validate the
interval_seconds parameter and reject non-positive values to avoid tight loops;
check that interval_seconds is > 0 at the start of __init__ (before assigning to
self._interval) and raise a ValueError with a clear message if not, so callers
using ProviderHealthTracker/ConfigResolver get a fast failure instead of
continuous re-probing or invalid waits.
- Around line 52-53: Summary: Remove the hard-coded "ollama" vendor special-case
in the conditional that checks litellm_provider and ":11434". Change the health
probe to read a provider-specific ping path from provider/preset metadata (e.g.,
a "ping_path" or "ping_response_keyword" field) instead of matching vendor
names; replace the conditional that references litellm_provider and ":11434"
with a lookup like provider_config.get("ping_path") and use that value to decide
whether to return stripped. Update any provider presets to supply the
ping_path/ping_response_keyword as needed and ensure the prober uses that
metadata when evaluating stripped.
- Around line 136-141: In _run_loop(), the broad `except Exception:` around the
call to `await self._probe_all()` swallows fatal errors like
MemoryError/RecursionError and ExceptionGroup from TaskGroup; replace it with an
exception handler that only logs recoverable exceptions and re-raises fatal
ones: catch Exception as e, then if isinstance(e, BaseExceptionGroup) or
isinstance(e, (MemoryError, RecursionError)): raise, otherwise call
logger.exception(PROVIDER_HEALTH_PROBER_CYCLE_FAILED) — this preserves the
current logging via PROVIDER_HEALTH_PROBER_CYCLE_FAILED while allowing fatal
errors to propagate from _run_loop().
In `@tests/unit/providers/test_health_prober.py`:
- Around line 21-29: Replace hardcoded vendor names in the tests with
vendor-agnostic fixtures: rename test functions and any literal vendor strings
referencing "ollama" to generic names like "test-provider" or "test-local"
(e.g., change test_ollama_returns_root → test_provider_returns_root and the
input vendor arg "ollama" → "test-provider"), and update
test_ollama_detected_by_port → test_local_detected_by_port (and similar
occurrences) so all assertions still call _build_ping_url with the same URLs but
use generic vendor identifiers; apply the same renaming pattern to the other
test cases highlighted in the comment (the other occurrences around lines noted)
to ensure no real vendor names remain in test names or fixture strings.
In `@web/src/pages/providers/ProviderHealthMetrics.stories.tsx`:
- Around line 37-47: The story "NoUsage" currently inherits baseHealth which
leaves health_status as 'up'; update the NoUsage story's args (the health object
in ProviderHealthMetrics.stories.tsx) to set health_status: 'unknown' so the
zero-calls/zero-tokens case matches backend semantics and exercises the
unknown-state rendering; locate the NoUsage export and add/override
health_status: 'unknown' on the health object (near baseHealth, calls_last_24h,
total_tokens_24h fields).
In `@web/src/pages/providers/ProviderModelList.stories.tsx`:
- Around line 59-71: The story fixture NoCapabilities uses vendor-specific
identifiers in the models array (id 'local-llama' and alias 'llama'); update the
model object inside NoCapabilities.args.models to use generic, project-owned
identifiers (e.g., change id and alias to non-vendor names like 'local-model' /
'model' or similar) while leaving the other properties (cost_per_1k_input,
max_context, supports_tools, supports_vision, supports_streaming, etc.)
unchanged so the story remains functionally identical but vendor-agnostic.
In `@web/src/utils/providers.ts`:
- Around line 148-151: The formatCost function currently defaults currencyCode
to 'EUR', which can mislabel provider costs that are USD-based; update the
function signature for formatCost(cost: number, currencyCode: string = 'EUR') to
use 'USD' as the default (or require callers to pass currency explicitly) and
audit callers of formatCost to ensure they either pass the correct currencyCode
or rely on the new 'USD' default so provider spend (cost_usd semantics) is
represented correctly.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI
Review profile: ASSERTIVE
Plan: Pro
Run ID: 75838f50-67d5-40ec-b7f6-3dd9b36c7a1d
📒 Files selected for processing (35)
CLAUDE.mddocs/design/brand-and-ux.mddocs/design/operations.mddocs/design/page-structure.mdsrc/synthorg/api/controllers/providers.pysrc/synthorg/api/dto.pysrc/synthorg/api/dto_providers.pysrc/synthorg/budget/tracker.pysrc/synthorg/observability/events/api.pysrc/synthorg/observability/events/budget.pysrc/synthorg/observability/events/provider.pysrc/synthorg/providers/health.pysrc/synthorg/providers/health_prober.pytests/unit/api/controllers/test_provider_health.pytests/unit/api/test_dto_provider_model.pytests/unit/budget/test_tracker_provider_filter.pytests/unit/budget/test_tracker_provider_usage.pytests/unit/providers/test_health.pytests/unit/providers/test_health_prober.pyweb/CLAUDE.mdweb/src/__tests__/pages/ProviderDetailPage.test.tsxweb/src/__tests__/utils/providers.test.tsweb/src/api/endpoints/providers.tsweb/src/api/types.tsweb/src/components/ui/provider-health-badge.stories.tsxweb/src/components/ui/provider-health-badge.tsxweb/src/hooks/useProviderDetailData.tsweb/src/pages/providers/ProviderCard.stories.tsxweb/src/pages/providers/ProviderCard.tsxweb/src/pages/providers/ProviderHealthMetrics.stories.tsxweb/src/pages/providers/ProviderHealthMetrics.tsxweb/src/pages/providers/ProviderModelList.stories.tsxweb/src/pages/providers/ProviderModelList.tsxweb/src/stores/providers.tsweb/src/utils/providers.ts
| except MemoryError, RecursionError: | ||
| raise |
There was a problem hiding this comment.
Log fatal enrichment failures before re-raising.
These branches abort the request without emitting any provider/model context, which makes the hardest failures much harder to diagnose. Emit an error log entry before raise.
As per coding guidelines, "All error paths must log at WARNING or ERROR with context before raising".
Also applies to: 281-282
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@src/synthorg/api/controllers/providers.py` around lines 223 - 224, In the
except blocks that currently read "except MemoryError, RecursionError:" in
src/synthorg/api/controllers/providers.py, add an error log call (e.g.,
logger.error or processLogger.error) that includes relevant context (provider
id/name, model, request id or enrichment input) and the exception details (use
exc_info=True or include the exception string) immediately before re-raising;
apply the same change to the second occurrence mentioned (around the other
except block). Ensure you reference the same logger used in this module and keep
the re-raise behavior unchanged.
src/synthorg/api/dto.py
Outdated
|
|
||
|
|
||
| # Re-export from dto_providers to keep imports stable. | ||
| from synthorg.api.dto_providers import ( # noqa: E402, F401 | ||
| ProviderModelResponse, | ||
| to_provider_model_response, | ||
| ) |
There was a problem hiding this comment.
🧹 Nitpick | 🔵 Trivial
Consider moving re-exports to the top of the file.
While the # noqa: E402 suppression works, placing imports at the bottom after function definitions is unconventional and can be confusing for maintainers. A cleaner approach is to place the re-exports near the top with other imports, or use __all__ to explicitly declare the public API.
♻️ Alternative: move re-exports to top
from synthorg.providers.enums import AuthType
+from synthorg.api.dto_providers import (
+ ProviderModelResponse,
+ to_provider_model_response,
+)
DEFAULT_LIMIT: int = 50
...
-
-
-# Re-export from dto_providers to keep imports stable.
-from synthorg.api.dto_providers import ( # noqa: E402, F401
- ProviderModelResponse,
- to_provider_model_response,
-)🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@src/synthorg/api/dto.py` around lines 830 - 836, The re-export of
ProviderModelResponse and to_provider_model_response is placed at the bottom of
dto.py using a noqa E402; move that import statement to the top with the other
imports (or alternatively declare these names in __all__) so the module-level
re-exports are conventional and don't require E402 suppression; update the
import block to include ProviderModelResponse and to_provider_model_response
from synthorg.api.dto_providers and remove the inline noqa comment.
| if litellm_provider == "ollama" or ":11434" in stripped: | ||
| return stripped # Ollama root returns "Ollama is running" |
There was a problem hiding this comment.
🛠️ Refactor suggestion | 🟠 Major
Remove the hard-coded vendor special-case.
Baking "ollama" and a vendor-specific comment into core provider code violates the repo's vendor-agnostic rule. Push ping-path selection into provider config or preset metadata so the prober stays generic.
As per coding guidelines, "Vendor-agnostic everywhere: NEVER use real vendor names (Anthropic, OpenAI, Claude, GPT, etc.) in project-owned code, docstrings, comments, tests, or config examples; vendor names only in: design spec, .claude/ files, third-party imports, provider presets".
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@src/synthorg/providers/health_prober.py` around lines 52 - 53, Summary:
Remove the hard-coded "ollama" vendor special-case in the conditional that
checks litellm_provider and ":11434". Change the health probe to read a
provider-specific ping path from provider/preset metadata (e.g., a "ping_path"
or "ping_response_keyword" field) instead of matching vendor names; replace the
conditional that references litellm_provider and ":11434" with a lookup like
provider_config.get("ping_path") and use that value to decide whether to return
stripped. Update any provider presets to supply the
ping_path/ping_response_keyword as needed and ensure the prober uses that
metadata when evaluating stripped.
| try: | ||
| await self._probe_all() | ||
| except asyncio.CancelledError: | ||
| raise | ||
| except Exception: | ||
| logger.exception(PROVIDER_HEALTH_PROBER_CYCLE_FAILED) |
There was a problem hiding this comment.
🧩 Analysis chain
🌐 Web query:
In Python 3.14, when a child task in asyncio.TaskGroupraisesMemoryErrororRecursionError, does awaiting the task group raise an ExceptionGroupthat is caught by a surroundingexcept Exception?
💡 Result:
Yes.
In Python 3.14, when an asyncio.TaskGroup finishes and one or more child tasks failed (with exceptions other than asyncio.CancelledError), await/exiting the task group raises an ExceptionGroup or BaseExceptionGroup as appropriate. [1]
MemoryError and RecursionError are subclasses of Exception, so the task group will raise an ExceptionGroup (not a BaseExceptionGroup) in that case, and except Exception: will catch it (by design, except Exception catches ExceptionGroup but not BaseExceptionGroup). [1][2]
🏁 Script executed:
fd -p "health_prober.py" -x wc -l {}Repository: Aureliolo/synthorg
Length of output: 156
🏁 Script executed:
cat -n src/synthorg/providers/health_prober.py | head -160 | tail -40Repository: Aureliolo/synthorg
Length of output: 1937
🏁 Script executed:
# Search for _probe_all implementation
rg -n "_probe_all" src/synthorg/providers/health_prober.py -A 10 -B 2Repository: Aureliolo/synthorg
Length of output: 1310
🏁 Script executed:
# Search for TaskGroup usage in health_prober.py
rg -n "TaskGroup" src/synthorg/providers/health_prober.pyRepository: Aureliolo/synthorg
Length of output: 115
🏁 Script executed:
sed -n '151,200p' src/synthorg/providers/health_prober.pyRepository: Aureliolo/synthorg
Length of output: 2006
Broad except Exception: in _run_loop() suppresses fatal TaskGroup failures.
When TaskGroup child tasks fail with MemoryError or RecursionError, the task group raises ExceptionGroup, which is caught by the broad except Exception: on line 140. This silently swallows fatal errors—logging them once while allowing the prober to continue running. Either re-raise BaseExceptionGroup and other fatal exceptions, or explicitly catch only recoverable exceptions, following the guideline "Errors: handle explicitly, never silently swallow".
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@src/synthorg/providers/health_prober.py` around lines 136 - 141, In
_run_loop(), the broad `except Exception:` around the call to `await
self._probe_all()` swallows fatal errors like MemoryError/RecursionError and
ExceptionGroup from TaskGroup; replace it with an exception handler that only
logs recoverable exceptions and re-raises fatal ones: catch Exception as e, then
if isinstance(e, BaseExceptionGroup) or isinstance(e, (MemoryError,
RecursionError)): raise, otherwise call
logger.exception(PROVIDER_HEALTH_PROBER_CYCLE_FAILED) — this preserves the
current logging via PROVIDER_HEALTH_PROBER_CYCLE_FAILED while allowing fatal
errors to propagate from _run_loop().
| def test_ollama_returns_root(self) -> None: | ||
| assert ( | ||
| _build_ping_url("http://localhost:11434", "ollama") | ||
| == "http://localhost:11434" | ||
| ) | ||
|
|
||
| def test_ollama_detected_by_port(self) -> None: | ||
| assert _build_ping_url("http://host:11434/", None) == "http://host:11434" | ||
|
|
There was a problem hiding this comment.
Use vendor-agnostic test fixtures instead of hardcoded vendor names.
These tests hardcode a real vendor identifier in fixture data. Please switch to generic test naming (e.g., test-provider, test-local) and avoid embedding vendor names directly in test literals.
Based on learnings: “Tests must use test-provider, test-small-001, etc. for vendor-agnostic test data.” As per coding guidelines: “Vendor-agnostic everywhere: NEVER use real vendor names … in … tests …; use generic names.”
Also applies to: 50-52, 88-90, 141-142
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@tests/unit/providers/test_health_prober.py` around lines 21 - 29, Replace
hardcoded vendor names in the tests with vendor-agnostic fixtures: rename test
functions and any literal vendor strings referencing "ollama" to generic names
like "test-provider" or "test-local" (e.g., change test_ollama_returns_root →
test_provider_returns_root and the input vendor arg "ollama" → "test-provider"),
and update test_ollama_detected_by_port → test_local_detected_by_port (and
similar occurrences) so all assertions still call _build_ping_url with the same
URLs but use generic vendor identifiers; apply the same renaming pattern to the
other test cases highlighted in the comment (the other occurrences around lines
noted) to ensure no real vendor names remain in test names or fixture strings.
| export function formatCost( | ||
| cost: number, | ||
| currencyCode: string = 'EUR', | ||
| ): string { |
There was a problem hiding this comment.
Avoid defaulting provider cost formatting to EUR.
Using 'EUR' as the default can mislabel costs when callers don’t pass currency. Provider spend in this codebase is USD-based by default (cost_usd semantics), so this default is unsafe.
Suggested fix
export function formatCost(
cost: number,
- currencyCode: string = 'EUR',
+ currencyCode: string = 'USD',
): string {📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| export function formatCost( | |
| cost: number, | |
| currencyCode: string = 'EUR', | |
| ): string { | |
| export function formatCost( | |
| cost: number, | |
| currencyCode: string = 'USD', | |
| ): string { |
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@web/src/utils/providers.ts` around lines 148 - 151, The formatCost function
currently defaults currencyCode to 'EUR', which can mislabel provider costs that
are USD-based; update the function signature for formatCost(cost: number,
currencyCode: string = 'EUR') to use 'USD' as the default (or require callers to
pass currency explicitly) and audit callers of formatCost to ensure they either
pass the correct currencyCode or rely on the new 'USD' default so provider spend
(cost_usd semantics) is represented correctly.
Add total_tokens_24h and total_cost_24h to ProviderHealthSummary by enriching the health endpoint with CostTracker data. Add provider filter to CostTracker.get_records() and new get_provider_usage() method. Create ProviderModelResponse DTO that surfaces model capabilities (tools, vision, streaming) from the driver layer via the models endpoint. Frontend: 2 new MetricCards in ProviderHealthMetrics, token/cost display in ProviderCard, capability badges in ProviderModelList. New Storybook stories for ProviderModelList and ProviderHealthMetrics. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add ProviderHealthProber background service that pings provider endpoints every 30 minutes with lightweight HTTP requests (no model loading). Providers with 0 calls in 24h now show "unknown" instead of "up". Real API call outcomes reset the probe timer. Only providers with base_url (local/self-hosted) are probed; cloud providers rely on real call outcomes. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Replace formatCost USD hardcode with existing formatCurrency (EUR default) - Extract ProviderModelResponse to dto_providers.py (dto.py over 800 lines) - Add MemoryError/RecursionError re-raise + error context in all catch blocks - Wrap get_provider_usage() in try/except for graceful degradation - Use asyncio.TaskGroup for parallel health probing - Add exception handling in prober _run_loop to prevent silent task death - Fix config: object typing to ProviderConfig with direct attribute access - Replace vendor name "openai" with "test-api" in test - Use cn() for class merging in CapabilityBadges - Remove unnecessary ?? 0 on non-nullable fields - Update docs: operations.md, brand-and-ux.md, page-structure.md, CLAUDE.md Pre-reviewed by 8 agents, 12 findings addressed Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…abbit) SSRF: add discovery policy allowlist guard in ProviderHealthProber before sending auth headers to provider base_url endpoints. Accept an optional discovery_policy_loader callback to validate probe URLs. Security: truncate error messages to 200 chars, improve connect error fidelity, add MemoryError/RecursionError guard in _run_loop. Conventions: fix vendor names in tests/stories (anthropic, ollama, local-llama), split _probe_one into _execute_probe helper (50-line limit), extract _enrich_with_usage from get_provider_health, remove dto.py re-export block (800-line limit), replace type:ignore with explicit DTO construction, validate interval_seconds >= 1, use NotBlankStr for provider param in CostTracker, hoist _server_error_threshold to module constant. Docs: update ProviderHealthSummary docstring (new fields + UNKNOWN), fix get_provider_health docstring, add formatTokenCount/formatCost to brand-and-ux.md utility table, remove incorrect USD references. Tests: add usage enrichment failure graceful degradation test, add _build_auth_headers parametrized tests, add HTTP 5xx and timeout prober tests, add interval validation tests, verify token/cost rendering in ProviderDetailPage test. Stories: set NoUsage health_status to 'unknown', fix vendor fixtures. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
4e07097 to
8f84bee
Compare
There was a problem hiding this comment.
Actionable comments posted: 6
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
web/src/__tests__/pages/ProviderDetailPage.test.tsx (1)
104-124: 🧹 Nitpick | 🔵 TrivialAdd an assertion for the new cost metric rendering.
This test now covers token formatting, but it still doesn’t validate
total_cost_24houtput.✅ Suggested test addition
expect(screen.getByText('500')).toBeInTheDocument() expect(screen.getByText('250ms')).toBeInTheDocument() expect(screen.getByText('50.0K')).toBeInTheDocument() + expect(screen.getByText(/\$?\s*1\.25/)).toBeInTheDocument()🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@web/src/__tests__/pages/ProviderDetailPage.test.tsx` around lines 104 - 124, Add an assertion to the 'renders health metrics when health available' test to verify the rendered total_cost_24h value from hookReturn.health (currently 1.25); locate the test in ProviderDetailPage.test.tsx (the same block using makeProvider, hookReturn and renderDetail) and add an expect that checks the UI shows the formatted cost string (match the component's formatting, e.g. "$1.25" or "1.25") immediately after the existing expects for calls, response time and tokens.
♻️ Duplicate comments (4)
src/synthorg/budget/tracker.py (1)
631-631: 🛠️ Refactor suggestion | 🟠 MajorUse
NotBlankStr | Nonefor the internalproviderfilter type.Line 631 currently weakens the provider identifier constraint back to
str | None, which makes this helper inconsistent with the stricter public API contract.As per coding guidelines, "Use `NotBlankStr` ... for all identifier/name fields."Suggested fix
def _filter_records( # noqa: PLR0913 records: Sequence[CostRecord], *, agent_id: str | None = None, task_id: str | None = None, - provider: str | None = None, + provider: NotBlankStr | None = None, start: datetime | None = None, end: datetime | None = None, ) -> tuple[CostRecord, ...]:🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@src/synthorg/budget/tracker.py` at line 631, Change the internal provider filter type from a plain str to the stricter NotBlankStr union: replace the parameter annotation "provider: str | None" with "provider: NotBlankStr | None" wherever that helper function defines the provider parameter (the function that currently declares provider in src/synthorg/budget/tracker.py). Also add or update the import for NotBlankStr at the top of the module if missing, and adjust any local variable/type hints or docstrings that reference the old "str | None" to use NotBlankStr | None so the internal filter matches the public API contract.src/synthorg/api/controllers/providers.py (1)
90-91:⚠️ Potential issue | 🟡 MinorLog fatal exceptions before re-raising.
These branches re-raise without provider/model context, which weakens incident diagnostics on OOM/recursion failures.
Suggested fix
- except MemoryError, RecursionError: + except MemoryError, RecursionError as exc: + logger.error( + API_PROVIDER_USAGE_ENRICHMENT_FAILED, + provider=name, + error=str(exc), + error_type=type(exc).__qualname__, + ) raise @@ - except MemoryError, RecursionError: + except MemoryError, RecursionError as exc: + logger.error( + API_MODEL_CAPABILITIES_LOOKUP_FAILED, + provider=name, + model=model_config.id, + error=str(exc), + error_type=type(exc).__qualname__, + ) raiseAs per coding guidelines: “All error paths must log at WARNING or ERROR with context before raising.”
Also applies to: 265-266
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@src/synthorg/api/controllers/providers.py` around lines 90 - 91, The except blocks that catch MemoryError and RecursionError should log contextual details before re-raising: update both occurrences (the block shown and the similar one at lines ~265-266) to call processLogger.error or module logger with a message including provider and model context (e.g., provider, model, provider_id, model_name or equivalent variables in scope) plus the exception info (use exc_info=True or logger.exception) and then re-raise; keep the existing re-raise behavior but ensure the log includes the exception type and relevant provider/model identifiers to meet the error-path logging guideline.tests/unit/providers/test_health_prober.py (1)
24-25:⚠️ Potential issue | 🟠 MajorUse vendor-agnostic fixtures in tests.
These literals/comments still hardcode a real vendor identifier in project-owned test code.
Suggested fix
def _make_local_config( *, base_url: str = "http://localhost:11434", - litellm_provider: str | None = "ollama", + litellm_provider: str | None = "test-provider", @@ class TestBuildPingUrl: def test_root_url_provider_returns_root(self) -> None: - # Provider type "ollama" uses root URL (liveness string) + # Local default port uses root URL (liveness string) assert ( - _build_ping_url("http://localhost:11434", "ollama") + _build_ping_url("http://localhost:11434", "test-provider") == "http://localhost:11434" )As per coding guidelines: “Vendor-agnostic everywhere: NEVER use real vendor names … in project-owned code, docstrings, comments, tests, or config examples; use generic names.”
Also applies to: 40-43
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/unit/providers/test_health_prober.py` around lines 24 - 25, Replace vendor-specific literals in the test fixtures (e.g., the litellm_provider default "ollama") with a vendor-agnostic placeholder such as "generic-provider" or "test-provider"; update any other occurrences (the other literals around the same fixture) and comments referenced in the test to use the same generic name and ensure assertions / mocks rely on capability flags or injected behavior rather than a real vendor string—look for the litellm_provider fixture and nearby auth_type defaults and change their values and any vendor mentions accordingly.web/src/utils/providers.ts (1)
148-151:⚠️ Potential issue | 🟠 MajorDefault currency should not be EUR.
Using EUR as fallback can mislabel provider spend whenever callers don’t pass a currency code.
Suggested fix
export function formatCost( cost: number, - currencyCode: string = 'EUR', + currencyCode: string = 'USD', ): string {🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@web/src/utils/providers.ts` around lines 148 - 151, The function formatCost currently defaults currencyCode to 'EUR', which can silently mislabel costs; remove the 'EUR' default in the formatCost signature (make currencyCode a required parameter: formatCost(cost: number, currencyCode: string)) and add a runtime guard inside formatCost that throws or asserts when currencyCode is falsy so callers must pass an explicit currency; update any callers to provide the correct currency code.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@docs/design/page-structure.md`:
- Line 118: Update the sentence containing "health metrics (average response
time, error rate percentage, call count, total tokens, cost)" to explicitly
state these metrics are 24-hour aggregates (e.g., append "— 24h" or "24-hour" to
the metrics list) so readers understand these values are daily totals; keep the
rest of the provider description and the `/providers/{name}` detail unchanged.
In `@src/synthorg/providers/health_prober.py`:
- Around line 150-169: The ProviderHealthProber background loop is never started
because the prober is not constructed or registered in the application
lifecycle; update the app startup/shutdown wiring in the _build_lifecycle() /
on_startup() code to instantiate ProviderHealthProber, store it on the app or a
lifecycle container, and call its start() during startup and stop() during
shutdown. Specifically, create the ProviderHealthProber (using whatever
dependencies it needs), add the instance to app state, call
provider_prober.start() in the on_startup handler (or lifecycle startup hook
created in _build_lifecycle()), and ensure provider_prober.stop() is awaited in
the on_shutdown handler so the background task is cancelled cleanly; reference
the ProviderHealthProber class and its start() and stop() methods and the
app-side functions _build_lifecycle() and on_startup() to locate where to add
this wiring.
- Around line 202-208: The branch that logs PROVIDER_HEALTH_PROBE_FAILED when a
URL is disallowed by policy must also record that failure in the health tracker;
after the logger.warning(...) in the is_url_allowed failure branch, call the
health tracker’s record method (e.g. ProviderHealthTracker.record or whatever
instance is used in this module) with the provider name and an appropriate
failed status/error payload (use the same
error="url_not_allowed_by_discovery_policy") so the provider health moves to a
failed state rather than remaining UNKNOWN/stale; ensure you locate the health
tracker instance used elsewhere in health_prober.py and reuse its record(...)
signature.
In `@src/synthorg/providers/health.py`:
- Around line 44-45: Add 'unknown' to the client-side health filter set so the
validator allows filtering by ProviderHealthStatus.UNKNOWN: update the
VALID_HEALTH Set used in ProviderFilters.tsx (the constant currently defined as
new Set(['up','degraded','down'])) to include 'unknown', ensuring the filter
validator name VALID_HEALTH recognizes the new status and matches
backend/display handling (ProviderHealthStatus.UNKNOWN / ProviderHealthBadge).
In `@web/src/__tests__/utils/providers.test.ts`:
- Around line 240-242: The tests assume EUR output but source amounts are USD;
update the assertions to use USD formatting or explicitly pass USD to the
formatter. Locate the tests that call formatCost (e.g., the spec with "returns
EUR 0.00 for zero" and the other failing block) and either change
expect(formatCost(0)).toContain('0.00') to assert the USD symbol/format (e.g.,
expect(...).toContain('$0.00')) or call formatCost with an explicit currency
argument like formatCost(0, 'USD') (or the equivalent API your formatter
exposes) so the test aligns with cost_usd semantics.
In `@web/src/pages/providers/ProviderCard.tsx`:
- Around line 61-70: ProviderCard currently hides token/cost metrics when they
are zero due to checks like health.total_tokens_24h > 0 and
health.total_cost_24h > 0; change the condition to render these spans whenever
health exists (e.g., check health only) so
formatTokenCount(health.total_tokens_24h) and formatCost(health.total_cost_24h)
are displayed even for zero values, keeping the existing span classes and using
formatTokenCount/formatCost to format zeros consistently.
---
Outside diff comments:
In `@web/src/__tests__/pages/ProviderDetailPage.test.tsx`:
- Around line 104-124: Add an assertion to the 'renders health metrics when
health available' test to verify the rendered total_cost_24h value from
hookReturn.health (currently 1.25); locate the test in
ProviderDetailPage.test.tsx (the same block using makeProvider, hookReturn and
renderDetail) and add an expect that checks the UI shows the formatted cost
string (match the component's formatting, e.g. "$1.25" or "1.25") immediately
after the existing expects for calls, response time and tokens.
---
Duplicate comments:
In `@src/synthorg/api/controllers/providers.py`:
- Around line 90-91: The except blocks that catch MemoryError and RecursionError
should log contextual details before re-raising: update both occurrences (the
block shown and the similar one at lines ~265-266) to call processLogger.error
or module logger with a message including provider and model context (e.g.,
provider, model, provider_id, model_name or equivalent variables in scope) plus
the exception info (use exc_info=True or logger.exception) and then re-raise;
keep the existing re-raise behavior but ensure the log includes the exception
type and relevant provider/model identifiers to meet the error-path logging
guideline.
In `@src/synthorg/budget/tracker.py`:
- Line 631: Change the internal provider filter type from a plain str to the
stricter NotBlankStr union: replace the parameter annotation "provider: str |
None" with "provider: NotBlankStr | None" wherever that helper function defines
the provider parameter (the function that currently declares provider in
src/synthorg/budget/tracker.py). Also add or update the import for NotBlankStr
at the top of the module if missing, and adjust any local variable/type hints or
docstrings that reference the old "str | None" to use NotBlankStr | None so the
internal filter matches the public API contract.
In `@tests/unit/providers/test_health_prober.py`:
- Around line 24-25: Replace vendor-specific literals in the test fixtures
(e.g., the litellm_provider default "ollama") with a vendor-agnostic placeholder
such as "generic-provider" or "test-provider"; update any other occurrences (the
other literals around the same fixture) and comments referenced in the test to
use the same generic name and ensure assertions / mocks rely on capability flags
or injected behavior rather than a real vendor string—look for the
litellm_provider fixture and nearby auth_type defaults and change their values
and any vendor mentions accordingly.
In `@web/src/utils/providers.ts`:
- Around line 148-151: The function formatCost currently defaults currencyCode
to 'EUR', which can silently mislabel costs; remove the 'EUR' default in the
formatCost signature (make currencyCode a required parameter: formatCost(cost:
number, currencyCode: string)) and add a runtime guard inside formatCost that
throws or asserts when currencyCode is falsy so callers must pass an explicit
currency; update any callers to provide the correct currency code.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI
Review profile: ASSERTIVE
Plan: Pro
Run ID: b04cbb96-8239-4192-9166-21b56f470b2f
📒 Files selected for processing (34)
CLAUDE.mddocs/design/brand-and-ux.mddocs/design/operations.mddocs/design/page-structure.mdsrc/synthorg/api/controllers/providers.pysrc/synthorg/api/dto_providers.pysrc/synthorg/budget/tracker.pysrc/synthorg/observability/events/api.pysrc/synthorg/observability/events/budget.pysrc/synthorg/observability/events/provider.pysrc/synthorg/providers/health.pysrc/synthorg/providers/health_prober.pytests/unit/api/controllers/test_provider_health.pytests/unit/api/test_dto_provider_model.pytests/unit/budget/test_tracker_provider_filter.pytests/unit/budget/test_tracker_provider_usage.pytests/unit/providers/test_health.pytests/unit/providers/test_health_prober.pyweb/CLAUDE.mdweb/src/__tests__/pages/ProviderDetailPage.test.tsxweb/src/__tests__/utils/providers.test.tsweb/src/api/endpoints/providers.tsweb/src/api/types.tsweb/src/components/ui/provider-health-badge.stories.tsxweb/src/components/ui/provider-health-badge.tsxweb/src/hooks/useProviderDetailData.tsweb/src/pages/providers/ProviderCard.stories.tsxweb/src/pages/providers/ProviderCard.tsxweb/src/pages/providers/ProviderHealthMetrics.stories.tsxweb/src/pages/providers/ProviderHealthMetrics.tsxweb/src/pages/providers/ProviderModelList.stories.tsxweb/src/pages/providers/ProviderModelList.tsxweb/src/stores/providers.tsweb/src/utils/providers.ts
| it('returns EUR 0.00 for zero', () => { | ||
| expect(formatCost(0)).toContain('0.00') | ||
| }) |
There was a problem hiding this comment.
Cost formatter expectation appears currency-misaligned with upstream data.
These assertions encode EUR as the default presentation, but usage totals are sourced from USD-denominated fields (cost_usd). Unless there is explicit conversion before formatting, this bakes in incorrect currency semantics.
Also applies to: 249-251
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@web/src/__tests__/utils/providers.test.ts` around lines 240 - 242, The tests
assume EUR output but source amounts are USD; update the assertions to use USD
formatting or explicitly pass USD to the formatter. Locate the tests that call
formatCost (e.g., the spec with "returns EUR 0.00 for zero" and the other
failing block) and either change expect(formatCost(0)).toContain('0.00') to
assert the USD symbol/format (e.g., expect(...).toContain('$0.00')) or call
formatCost with an explicit currency argument like formatCost(0, 'USD') (or the
equivalent API your formatter exposes) so the test aligns with cost_usd
semantics.
Wire ProviderHealthProber into app lifecycle: instantiate in on_startup after config is resolved, pass discovery_policy_loader from ProviderManagementService, stop cleanly in on_shutdown. Record health failure when probe URL is disallowed by discovery policy (prevents providers staying UNKNOWN/stale). Add 'unknown' to VALID_HEALTH filter set in ProviderFilters.tsx. Show token/cost metrics in ProviderCard even when zero (consistent with calls/24h always-visible pattern). Clarify 24-hour scope in page-structure.md provider metrics description. Use NotBlankStr for internal _filter_records provider parameter to match public API contract. Add cost assertion to ProviderDetailPage health metrics test. Rejected findings: EUR default is correct per project convention (2 items), logging MemoryError before re-raise is unsafe, test fixture needs "ollama" to cover vendor-specific code path. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Actionable comments posted: 2
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (2)
web/src/pages/providers/ProviderFilters.tsx (1)
7-12:⚠️ Potential issue | 🟡 MinorAdd 'unknown' option to
HEALTH_OPTIONSfor consistency.
VALID_HEALTHnow includes'unknown', butHEALTH_OPTIONSlacks a corresponding dropdown entry. Users cannot filter for providers with unknown health status via the UI. Per PR objectives, the UNKNOWN status exists for providers with zero recent calls—this should be filterable.Proposed fix
const HEALTH_OPTIONS: { value: string; label: string }[] = [ { value: '', label: 'All health' }, { value: 'up', label: 'Up' }, { value: 'degraded', label: 'Degraded' }, { value: 'down', label: 'Down' }, + { value: 'unknown', label: 'Unknown' }, ]Also applies to: 20-20
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@web/src/pages/providers/ProviderFilters.tsx` around lines 7 - 12, HEALTH_OPTIONS is missing the 'unknown' entry even though VALID_HEALTH includes 'unknown'; update the HEALTH_OPTIONS constant to add a matching option (e.g., { value: 'unknown', label: 'Unknown' }) so the dropdown can filter providers with UNKNOWN health; ensure the new entry uses the same string 'unknown' as in VALID_HEALTH and is placed where you want it in the options order inside ProviderFilters (HEALTH_OPTIONS).web/src/pages/providers/ProviderCard.tsx (1)
52-70:⚠️ Potential issue | 🟡 MinorInconsistent zero-value handling between metrics.
Token and cost metrics are now shown even when zero (lines 61-70), but
calls_last_24h(line 56) is still hidden when zero. This creates visual inconsistency across provider cards.Consider either:
- Show all metrics consistently (including zero calls), or
- Hide all when zero (revert tokens/cost to
> 0checks)Option 1: Show all metrics consistently
- {health && health.calls_last_24h > 0 && ( + {health && ( <span className="text-xs text-text-muted"> {health.calls_last_24h} calls/24h </span> )}🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@web/src/pages/providers/ProviderCard.tsx` around lines 52 - 70, The metrics rendering is inconsistent: calls_last_24h is hidden when zero but tokens/cost are shown; make them consistent by always rendering the calls metric even when zero. In ProviderCard.tsx update the conditional that currently checks "health && health.calls_last_24h > 0" to "health" so the calls span always renders (use the existing span text "{health.calls_last_24h} calls/24h"); keep the existing formatTokenCount(health.total_tokens_24h) and formatCost(health.total_cost_24h) logic so all three metrics appear whenever health exists, including zero values.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@src/synthorg/providers/health_prober.py`:
- Around line 297-301: The code creates a new httpx.AsyncClient inside the
_probe_one function, losing connection pooling; change the implementation to
create and reuse a single AsyncClient instance stored on the prober object
(e.g., self._httpx_client) instead of constructing a client per call in
_probe_one, initialize it once (lazy or in the class __init__/startup method),
ensure proper shutdown/close (in a close/stop method or destructor), and update
all places using httpx.AsyncClient in this class to use self._httpx_client to
preserve connection pooling and efficiency.
In `@web/src/__tests__/pages/ProviderDetailPage.test.tsx`:
- Around line 93-102: Update the "renders model list when models present" test
to also assert that model capability badges are rendered: after setting
hookReturn with makeProvider and testModels (which has supports_tools: true and
supports_streaming: true), call renderDetail() and add assertions that the
capability labels (e.g., "Tools" and "Streaming") are present via
screen.getByText; ensure the assertions reference the same testModels fixture so
capabilities map to the rendered ProviderModelList output.
---
Outside diff comments:
In `@web/src/pages/providers/ProviderCard.tsx`:
- Around line 52-70: The metrics rendering is inconsistent: calls_last_24h is
hidden when zero but tokens/cost are shown; make them consistent by always
rendering the calls metric even when zero. In ProviderCard.tsx update the
conditional that currently checks "health && health.calls_last_24h > 0" to
"health" so the calls span always renders (use the existing span text
"{health.calls_last_24h} calls/24h"); keep the existing
formatTokenCount(health.total_tokens_24h) and formatCost(health.total_cost_24h)
logic so all three metrics appear whenever health exists, including zero values.
In `@web/src/pages/providers/ProviderFilters.tsx`:
- Around line 7-12: HEALTH_OPTIONS is missing the 'unknown' entry even though
VALID_HEALTH includes 'unknown'; update the HEALTH_OPTIONS constant to add a
matching option (e.g., { value: 'unknown', label: 'Unknown' }) so the dropdown
can filter providers with UNKNOWN health; ensure the new entry uses the same
string 'unknown' as in VALID_HEALTH and is placed where you want it in the
options order inside ProviderFilters (HEALTH_OPTIONS).
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI
Review profile: ASSERTIVE
Plan: Pro
Run ID: 5e5c488d-5a65-488e-855e-30936a398362
📒 Files selected for processing (7)
docs/design/page-structure.mdsrc/synthorg/api/app.pysrc/synthorg/budget/tracker.pysrc/synthorg/providers/health_prober.pyweb/src/__tests__/pages/ProviderDetailPage.test.tsxweb/src/pages/providers/ProviderCard.tsxweb/src/pages/providers/ProviderFilters.tsx
📜 Review details
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (7)
- GitHub Check: Dashboard Test
- GitHub Check: Test (Python 3.14)
- GitHub Check: Build Backend
- GitHub Check: Build Sandbox
- GitHub Check: Build Web
- GitHub Check: Dependency Review
- GitHub Check: Analyze (python)
🧰 Additional context used
📓 Path-based instructions (14)
**/*.{py,ts,tsx,md}
📄 CodeRabbit inference engine (CLAUDE.md)
Vendor-agnostic everywhere: NEVER use real vendor names (Anthropic, OpenAI, Claude, GPT, etc.) in project-owned code, docstrings, comments, tests, or config examples; use generic names: example-provider, example-large-001, etc.; vendor names only in: design spec, .claude/ files, third-party imports, provider presets
Files:
web/src/pages/providers/ProviderFilters.tsxdocs/design/page-structure.mdweb/src/__tests__/pages/ProviderDetailPage.test.tsxweb/src/pages/providers/ProviderCard.tsxsrc/synthorg/api/app.pysrc/synthorg/budget/tracker.pysrc/synthorg/providers/health_prober.py
web/src/**/*.{ts,tsx}
📄 CodeRabbit inference engine (CLAUDE.md)
See web/CLAUDE.md for web dashboard commands, design system, and component inventory; ALWAYS reuse existing components from web/src/components/ui/ before creating new ones; NEVER hardcode hex colors, font-family, or pixel spacing—use design tokens
web/src/**/*.{ts,tsx}: Use Tailwind semantic color classes (text-foreground,bg-card,text-accent,text-success,bg-danger, etc.) or CSS variables (var(--so-accent)). NEVER hardcode hex values in.tsx/.tsfiles
Usefont-sansorfont-monoTailwind classes (maps to Geist tokens). NEVER setfontFamilydirectly in code
Use density-aware spacing tokens (p-card,gap-section-gap,gap-grid-gap) or standard Tailwind spacing. NEVER hardcode pixel values for layout spacing
Use token variables for shadows and borders (var(--so-shadow-card-hover),border-border,border-bright). NEVER hardcode shadow or border styles
Do NOT usergba()with hardcoded values -- use design token variables
A PostToolUse hook (scripts/check_web_design_system.py) runs automatically on every Edit/Write toweb/src/files. It catches hardcoded hex colors, rgba values, font-family declarations, new components without Storybook stories, duplicate patterns, and complex.map()blocks. Fix all violations before proceeding -- do not suppress or ignore hook output
Files:
web/src/pages/providers/ProviderFilters.tsxweb/src/__tests__/pages/ProviderDetailPage.test.tsxweb/src/pages/providers/ProviderCard.tsx
web/src/**/*
📄 CodeRabbit inference engine (CLAUDE.md)
A PostToolUse hook (scripts/check_web_design_system.py) enforces design system rules on every Edit/Write to web/src/
Files:
web/src/pages/providers/ProviderFilters.tsxweb/src/__tests__/pages/ProviderDetailPage.test.tsxweb/src/pages/providers/ProviderCard.tsx
web/src/{components,pages}/**/*.{ts,tsx}
📄 CodeRabbit inference engine (web/CLAUDE.md)
ALWAYS reuse existing components from
web/src/components/ui/before creating new ones
Files:
web/src/pages/providers/ProviderFilters.tsxweb/src/pages/providers/ProviderCard.tsx
web/src/{components,pages}/**/*.tsx
📄 CodeRabbit inference engine (web/CLAUDE.md)
web/src/{components,pages}/**/*.tsx: UseStatusBadgecomponent for agent/task/system status indicators (colored dot + optional built-in label toggle)
UseMetricCardcomponent for numeric KPIs with sparkline, change badge, progress bar
UseSectionCardcomponent for titled card wrapper with icon and action slot
UseAgentCardcomponent for agent display: avatar, name, role, status, current task
UseAvatarcomponent for circular initials avatar with optionalborderColor?prop instead of rendering manually
UseToast/ToastContainercomponents for success/error/warning/info notifications with auto-dismiss queue (mountToastContaineronce in AppLayout)
UseSkeleton/SkeletonCard/SkeletonMetric/SkeletonTable/SkeletonTextcomponents for loading placeholders matching component shapes with shimmer animation
UseEmptyStatecomponent for no-data / no-results placeholder with icon, title, description, optional action button
UseErrorBoundarycomponent withlevelprop (page/section/component) for error handling with retry capability
UseConfirmDialogcomponent for confirmation modals withdefault/destructivevariants andloadingstate
UseInlineEditcomponent for click-to-edit text with Enter/Escape, validation, optimistic save with rollback
UseStaggerGroup/StaggerItemcomponents for card entrance stagger container with configurable delay
UseDrawercomponent withsideprop (left or right, default right), spring animation, focus trap, Escape-to-close, optional header (title),ariaLabelfor accessible name (one oftitleorariaLabelrequired), andcontentClassNameoverride
UseInputFieldcomponent for labeled text input with error/hint display, optional multiline textarea mode
UseSelectFieldcomponent for labeled select dropdown with error/hint and placeholder support
UseSliderFieldcomponent for labeled range slider with custom value formatter and aria-live display
UseToggleFieldcomp...
Files:
web/src/pages/providers/ProviderFilters.tsxweb/src/pages/providers/ProviderCard.tsx
web/src/{components,pages,layout}/**/*.tsx
📄 CodeRabbit inference engine (web/CLAUDE.md)
web/src/{components,pages,layout}/**/*.tsx: MountCommandPalettecomponent once in AppLayout; register commands viauseCommandPalettehook for global Cmd+K search
UseMobileUnsupportedOverlaycomponent for full-screen overlay at<768pxviewports directing users to desktop or CLI; component self-manages visibility viauseBreakpoint
Files:
web/src/pages/providers/ProviderFilters.tsxweb/src/pages/providers/ProviderCard.tsx
web/src/{pages,router}/**/*.tsx
📄 CodeRabbit inference engine (web/CLAUDE.md)
Use
AnimatedPresencecomponent as page transition wrapper with Framer Motion AnimatePresence keyed by route
Files:
web/src/pages/providers/ProviderFilters.tsxweb/src/pages/providers/ProviderCard.tsx
web/{tsconfig.json,src/**/*.{ts,tsx}}
📄 CodeRabbit inference engine (web/CLAUDE.md)
In TypeScript 6.0,
noUncheckedSideEffectImportsdefaults totrue; CSS side-effect imports need type declarations (Vite's/// <reference types="vite/client" />covers this)
Files:
web/src/pages/providers/ProviderFilters.tsxweb/src/__tests__/pages/ProviderDetailPage.test.tsxweb/src/pages/providers/ProviderCard.tsx
web/src/pages/**/*.tsx
📄 CodeRabbit inference engine (web/CLAUDE.md)
Place page-scoped sub-components in
pages/<page-name>/subdirs (e.g., tasks/, org-edit/, settings/) alongside the main page component
Files:
web/src/pages/providers/ProviderFilters.tsxweb/src/pages/providers/ProviderCard.tsx
web/src/__tests__/**/*.{test,spec}.{ts,tsx}
📄 CodeRabbit inference engine (web/CLAUDE.md)
Mirror test file structure with
web/src/__tests__/directory that matchesweb/src/structure for Vitest unit + property tests
Files:
web/src/__tests__/pages/ProviderDetailPage.test.tsx
**/*.py
📄 CodeRabbit inference engine (CLAUDE.md)
**/*.py: Nofrom __future__ import annotations—Python 3.14 has PEP 649
PEP 758 except syntax: useexcept A, B:(no parentheses)—ruff enforces this on Python 3.14
Type hints: all public functions, mypy strict mode
Docstrings: Google style, required on public classes/functions (enforced by ruff D rules)
Immutability: create new objects, never mutate existing ones. For non-Pydantic internal collections (registries, BaseTool), use copy.deepcopy() at construction + MappingProxyType wrapping for read-only enforcement
Config vs runtime state: frozen Pydantic models for config/identity; separate mutable-via-copy models (using model_copy(update=...)) for runtime state that evolves. Never mix static config fields with mutable runtime fields in one model
Models: Pydantic v2 (BaseModel, model_validator, computed_field, ConfigDict). Adopted conventions: use allow_inf_nan=False in all ConfigDict declarations; use@computed_fieldfor derived values; use NotBlankStr for all identifier/name fields
Async concurrency: prefer asyncio.TaskGroup for fan-out/fan-in parallel operations in new code; prefer structured concurrency over bare create_task
Line length: 88 characters (ruff)
Functions: < 50 lines, files < 800 lines
Errors: handle explicitly, never silently swallow
Validate: at system boundaries (user input, external APIs, config files)
Files:
src/synthorg/api/app.pysrc/synthorg/budget/tracker.pysrc/synthorg/providers/health_prober.py
src/synthorg/**/*.py
📄 CodeRabbit inference engine (CLAUDE.md)
src/synthorg/**/*.py: Every module with business logic MUST have: from synthorg.observability import get_logger then logger = get_logger(name)
Variable name: always logger (not _logger, not log)
Event names: always use constants from domain-specific modules under synthorg.observability.events; import directly from synthorg.observability.events.
Structured kwargs: always logger.info(EVENT, key=value)—never logger.info('msg %s', val)
All error paths must log at WARNING or ERROR with context before raising
All state transitions must log at INFO
DEBUG for object creation, internal flow, entry/exit of key functions
All provider calls go through BaseCompletionProvider which applies retry + rate limiting automatically; never implement retry logic in driver subclasses or calling code
Files:
src/synthorg/api/app.pysrc/synthorg/budget/tracker.pysrc/synthorg/providers/health_prober.py
src/synthorg/**/!(setup|sinks).py
📄 CodeRabbit inference engine (CLAUDE.md)
Never use import logging / logging.getLogger() / print() in application code; exception: observability/setup.py and observability/sinks.py may use stdlib logging and print() for bootstrap code
Files:
src/synthorg/api/app.pysrc/synthorg/budget/tracker.pysrc/synthorg/providers/health_prober.py
src/synthorg/providers/**/*.py
📄 CodeRabbit inference engine (CLAUDE.md)
RetryConfig and RateLimiterConfig are set per-provider in ProviderConfig
Files:
src/synthorg/providers/health_prober.py
🧠 Learnings (36)
📓 Common learnings
Learnt from: CR
Repo: Aureliolo/synthorg PR: 0
File: web/CLAUDE.md:0-0
Timestamp: 2026-03-30T16:37:21.464Z
Learning: Applies to web/src/{components,pages}/**/*.tsx : Use `ProviderHealthBadge` component for provider health status indicator (up/degraded/down colored dot + optional label)
Learnt from: CR
Repo: Aureliolo/synthorg PR: 0
File: web/CLAUDE.md:0-0
Timestamp: 2026-03-30T16:37:21.464Z
Learning: Applies to web/src/{components,pages}/**/*.tsx : Use `TokenUsageBar` component for segmented horizontal meter bar for token usage (multi-segment with auto-colors, `role="meter"`, animated transitions)
📚 Learning: 2026-03-30T16:37:21.464Z
Learnt from: CR
Repo: Aureliolo/synthorg PR: 0
File: web/CLAUDE.md:0-0
Timestamp: 2026-03-30T16:37:21.464Z
Learning: Applies to web/src/{components,pages}/**/*.tsx : Use `ProviderHealthBadge` component for provider health status indicator (up/degraded/down colored dot + optional label)
Applied to files:
web/src/pages/providers/ProviderFilters.tsxdocs/design/page-structure.mdweb/src/__tests__/pages/ProviderDetailPage.test.tsxweb/src/pages/providers/ProviderCard.tsx
📚 Learning: 2026-03-20T08:28:32.845Z
Learnt from: CR
Repo: Aureliolo/synthorg PR: 0
File: CLAUDE.md:0-0
Timestamp: 2026-03-20T08:28:32.845Z
Learning: Applies to src/synthorg/providers/**/*.py : Providers: LLM provider abstraction (LiteLLM adapter), auth types (api_key/oauth/custom_header/none), presets (PROVIDER_PRESETS), runtime CRUD (ProviderManagementService with asyncio.Lock serialization), hot-reload via AppState swap.
Applied to files:
docs/design/page-structure.mdsrc/synthorg/api/app.pysrc/synthorg/providers/health_prober.py
📚 Learning: 2026-03-15T18:28:13.207Z
Learnt from: CR
Repo: Aureliolo/synthorg PR: 0
File: CLAUDE.md:0-0
Timestamp: 2026-03-15T18:28:13.207Z
Learning: Applies to tests/**/*.py : Tests must use test-provider, test-small-001, etc. for vendor-agnostic test data.
Applied to files:
web/src/__tests__/pages/ProviderDetailPage.test.tsxsrc/synthorg/providers/health_prober.py
📚 Learning: 2026-03-30T16:37:21.464Z
Learnt from: CR
Repo: Aureliolo/synthorg PR: 0
File: web/CLAUDE.md:0-0
Timestamp: 2026-03-30T16:37:21.464Z
Learning: Applies to web/src/{components,pages}/**/*.tsx : Use `MetricCard` component for numeric KPIs with sparkline, change badge, progress bar
Applied to files:
web/src/pages/providers/ProviderCard.tsx
📚 Learning: 2026-03-30T16:37:21.464Z
Learnt from: CR
Repo: Aureliolo/synthorg PR: 0
File: web/CLAUDE.md:0-0
Timestamp: 2026-03-30T16:37:21.464Z
Learning: Applies to web/src/{components,pages}/**/*.tsx : Use `TokenUsageBar` component for segmented horizontal meter bar for token usage (multi-segment with auto-colors, `role="meter"`, animated transitions)
Applied to files:
web/src/pages/providers/ProviderCard.tsx
📚 Learning: 2026-03-30T16:37:21.464Z
Learnt from: CR
Repo: Aureliolo/synthorg PR: 0
File: web/CLAUDE.md:0-0
Timestamp: 2026-03-30T16:37:21.464Z
Learning: Applies to web/src/{components,pages}/**/*.tsx : Do NOT create metric displays with `text-metric font-bold` -- use `<MetricCard>`
Applied to files:
web/src/pages/providers/ProviderCard.tsx
📚 Learning: 2026-03-30T16:37:21.464Z
Learnt from: CR
Repo: Aureliolo/synthorg PR: 0
File: web/CLAUDE.md:0-0
Timestamp: 2026-03-30T16:37:21.464Z
Learning: Applies to web/src/{components,pages}/**/*.tsx : Use `AgentCard` component for agent display: avatar, name, role, status, current task
Applied to files:
web/src/pages/providers/ProviderCard.tsx
📚 Learning: 2026-03-30T16:37:21.464Z
Learnt from: CR
Repo: Aureliolo/synthorg PR: 0
File: web/CLAUDE.md:0-0
Timestamp: 2026-03-30T16:37:21.464Z
Learning: Applies to web/src/{components,pages}/**/*.tsx : Use `SectionCard` component for titled card wrapper with icon and action slot
Applied to files:
web/src/pages/providers/ProviderCard.tsx
📚 Learning: 2026-03-30T16:37:21.464Z
Learnt from: CR
Repo: Aureliolo/synthorg PR: 0
File: web/CLAUDE.md:0-0
Timestamp: 2026-03-30T16:37:21.464Z
Learning: Applies to web/src/**/*.{ts,tsx} : Use token variables for shadows and borders (`var(--so-shadow-card-hover)`, `border-border`, `border-bright`). NEVER hardcode shadow or border styles
Applied to files:
web/src/pages/providers/ProviderCard.tsx
📚 Learning: 2026-03-30T16:37:21.464Z
Learnt from: CR
Repo: Aureliolo/synthorg PR: 0
File: web/CLAUDE.md:0-0
Timestamp: 2026-03-30T16:37:21.464Z
Learning: Applies to web/src/**/*.{ts,tsx} : Use density-aware spacing tokens (`p-card`, `gap-section-gap`, `gap-grid-gap`) or standard Tailwind spacing. NEVER hardcode pixel values for layout spacing
Applied to files:
web/src/pages/providers/ProviderCard.tsx
📚 Learning: 2026-03-20T11:18:48.128Z
Learnt from: CR
Repo: Aureliolo/synthorg PR: 0
File: CLAUDE.md:0-0
Timestamp: 2026-03-20T11:18:48.128Z
Learning: Applies to src/synthorg/**/*.py : Set `RetryConfig` and `RateLimiterConfig` per-provider in `ProviderConfig`.
Applied to files:
src/synthorg/api/app.pysrc/synthorg/providers/health_prober.py
📚 Learning: 2026-03-17T06:30:14.180Z
Learnt from: CR
Repo: Aureliolo/synthorg PR: 0
File: CLAUDE.md:0-0
Timestamp: 2026-03-17T06:30:14.180Z
Learning: Applies to src/synthorg/persistence/**/*.py : Persistence uses pluggable PersistenceBackend protocol. SQLite is the initial backend. Settings use SettingsRepository (namespaced settings CRUD).
Applied to files:
src/synthorg/api/app.py
📚 Learning: 2026-03-17T22:08:13.456Z
Learnt from: CR
Repo: Aureliolo/synthorg PR: 0
File: CLAUDE.md:0-0
Timestamp: 2026-03-17T22:08:13.456Z
Learning: Persistence backend: pluggable PersistenceBackend protocol in `src/synthorg/persistence/`, SQLite initial, SettingsRepository (namespaced settings CRUD).
Applied to files:
src/synthorg/api/app.py
📚 Learning: 2026-03-16T06:24:56.341Z
Learnt from: CR
Repo: Aureliolo/synthorg PR: 0
File: CLAUDE.md:0-0
Timestamp: 2026-03-16T06:24:56.341Z
Learning: Applies to src/synthorg/memory/**/*.py : Use MemoryBackend protocol with pluggable backends (Mem0 adapter available at backends/mem0/) for persistent agent memory
Applied to files:
src/synthorg/api/app.py
📚 Learning: 2026-03-19T07:12:14.508Z
Learnt from: CR
Repo: Aureliolo/synthorg PR: 0
File: CLAUDE.md:0-0
Timestamp: 2026-03-19T07:12:14.508Z
Learning: Applies to src/synthorg/memory/**/*.py : Memory package (memory/): pluggable MemoryBackend protocol, backends/ (Mem0 adapter), retrieval pipeline (ranking, RRF fusion, injection, formatting, non-inferable filtering), shared org memory (org/), consolidation/archival (density-aware: DensityClassifier, AbstractiveSummarizer, ExtractivePreserver, DualModeConsolidationStrategy)
Applied to files:
src/synthorg/api/app.py
📚 Learning: 2026-03-19T07:13:44.964Z
Learnt from: CR
Repo: Aureliolo/synthorg PR: 0
File: CLAUDE.md:0-0
Timestamp: 2026-03-19T07:13:44.964Z
Learning: Applies to src/synthorg/budget/**/*.py : Budget package (budget/): cost tracking, budget enforcement (pre-flight/in-flight checks, auto-downgrade), billing periods, cost tiers, quota/subscription tracking, CFO cost optimization (anomaly detection, efficiency analysis, downgrade recommendations, approval decisions), spending reports, budget errors (BudgetExhaustedError, DailyLimitExceededError, QuotaExhaustedError)
Applied to files:
src/synthorg/budget/tracker.py
📚 Learning: 2026-03-17T06:30:14.180Z
Learnt from: CR
Repo: Aureliolo/synthorg PR: 0
File: CLAUDE.md:0-0
Timestamp: 2026-03-17T06:30:14.180Z
Learning: Applies to src/synthorg/budget/**/*.py : Budget tracking includes pre-flight/in-flight checks, auto-downgrade, billing periods, cost tiers, quota/subscription. CFO includes anomaly detection, efficiency analysis, downgrade recommendations.
Applied to files:
src/synthorg/budget/tracker.py
📚 Learning: 2026-03-17T22:08:13.456Z
Learnt from: CR
Repo: Aureliolo/synthorg PR: 0
File: CLAUDE.md:0-0
Timestamp: 2026-03-17T22:08:13.456Z
Learning: Budget: Cost tracking, budget enforcement (pre-flight/in-flight checks, auto-downgrade), billing periods, cost tiers, quota/subscription tracking, CFO cost optimization (anomaly detection, efficiency analysis, downgrade recommendations, approval decisions), spending reports, budget errors (BudgetExhaustedError, DailyLimitExceededError, QuotaExhaustedError).
Applied to files:
src/synthorg/budget/tracker.py
📚 Learning: 2026-03-20T21:44:04.528Z
Learnt from: CR
Repo: Aureliolo/synthorg PR: 0
File: CLAUDE.md:0-0
Timestamp: 2026-03-20T21:44:04.528Z
Learning: Applies to src/synthorg/**/*.py : Use `NotBlankStr` from `core.types` for all identifier/name fields (including optional and tuple variants) instead of manual whitespace validators
Applied to files:
src/synthorg/budget/tracker.py
📚 Learning: 2026-03-15T18:28:13.207Z
Learnt from: CR
Repo: Aureliolo/synthorg PR: 0
File: CLAUDE.md:0-0
Timestamp: 2026-03-15T18:28:13.207Z
Learning: Applies to {src,tests,web,cli,site}/**/*.{py,ts,tsx,go,astro} : Vendor-agnostic everywhere: NEVER use real vendor names (Anthropic, OpenAI, Claude, GPT, etc.) in project-owned code, docstrings, comments, tests, or config examples. Use generic names: example-provider, example-large-001, example-medium-001, example-small-001. Vendor names may only appear in: (1) Operations design page provider list (docs/design/operations.md), (2) .claude/ skill/agent files, (3) third-party import paths/module names.
Applied to files:
src/synthorg/budget/tracker.pysrc/synthorg/providers/health_prober.py
📚 Learning: 2026-03-17T22:08:13.456Z
Learnt from: CR
Repo: Aureliolo/synthorg PR: 0
File: CLAUDE.md:0-0
Timestamp: 2026-03-17T22:08:13.456Z
Learning: Applies to src/synthorg/**/*.py : Use Pydantic v2 conventions: `BaseModel`, `model_validator`, `computed_field`, `ConfigDict`. For derived values use `computed_field` instead of storing + validating redundant fields. Use `NotBlankStr` (from `core.types`) for all identifier/name fields — including optional (`NotBlankStr | None`) and tuple (`tuple[NotBlankStr, ...]`) variants — instead of manual whitespace validators.
Applied to files:
src/synthorg/budget/tracker.py
📚 Learning: 2026-03-15T19:14:27.144Z
Learnt from: CR
Repo: Aureliolo/synthorg PR: 0
File: CLAUDE.md:0-0
Timestamp: 2026-03-15T19:14:27.144Z
Learning: Applies to src/synthorg/**/*.py : Use Pydantic v2 with adopted conventions: use computed_field for derived values instead of storing + validating redundant fields; use NotBlankStr from core.types for all identifier/name fields (including optional and tuple variants) instead of manual whitespace validators.
Applied to files:
src/synthorg/budget/tracker.py
📚 Learning: 2026-03-30T16:36:12.059Z
Learnt from: CR
Repo: Aureliolo/synthorg PR: 0
File: CLAUDE.md:0-0
Timestamp: 2026-03-30T16:36:12.059Z
Learning: Applies to **/*.{py,ts,tsx,md} : Vendor-agnostic everywhere: NEVER use real vendor names (Anthropic, OpenAI, Claude, GPT, etc.) in project-owned code, docstrings, comments, tests, or config examples; use generic names: example-provider, example-large-001, etc.; vendor names only in: design spec, .claude/ files, third-party imports, provider presets
Applied to files:
src/synthorg/budget/tracker.pysrc/synthorg/providers/health_prober.py
📚 Learning: 2026-03-16T07:22:28.134Z
Learnt from: CR
Repo: Aureliolo/synthorg PR: 0
File: CLAUDE.md:0-0
Timestamp: 2026-03-16T07:22:28.134Z
Learning: Applies to **/*.py : Handle errors explicitly; never silently swallow exceptions
Applied to files:
src/synthorg/providers/health_prober.py
📚 Learning: 2026-03-14T15:43:05.601Z
Learnt from: CR
Repo: Aureliolo/synthorg PR: 0
File: CLAUDE.md:0-0
Timestamp: 2026-03-14T15:43:05.601Z
Learning: Applies to **/*.py : Handle errors explicitly, never silently swallow exceptions
Applied to files:
src/synthorg/providers/health_prober.py
📚 Learning: 2026-03-20T21:44:04.528Z
Learnt from: CR
Repo: Aureliolo/synthorg PR: 0
File: CLAUDE.md:0-0
Timestamp: 2026-03-20T21:44:04.528Z
Learning: Applies to **/*.py : Use `except A, B:` syntax (without parentheses) per PEP 758 for exception handling in Python 3.14
Applied to files:
src/synthorg/providers/health_prober.py
📚 Learning: 2026-03-16T07:22:28.134Z
Learnt from: CR
Repo: Aureliolo/synthorg PR: 0
File: CLAUDE.md:0-0
Timestamp: 2026-03-16T07:22:28.134Z
Learning: Applies to **/*.py : Use `except A, B:` syntax (no parentheses) for exception handling — PEP 758 exception syntax enforced by ruff on Python 3.14
Applied to files:
src/synthorg/providers/health_prober.py
📚 Learning: 2026-03-14T16:18:57.267Z
Learnt from: CR
Repo: Aureliolo/synthorg PR: 0
File: CLAUDE.md:0-0
Timestamp: 2026-03-14T16:18:57.267Z
Learning: Applies to **/*.py : Use PEP 758 except syntax with `except A, B:` (no parentheses) for multiple exceptions—ruff enforces this on Python 3.14.
Applied to files:
src/synthorg/providers/health_prober.py
📚 Learning: 2026-03-14T16:18:57.267Z
Learnt from: CR
Repo: Aureliolo/synthorg PR: 0
File: CLAUDE.md:0-0
Timestamp: 2026-03-14T16:18:57.267Z
Learning: Applies to **/*.py : Handle errors explicitly—never silently swallow exceptions.
Applied to files:
src/synthorg/providers/health_prober.py
📚 Learning: 2026-03-19T11:33:01.580Z
Learnt from: CR
Repo: Aureliolo/synthorg PR: 0
File: CLAUDE.md:0-0
Timestamp: 2026-03-19T11:33:01.580Z
Learning: Applies to web/**/*.vue : Never use real vendor names (Anthropic, OpenAI, Claude, GPT, etc.) in dashboard code — use generic names: `example-provider`, `example-large-001`, etc.
Applied to files:
src/synthorg/providers/health_prober.py
📚 Learning: 2026-03-17T22:08:13.456Z
Learnt from: CR
Repo: Aureliolo/synthorg PR: 0
File: CLAUDE.md:0-0
Timestamp: 2026-03-17T22:08:13.456Z
Learning: Applies to src/synthorg/**/*.py : Prefer `asyncio.TaskGroup` for fan-out/fan-in parallel operations in new code (e.g. multiple tool invocations, parallel agent calls). Prefer structured concurrency over bare `create_task`. Existing code is being migrated incrementally.
Applied to files:
src/synthorg/providers/health_prober.py
📚 Learning: 2026-03-17T18:52:05.142Z
Learnt from: CR
Repo: Aureliolo/synthorg PR: 0
File: CLAUDE.md:0-0
Timestamp: 2026-03-17T18:52:05.142Z
Learning: Applies to **/*.py : Async concurrency: prefer asyncio.TaskGroup for fan-out/fan-in parallel operations in new code (e.g. multiple tool invocations, parallel agent calls). Prefer structured concurrency over bare create_task.
Applied to files:
src/synthorg/providers/health_prober.py
📚 Learning: 2026-03-30T16:36:12.059Z
Learnt from: CR
Repo: Aureliolo/synthorg PR: 0
File: CLAUDE.md:0-0
Timestamp: 2026-03-30T16:36:12.059Z
Learning: Applies to **/*.py : Async concurrency: prefer asyncio.TaskGroup for fan-out/fan-in parallel operations in new code; prefer structured concurrency over bare create_task
Applied to files:
src/synthorg/providers/health_prober.py
📚 Learning: 2026-03-20T11:18:48.128Z
Learnt from: CR
Repo: Aureliolo/synthorg PR: 0
File: CLAUDE.md:0-0
Timestamp: 2026-03-20T11:18:48.128Z
Learning: Applies to src/synthorg/**/*.py : Retryable errors (`is_retryable=True`): `RateLimitError`, `ProviderTimeoutError`, `ProviderConnectionError`, `ProviderInternalError`. Non-retryable errors raise immediately without retry.
Applied to files:
src/synthorg/providers/health_prober.py
📚 Learning: 2026-03-26T15:18:16.848Z
Learnt from: CR
Repo: Aureliolo/synthorg PR: 0
File: CLAUDE.md:0-0
Timestamp: 2026-03-26T15:18:16.848Z
Learning: Applies to src/synthorg/api/**/*.py : Litestar API must include setup wizard, auth/, auto-wiring, and lifecycle management
Applied to files:
src/synthorg/providers/health_prober.py
🔇 Additional comments (15)
docs/design/page-structure.md (1)
118-118: Good clarification on provider health metrics scope.The updated wording is precise (
24-hour) and matches the backend/UI behavior described in this PR.web/src/__tests__/pages/ProviderDetailPage.test.tsx (3)
5-5: LGTM! Well-typed model fixtures with capability flags.The introduction of the typed
testModelsconstant withProviderModelResponse[]properly aligns with the new DTO structure including capability flags (supports_tools,supports_vision,supports_streaming).Also applies to: 40-42
22-22: LGTM! Vendor-agnostic test data.All provider identifiers correctly changed from real vendor names to
test-provider, complying with the vendor-agnostic policy. As per coding guidelines: "Tests must use test-provider, test-small-001, etc. for vendor-agnostic test data."Also applies to: 54-54, 83-83, 90-90, 94-94, 105-105, 128-128
116-124: LGTM! New health metrics properly tested.The assertions correctly verify the formatted output:
total_tokens_24h: 50000→"50.0K"(viaformatTokenCount)total_cost_24h: 1.25→ matches/1\.25/(viaformatCost, currency-agnostic regex)This aligns with the formatting functions in
web/src/utils/providers.tsand theProviderHealthMetricscomponent rendering.src/synthorg/budget/tracker.py (5)
40-41: Good import wiring for new provider-usage path.Event and
NotBlankStrimports are consistent with the existing observability and typing patterns.Also applies to: 61-62
69-73:ProviderUsageSummaryis clean and fit-for-purpose.Strongly typed token/cost fields make the new aggregation contract explicit.
248-293: Provider filtering is correctly threaded throughget_records.The filter is surfaced in the API, logged, and forwarded to the shared filter helper consistently.
294-335:get_provider_usageimplementation looks solid.Validation, structured logging, empty-result handling, and aggregation behavior are all correctly implemented.
626-645: Provider predicate integration in_filter_recordsis correct.The new condition composes safely with existing agent/task/time filters.
src/synthorg/providers/health_prober.py (3)
48-66: Hard-coded vendor name "ollama" violates vendor-agnostic guideline.The conditional on line 64 references
"ollama"directly. Per past review feedback and coding guidelines, vendor names should not appear in project-owned code. Push ping-path selection into provider config or preset metadata (e.g., aping_pathfield inProviderConfig).Proposed approach
-def _build_ping_url(base_url: str, litellm_provider: str | None) -> str: +def _build_ping_url(base_url: str, ping_path: str | None) -> str: """Build a lightweight ping URL for a provider. Uses the cheapest possible endpoint -- no model loading. - Providers whose ``litellm_provider`` is ``"ollama"`` (or whose - URL contains the default port ``:11434``) use the root URL; - all others append ``/models``. + When ``ping_path`` is empty or ``None``, returns the root URL; + otherwise appends the configured path. Args: base_url: Provider base URL. - litellm_provider: LiteLLM provider identifier for path selection. + ping_path: Optional path to append (e.g. ``"/models"``). Returns: URL to ping. """ stripped = base_url.rstrip("/") - if litellm_provider == "ollama" or ":11434" in stripped: - return stripped # Root URL returns a liveness string - return f"{stripped}/models" + if not ping_path: + return stripped + return f"{stripped}/{ping_path.lstrip('/')}"Then add
ping_path: str | None = "/models"toProviderConfigand set it toNoneor""in presets for providers that use the root URL.As per coding guidelines, "Vendor-agnostic everywhere: NEVER use real vendor names (Anthropic, OpenAI, Claude, GPT, etc.) in project-owned code".
171-189: LGTM!The run loop correctly handles shutdown via
_stop_event, re-raises fatal errors (MemoryError,RecursionError,CancelledError), and useswait_forwith timeout for interruptible sleep. The PEP 758 exception syntax is correct.
191-234: LGTM!Discovery policy blocks are now recorded as failed health records (lines 208-216), addressing the prior review concern. The parallel probing with
asyncio.TaskGroupfollows the structured concurrency guideline.src/synthorg/api/app.py (2)
375-387: LGTM!The prober is correctly wired:
- Conditionally created only when required dependencies (
provider_health_tracker,config_resolver) exist- Discovery policy loader correctly passed from
provider_managementwhen available- Started after config resolution and agent bootstrap
397-403: LGTM!Proper shutdown handling using
_try_stophelper that logs but swallows exceptions (except fatal ones), allowing subsequent cleanup steps to proceed.web/src/pages/providers/ProviderCard.tsx (1)
6-6: LGTM!Token and cost metrics are now rendered even when zero, addressing the past review concern. The formatters handle zero values gracefully (
'0'for tokens,'€0.00'for cost per the utility implementations).Also applies to: 61-70
| try: | ||
| async with httpx.AsyncClient( | ||
| timeout=_PROBE_TIMEOUT_SECONDS, | ||
| follow_redirects=False, | ||
| ) as client: |
There was a problem hiding this comment.
🧹 Nitpick | 🔵 Trivial
Consider reusing httpx.AsyncClient across probes.
Creating a new client per probe loses connection pooling benefits. For a background service that probes multiple providers in parallel, a shared client instance would be more efficient.
Suggested approach
Store a client as an instance attribute and reuse it across _probe_one calls:
class ProviderHealthProber:
__slots__ = (
+ "_client",
"_config_resolver",
# ...
)
def __init__(self, ...) -> None:
# ...
+ self._client: httpx.AsyncClient | None = None
async def start(self) -> None:
+ self._client = httpx.AsyncClient(
+ timeout=_PROBE_TIMEOUT_SECONDS,
+ follow_redirects=False,
+ )
# ...
async def stop(self) -> None:
+ if self._client is not None:
+ await self._client.aclose()
+ self._client = None
# ...
- `@staticmethod`
- async def _execute_probe(
+ async def _execute_probe(
+ self,
url: str,
headers: dict[str, str],
) -> tuple[float, bool, str | None]:
# ...
- async with httpx.AsyncClient(...) as client:
- resp = await client.get(url, headers=headers)
+ if self._client is None:
+ raise RuntimeError("Prober not started")
+ resp = await self._client.get(url, headers=headers)🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@src/synthorg/providers/health_prober.py` around lines 297 - 301, The code
creates a new httpx.AsyncClient inside the _probe_one function, losing
connection pooling; change the implementation to create and reuse a single
AsyncClient instance stored on the prober object (e.g., self._httpx_client)
instead of constructing a client per call in _probe_one, initialize it once
(lazy or in the class __init__/startup method), ensure proper shutdown/close (in
a close/stop method or destructor), and update all places using
httpx.AsyncClient in this class to use self._httpx_client to preserve connection
pooling and efficiency.
Make ProviderCard metrics consistent: always show calls/24h, tokens, and cost when health exists (even when zero). Add 'unknown' option to HEALTH_OPTIONS dropdown in ProviderFilters to match VALID_HEALTH set. Assert capability badges (tools, stream) render in ProviderDetailPage model list test. Rejected: reusing httpx.AsyncClient in 30-min prober provides zero connection pooling benefit across different hosts. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
web/src/__tests__/pages/ProviderDetailPage.test.tsx (1)
106-127: 🧹 Nitpick | 🔵 TrivialAdd one explicit test for
health_status: 'unknown'.Given this PR introduces unknown health handling, add a dedicated case to lock that rendering path and avoid regressions.
Suggested test addition
+ it('renders unknown health status when provider has no recent calls', () => { + const provider = makeProvider('test-provider') + hookReturn = { + ...defaultReturn, + provider, + models: [], + health: { + last_check_timestamp: null, + avg_response_time_ms: null, + error_rate_percent_24h: 0, + calls_last_24h: 0, + health_status: 'unknown', + total_tokens_24h: 0, + total_cost_24h: 0, + }, + } + renderDetail() + expect(screen.getByText(/unknown/i)).toBeInTheDocument() + })🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@web/src/__tests__/pages/ProviderDetailPage.test.tsx` around lines 106 - 127, Add a new test in ProviderDetailPage.test.tsx that sets hookReturn (like the existing test) to use makeProvider('test-provider') and a health object with health_status: 'unknown', then call renderDetail() and assert the UI shows the unknown health indicator (e.g., expect(screen.getByText(/unknown/i)).toBeInTheDocument()) to lock the unknown rendering path; reuse the same pattern/variables (hookReturn, defaultReturn, renderDetail) as in the other health tests so the test focuses only on the health_status:'unknown' case.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Outside diff comments:
In `@web/src/__tests__/pages/ProviderDetailPage.test.tsx`:
- Around line 106-127: Add a new test in ProviderDetailPage.test.tsx that sets
hookReturn (like the existing test) to use makeProvider('test-provider') and a
health object with health_status: 'unknown', then call renderDetail() and assert
the UI shows the unknown health indicator (e.g.,
expect(screen.getByText(/unknown/i)).toBeInTheDocument()) to lock the unknown
rendering path; reuse the same pattern/variables (hookReturn, defaultReturn,
renderDetail) as in the other health tests so the test focuses only on the
health_status:'unknown' case.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI
Review profile: ASSERTIVE
Plan: Pro
Run ID: 3f899c9d-ae13-4496-9d5d-33890d2c6e05
📒 Files selected for processing (3)
web/src/__tests__/pages/ProviderDetailPage.test.tsxweb/src/pages/providers/ProviderCard.tsxweb/src/pages/providers/ProviderFilters.tsx
📜 Review details
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (7)
- GitHub Check: Dashboard Test
- GitHub Check: Test (Python 3.14)
- GitHub Check: Build Sandbox
- GitHub Check: Build Web
- GitHub Check: Build Backend
- GitHub Check: Analyze (python)
- GitHub Check: Dependency Review
🧰 Additional context used
📓 Path-based instructions (10)
**/*.{py,ts,tsx,md}
📄 CodeRabbit inference engine (CLAUDE.md)
Vendor-agnostic everywhere: NEVER use real vendor names (Anthropic, OpenAI, Claude, GPT, etc.) in project-owned code, docstrings, comments, tests, or config examples; use generic names: example-provider, example-large-001, etc.; vendor names only in: design spec, .claude/ files, third-party imports, provider presets
Files:
web/src/pages/providers/ProviderFilters.tsxweb/src/pages/providers/ProviderCard.tsxweb/src/__tests__/pages/ProviderDetailPage.test.tsx
web/src/**/*.{ts,tsx}
📄 CodeRabbit inference engine (CLAUDE.md)
See web/CLAUDE.md for web dashboard commands, design system, and component inventory; ALWAYS reuse existing components from web/src/components/ui/ before creating new ones; NEVER hardcode hex colors, font-family, or pixel spacing—use design tokens
web/src/**/*.{ts,tsx}: Use Tailwind semantic color classes (text-foreground,bg-card,text-accent,text-success,bg-danger, etc.) or CSS variables (var(--so-accent)). NEVER hardcode hex values in.tsx/.tsfiles
Usefont-sansorfont-monoTailwind classes (maps to Geist tokens). NEVER setfontFamilydirectly in code
Use density-aware spacing tokens (p-card,gap-section-gap,gap-grid-gap) or standard Tailwind spacing. NEVER hardcode pixel values for layout spacing
Use token variables for shadows and borders (var(--so-shadow-card-hover),border-border,border-bright). NEVER hardcode shadow or border styles
Do NOT usergba()with hardcoded values -- use design token variables
A PostToolUse hook (scripts/check_web_design_system.py) runs automatically on every Edit/Write toweb/src/files. It catches hardcoded hex colors, rgba values, font-family declarations, new components without Storybook stories, duplicate patterns, and complex.map()blocks. Fix all violations before proceeding -- do not suppress or ignore hook output
Files:
web/src/pages/providers/ProviderFilters.tsxweb/src/pages/providers/ProviderCard.tsxweb/src/__tests__/pages/ProviderDetailPage.test.tsx
web/src/**/*
📄 CodeRabbit inference engine (CLAUDE.md)
A PostToolUse hook (scripts/check_web_design_system.py) enforces design system rules on every Edit/Write to web/src/
Files:
web/src/pages/providers/ProviderFilters.tsxweb/src/pages/providers/ProviderCard.tsxweb/src/__tests__/pages/ProviderDetailPage.test.tsx
web/src/{components,pages}/**/*.{ts,tsx}
📄 CodeRabbit inference engine (web/CLAUDE.md)
ALWAYS reuse existing components from
web/src/components/ui/before creating new ones
Files:
web/src/pages/providers/ProviderFilters.tsxweb/src/pages/providers/ProviderCard.tsx
web/src/{components,pages}/**/*.tsx
📄 CodeRabbit inference engine (web/CLAUDE.md)
web/src/{components,pages}/**/*.tsx: UseStatusBadgecomponent for agent/task/system status indicators (colored dot + optional built-in label toggle)
UseMetricCardcomponent for numeric KPIs with sparkline, change badge, progress bar
UseSectionCardcomponent for titled card wrapper with icon and action slot
UseAgentCardcomponent for agent display: avatar, name, role, status, current task
UseAvatarcomponent for circular initials avatar with optionalborderColor?prop instead of rendering manually
UseToast/ToastContainercomponents for success/error/warning/info notifications with auto-dismiss queue (mountToastContaineronce in AppLayout)
UseSkeleton/SkeletonCard/SkeletonMetric/SkeletonTable/SkeletonTextcomponents for loading placeholders matching component shapes with shimmer animation
UseEmptyStatecomponent for no-data / no-results placeholder with icon, title, description, optional action button
UseErrorBoundarycomponent withlevelprop (page/section/component) for error handling with retry capability
UseConfirmDialogcomponent for confirmation modals withdefault/destructivevariants andloadingstate
UseInlineEditcomponent for click-to-edit text with Enter/Escape, validation, optimistic save with rollback
UseStaggerGroup/StaggerItemcomponents for card entrance stagger container with configurable delay
UseDrawercomponent withsideprop (left or right, default right), spring animation, focus trap, Escape-to-close, optional header (title),ariaLabelfor accessible name (one oftitleorariaLabelrequired), andcontentClassNameoverride
UseInputFieldcomponent for labeled text input with error/hint display, optional multiline textarea mode
UseSelectFieldcomponent for labeled select dropdown with error/hint and placeholder support
UseSliderFieldcomponent for labeled range slider with custom value formatter and aria-live display
UseToggleFieldcomp...
Files:
web/src/pages/providers/ProviderFilters.tsxweb/src/pages/providers/ProviderCard.tsx
web/src/{components,pages,layout}/**/*.tsx
📄 CodeRabbit inference engine (web/CLAUDE.md)
web/src/{components,pages,layout}/**/*.tsx: MountCommandPalettecomponent once in AppLayout; register commands viauseCommandPalettehook for global Cmd+K search
UseMobileUnsupportedOverlaycomponent for full-screen overlay at<768pxviewports directing users to desktop or CLI; component self-manages visibility viauseBreakpoint
Files:
web/src/pages/providers/ProviderFilters.tsxweb/src/pages/providers/ProviderCard.tsx
web/src/{pages,router}/**/*.tsx
📄 CodeRabbit inference engine (web/CLAUDE.md)
Use
AnimatedPresencecomponent as page transition wrapper with Framer Motion AnimatePresence keyed by route
Files:
web/src/pages/providers/ProviderFilters.tsxweb/src/pages/providers/ProviderCard.tsx
web/{tsconfig.json,src/**/*.{ts,tsx}}
📄 CodeRabbit inference engine (web/CLAUDE.md)
In TypeScript 6.0,
noUncheckedSideEffectImportsdefaults totrue; CSS side-effect imports need type declarations (Vite's/// <reference types="vite/client" />covers this)
Files:
web/src/pages/providers/ProviderFilters.tsxweb/src/pages/providers/ProviderCard.tsxweb/src/__tests__/pages/ProviderDetailPage.test.tsx
web/src/pages/**/*.tsx
📄 CodeRabbit inference engine (web/CLAUDE.md)
Place page-scoped sub-components in
pages/<page-name>/subdirs (e.g., tasks/, org-edit/, settings/) alongside the main page component
Files:
web/src/pages/providers/ProviderFilters.tsxweb/src/pages/providers/ProviderCard.tsx
web/src/__tests__/**/*.{test,spec}.{ts,tsx}
📄 CodeRabbit inference engine (web/CLAUDE.md)
Mirror test file structure with
web/src/__tests__/directory that matchesweb/src/structure for Vitest unit + property tests
Files:
web/src/__tests__/pages/ProviderDetailPage.test.tsx
🧠 Learnings (10)
📓 Common learnings
Learnt from: CR
Repo: Aureliolo/synthorg PR: 0
File: web/CLAUDE.md:0-0
Timestamp: 2026-03-30T16:37:21.464Z
Learning: Applies to web/src/{components,pages}/**/*.tsx : Use `ProviderHealthBadge` component for provider health status indicator (up/degraded/down colored dot + optional label)
Learnt from: CR
Repo: Aureliolo/synthorg PR: 0
File: web/CLAUDE.md:0-0
Timestamp: 2026-03-30T16:37:21.464Z
Learning: Applies to web/src/{components,pages}/**/*.tsx : Use `TokenUsageBar` component for segmented horizontal meter bar for token usage (multi-segment with auto-colors, `role="meter"`, animated transitions)
📚 Learning: 2026-03-30T16:37:21.464Z
Learnt from: CR
Repo: Aureliolo/synthorg PR: 0
File: web/CLAUDE.md:0-0
Timestamp: 2026-03-30T16:37:21.464Z
Learning: Applies to web/src/{components,pages}/**/*.tsx : Use `ProviderHealthBadge` component for provider health status indicator (up/degraded/down colored dot + optional label)
Applied to files:
web/src/pages/providers/ProviderFilters.tsxweb/src/pages/providers/ProviderCard.tsxweb/src/__tests__/pages/ProviderDetailPage.test.tsx
📚 Learning: 2026-03-30T16:37:21.464Z
Learnt from: CR
Repo: Aureliolo/synthorg PR: 0
File: web/CLAUDE.md:0-0
Timestamp: 2026-03-30T16:37:21.464Z
Learning: Applies to web/src/{components,pages}/**/*.tsx : Use `MetricCard` component for numeric KPIs with sparkline, change badge, progress bar
Applied to files:
web/src/pages/providers/ProviderCard.tsx
📚 Learning: 2026-03-30T16:37:21.464Z
Learnt from: CR
Repo: Aureliolo/synthorg PR: 0
File: web/CLAUDE.md:0-0
Timestamp: 2026-03-30T16:37:21.464Z
Learning: Applies to web/src/{components,pages}/**/*.tsx : Use `TokenUsageBar` component for segmented horizontal meter bar for token usage (multi-segment with auto-colors, `role="meter"`, animated transitions)
Applied to files:
web/src/pages/providers/ProviderCard.tsx
📚 Learning: 2026-03-30T16:37:21.464Z
Learnt from: CR
Repo: Aureliolo/synthorg PR: 0
File: web/CLAUDE.md:0-0
Timestamp: 2026-03-30T16:37:21.464Z
Learning: Applies to web/src/{components,pages}/**/*.tsx : Do NOT create metric displays with `text-metric font-bold` -- use `<MetricCard>`
Applied to files:
web/src/pages/providers/ProviderCard.tsx
📚 Learning: 2026-03-30T16:37:21.464Z
Learnt from: CR
Repo: Aureliolo/synthorg PR: 0
File: web/CLAUDE.md:0-0
Timestamp: 2026-03-30T16:37:21.464Z
Learning: Applies to web/src/{components,pages}/**/*.tsx : Use `AgentCard` component for agent display: avatar, name, role, status, current task
Applied to files:
web/src/pages/providers/ProviderCard.tsx
📚 Learning: 2026-03-30T16:37:21.464Z
Learnt from: CR
Repo: Aureliolo/synthorg PR: 0
File: web/CLAUDE.md:0-0
Timestamp: 2026-03-30T16:37:21.464Z
Learning: Applies to web/src/{components,pages}/**/*.tsx : Use `SectionCard` component for titled card wrapper with icon and action slot
Applied to files:
web/src/pages/providers/ProviderCard.tsx
📚 Learning: 2026-03-30T16:37:21.464Z
Learnt from: CR
Repo: Aureliolo/synthorg PR: 0
File: web/CLAUDE.md:0-0
Timestamp: 2026-03-30T16:37:21.464Z
Learning: Applies to web/src/**/*.{ts,tsx} : Use token variables for shadows and borders (`var(--so-shadow-card-hover)`, `border-border`, `border-bright`). NEVER hardcode shadow or border styles
Applied to files:
web/src/pages/providers/ProviderCard.tsx
📚 Learning: 2026-03-30T16:37:21.464Z
Learnt from: CR
Repo: Aureliolo/synthorg PR: 0
File: web/CLAUDE.md:0-0
Timestamp: 2026-03-30T16:37:21.464Z
Learning: Applies to web/src/**/*.{ts,tsx} : Use density-aware spacing tokens (`p-card`, `gap-section-gap`, `gap-grid-gap`) or standard Tailwind spacing. NEVER hardcode pixel values for layout spacing
Applied to files:
web/src/pages/providers/ProviderCard.tsx
📚 Learning: 2026-03-15T18:28:13.207Z
Learnt from: CR
Repo: Aureliolo/synthorg PR: 0
File: CLAUDE.md:0-0
Timestamp: 2026-03-15T18:28:13.207Z
Learning: Applies to tests/**/*.py : Tests must use test-provider, test-small-001, etc. for vendor-agnostic test data.
Applied to files:
web/src/__tests__/pages/ProviderDetailPage.test.tsx
🔇 Additional comments (3)
web/src/pages/providers/ProviderFilters.tsx (1)
12-12: Unknown health filter is wired correctly.
'unknown'is consistently added to options and validation, and the handler now preserves it as a realProviderHealthStatusinstead of collapsing tonull.Also applies to: 21-47
web/src/pages/providers/ProviderCard.tsx (1)
6-6: Health usage metrics rendering is now consistent.Good change: calls, tokens, and cost are rendered whenever
healthexists, so zero values are no longer hidden and formatting is centralized.Also applies to: 56-70
web/src/__tests__/pages/ProviderDetailPage.test.tsx (1)
22-22: Test updates align well with the new provider model/usage surface.Nice coverage expansion: vendor-agnostic fixtures, typed
ProviderModelResponsetest data, capability assertions, and token/cost assertions are all in place.Also applies to: 40-42, 54-54, 83-104, 118-126
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
🤖 I have created a release *beep* *boop* --- ## [0.5.2](v0.5.1...v0.5.2) (2026-03-31) ### Features * harden activity feed API ([#838](#838), [#839](#839), [#840](#840)) ([#937](#937)) ([c0234ad](c0234ad)) * provider usage metrics, model capabilities, and active health probing ([#935](#935)) ([1434c9c](1434c9c)) * runtime sink configuration via SettingsService ([#934](#934)) ([16c3f23](16c3f23)) * Settings page comprehensive redesign ([#936](#936)) ([#939](#939)) ([6d9ac8b](6d9ac8b)) ### Maintenance * bump astro from 6.1.1 to 6.1.2 in /site in the all group ([#940](#940)) ([ffa24f0](ffa24f0)) * bump pygments from 2.19.2 to 2.20.0 ([#931](#931)) ([9993088](9993088)) * bump the all group with 2 updates ([#942](#942)) ([aea37f8](aea37f8)) * bump typescript-eslint from 8.57.2 to 8.58.0 in /web in the all group ([#941](#941)) ([24f024c](24f024c)) * split CLAUDE.md into subdirectory files for cli/ and web/ ([#932](#932)) ([f5cfe07](f5cfe07)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please). --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Summary
ProviderHealthSummarywith data fromCostTrackerProviderModelResponseDTO that queries the driver layer'sModelCapabilitiesProviderHealthProberbackground service that pings local providers every 30 minutes with lightweight HTTP requests (no model loading) to detect reachabilityUNKNOWNhealth status for providers with zero calls -- fixes providers like Ollama always showing "up" even when not runningTest plan
uv run python -m pytest tests/ -m unit -n auto -k provider(943+ tests pass)npm --prefix web run test(2216 tests pass)uv run mypy src/ tests/andnpm --prefix web run type-checkboth cleanuv run ruff check src/ tests/andnpm --prefix web run lintboth cleannpm --prefix web run storybookReview coverage
Pre-reviewed by 8 agents (code-reviewer, frontend-reviewer, api-contract-drift, silent-failure-hunter, async-concurrency-reviewer, conventions-enforcer, issue-resolution-verifier, docs-consistency). 12 findings addressed.
Closes #894
🤖 Generated with Claude Code