feat: add RFC 9457 structured error responses (Phase 1)#457
Conversation
Add machine-readable error metadata to all API error responses: - ErrorCategory (8-value StrEnum) and ErrorCode (15-value IntEnum) with 4-digit category-grouped codes - ErrorDetail frozen Pydantic model (message, error_code, error_category, retryable, retry_after, instance) - error_detail field on ApiResponse and PaginatedResponse envelopes - Class-level error_category/error_code/retryable on ApiError hierarchy - Request correlation ID binding in RequestLoggingMiddleware - All 10 exception handlers populate error_detail - Fix flaky rate limiter pause tests with mocked time Closes #419
- Add ClassVar annotations + __init_subclass__ validation on ApiError - Remove redundant retryable=False on non-retryable subclasses - Use INTERNAL_ERROR (8000) instead of PERSISTENCE_ERROR for client - Make _get_instance_id defensive (try/except for exception handlers) - Use clear_correlation_ids() instead of unbind_correlation_id - Extract _log_request_completion to reduce __call__ below 50 lines - Add Field(ge=0) on ErrorDetail.retry_after - Change ErrorDetail.instance to NotBlankStr - Remove unnecessary int() cast on IntEnum error_code - Fix bare pytest.raises(Exception) to use ValidationError - Add direct unit tests for _get_instance_id and _category_for_status - Add 5xx scrubbing test with custom ServiceUnavailableError message - Add ApiError instantiation tests (default/custom message, status) - Add __init_subclass__ mismatch rejection test - Update CLAUDE.md package structure and logging event constants - Add RFC 9457 error format docs to operations.md and architecture
Dependency Review✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.Scanned FilesNone |
|
Caution Review failedThe pull request is closed. ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: ASSERTIVE Plan: Pro Run ID: 📒 Files selected for processing (1)
📝 WalkthroughSummary by CodeRabbit
WalkthroughAdds RFC 9457 Phase 1 structured error support across API, including ErrorCategory/ErrorCode enums, ErrorDetail DTO, ApiResponse/ PaginatedResponse error_detail fields, exception handler/middleware changes to emit structured errors and correlation IDs, frontend type/client updates, docs, and comprehensive tests. Changes
Sequence DiagramsequenceDiagram
autonumber
actor Client
participant Middleware as Middleware (Correlation)
participant Handler as AppHandler
participant Exception as ExceptionHandler
participant DTO as ErrorDetailModel
participant Response as ApiResponse
Client->>Middleware: send HTTP request
Middleware->>Middleware: generate_correlation_id()
Middleware->>Middleware: bind_correlation_id(id)
Middleware->>Handler: forward request
alt handler succeeds
Handler->>Response: return ApiResponse(data, error=null, error_detail=null)
else handler raises
Handler->>Exception: raise ApiError / HTTP exception
Exception->>Exception: _category_for_status(status)
Exception->>Middleware: _get_instance_id() (correlation id)
Exception->>DTO: construct ErrorDetail(message, error_code, error_category, retryable, retry_after, instance)
DTO->>Exception: ErrorDetail
Exception->>Response: _build_error_response(with error_detail)
end
Response->>Middleware: send response
Middleware->>Middleware: _log_request_completion(status)
Middleware->>Middleware: clear_correlation_ids()
Response->>Client: ApiResponse (+ error_detail when error)
Estimated code review effort🎯 4 (Complex) | ⏱️ ~45 minutes Possibly related PRs
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches
🧪 Generate unit tests (beta)
✨ Simplify code
📝 Coding Plan
Comment |
Summary of ChangesHello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request significantly upgrades the API's error reporting capabilities by introducing structured, machine-readable error responses based on RFC 9457. This change provides clients with detailed error context, including standardized codes, categories, and retry information, which is crucial for building more resilient and intelligent client applications. Additionally, it improves the overall stability of the system by addressing test flakiness and enhancing logging with robust request correlation IDs. Highlights
Changelog
Activity
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
There was a problem hiding this comment.
Code Review
This pull request introduces a robust and well-designed structured error reporting mechanism following RFC 9457. The changes are comprehensive and of high quality, including a new error taxonomy, updated DTOs and exception handlers, and improved middleware for correlation IDs. The use of __init_subclass__ for validation and the addition of thorough tests are particularly noteworthy. My only suggestion is to consider expanding the ApiError hierarchy to map more domain-specific exceptions to this new structured format, which would further enhance the utility of this excellent feature.
| } | ||
|
|
||
|
|
||
| class ApiError(Exception): |
There was a problem hiding this comment.
This new error taxonomy is a great foundation. To fully leverage it, consider introducing more specific ApiError subclasses for other key domain errors. For example, domain exceptions like BudgetExhaustedError or provider-related errors could be caught at the API boundary and re-raised as new ApiBudgetExhaustedError(ApiError) or ApiProviderError(ApiError) subclasses.
This would allow you to return more specific error codes (e.g., BUDGET_EXHAUSTED, PROVIDER_ERROR) and categories to the client, rather than having them fall through to the generic INTERNAL_ERROR. This would make the API more informative for consumers.
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #457 +/- ##
==========================================
+ Coverage 93.67% 93.69% +0.01%
==========================================
Files 469 469
Lines 22219 22331 +112
Branches 2143 2152 +9
==========================================
+ Hits 20814 20922 +108
- Misses 1095 1098 +3
- Partials 310 311 +1 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@CLAUDE.md`:
- Line 190: The long inline list of event constants (e.g., PROVIDER_CALL_START,
BUDGET_RECORD_ADDED, CFO_ANOMALY_DETECTED, MEETING_STARTED,
CLASSIFICATION_START, MEMORY_RETRIEVAL_START, TASK_CREATED,
APPROVAL_GATE_ESCALATION_DETECTED, etc.) hurts readability; condense this by
replacing the exhaustive list with a short explanatory sentence pointing to
synthorg.observability.events and an example import pattern (e.g., "from
synthorg.observability.events.<domain> import EVENT_CONSTANT"), or move the full
list into a separate reference document and link to it from the guideline;
update the CLAUDE.md section that currently enumerates the constants so it
contains only the module reference and an example, and add the full enumeration
to a referenced doc for maintainability.
In `@tests/unit/providers/resilience/test_rate_limiter.py`:
- Line 100: Move the repeated local import "from unittest import mock" to the
module-level imports and delete the in-function imports found inside the test
methods (the occurrences currently inside the test functions at the five
locations where "from unittest import mock" is repeated); update the top of the
test module to include "from unittest import mock" and remove the redundant
lines from each test so all tests use the single module-level mock import.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro
Run ID: 74366d95-bf34-47d6-a6bb-11bada2bf271
📒 Files selected for processing (12)
CLAUDE.mddocs/architecture/index.mddocs/design/operations.mdsrc/synthorg/api/dto.pysrc/synthorg/api/errors.pysrc/synthorg/api/exception_handlers.pysrc/synthorg/api/middleware.pytests/unit/api/test_dto.pytests/unit/api/test_dto_properties.pytests/unit/api/test_errors.pytests/unit/api/test_exception_handlers.pytests/unit/providers/resilience/test_rate_limiter.py
📜 Review details
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (4)
- GitHub Check: Test (Python 3.14)
- GitHub Check: Build Backend
- GitHub Check: Build Web
- GitHub Check: Analyze (python)
🧰 Additional context used
📓 Path-based instructions (5)
docs/**/*.md
📄 CodeRabbit inference engine (CLAUDE.md)
All docstrings in public APIs must be documented in Google style and reflected in
docs/api/auto-generated library reference (via mkdocstrings + Griffe AST-based parsing)
Files:
docs/architecture/index.mddocs/design/operations.md
docs/design/**/*.md
📄 CodeRabbit inference engine (CLAUDE.md)
ALWAYS read the relevant design page before implementing any feature or planning any issue. If implementation deviates from the spec, alert the user and explain why — user decides whether to proceed or update the spec
Files:
docs/design/operations.md
**/*.py
📄 CodeRabbit inference engine (CLAUDE.md)
**/*.py: Nofrom __future__ import annotations— Python 3.14 has PEP 649 native lazy annotations
Useexcept A, B:syntax (no parentheses) instead ofexcept (A, B):— PEP 758 syntax enforced by ruff on Python 3.14
All public functions must have type hints; mypy strict mode is enforced
Use Google-style docstrings on all public classes and functions (enforced by ruff D rules)
UseNotBlankStrfromcore.typesfor all identifier/name fields (including optional and tuple variants) instead of manual whitespace validators
Use@computed_fieldin Pydantic models for derived values instead of storing and validating redundant fields
Preferasyncio.TaskGroupfor fan-out/fan-in parallel operations (e.g., multiple tool invocations, parallel agent calls) instead of barecreate_task
Line length: 88 characters (enforced by ruff)
Functions must be < 50 lines; files must be < 800 lines
Handle errors explicitly; never silently swallow exceptions
Validate input at system boundaries (user input, external APIs, config files)
Create new objects instead of mutating existing ones (immutability principle)
Files:
tests/unit/api/test_errors.pytests/unit/providers/resilience/test_rate_limiter.pytests/unit/api/test_dto_properties.pytests/unit/api/test_exception_handlers.pysrc/synthorg/api/exception_handlers.pytests/unit/api/test_dto.pysrc/synthorg/api/errors.pysrc/synthorg/api/dto.pysrc/synthorg/api/middleware.py
tests/**/*.py
📄 CodeRabbit inference engine (CLAUDE.md)
tests/**/*.py: Use@pytest.mark.unit,@pytest.mark.integration,@pytest.mark.e2e,@pytest.mark.slowto categorize tests
Maintain 80% minimum code coverage (enforced in CI)
Each test must complete within 30 seconds (timeout enforcement)
Always include-n autowhen running pytest viauv run python -m pytest— never run tests sequentially (pytest-xdist parallelism)
Prefer@pytest.mark.parametrizefor testing similar cases
Never use real vendor names (Anthropic, OpenAI, Claude, GPT, etc.) in project-owned code, docstrings, comments, tests, or config examples — use generic names likeexample-provider,example-large-001,test-provider,test-small-001, or size aliases (large/medium/small)
Use Hypothesis for property-based testing with@given+@settingsdecorators; control profiles viaHYPOTHESIS_PROFILEenv var (cifor 200 examples,devfor 1000 examples)
Files:
tests/unit/api/test_errors.pytests/unit/providers/resilience/test_rate_limiter.pytests/unit/api/test_dto_properties.pytests/unit/api/test_exception_handlers.pytests/unit/api/test_dto.py
src/synthorg/**/*.py
📄 CodeRabbit inference engine (CLAUDE.md)
src/synthorg/**/*.py: Every module with business logic must importfrom synthorg.observability import get_loggerand definelogger = get_logger(__name__)
Never useimport logging,logging.getLogger(), orprint()in application code — use the synthorg logger instead
Always use event name constants from domain-specific modules undersynthorg.observability.events(e.g.,PROVIDER_CALL_STARTfromevents.provider); import directly:from synthorg.observability.events.<domain> import EVENT_CONSTANT
Always log with structured kwargs:logger.info(EVENT, key=value)— never use old-style formattinglogger.info("msg %s", val)
All error paths must log at WARNING or ERROR with context before raising
All state transitions must log at INFO level
Use DEBUG level for object creation, internal flow, and entry/exit of key functions
Pure data models, enums, and re-exports do NOT need logging
Never implement retry logic in driver subclasses or calling code — all provider calls go throughBaseCompletionProviderwhich applies retry + rate limiting automatically
SetRetryConfigandRateLimiterConfigper-provider inProviderConfig; retryable errors areRateLimitError,ProviderTimeoutError,ProviderConnectionError,ProviderInternalError
Use frozen Pydantic models for config/identity; separate mutable-via-copy models (usingmodel_copy(update=...)) for runtime state
For dict/list fields in frozen Pydantic models, usecopy.deepcopy()at system boundaries (tool execution, LLM provider serialization, inter-agent delegation, persistence serialization)
Use Pydantic v2 conventions:BaseModel,model_validator,computed_field,ConfigDict
Files:
src/synthorg/api/exception_handlers.pysrc/synthorg/api/errors.pysrc/synthorg/api/dto.pysrc/synthorg/api/middleware.py
🧠 Learnings (22)
📚 Learning: 2026-03-15T11:48:14.867Z
Learnt from: CR
Repo: Aureliolo/synthorg PR: 0
File: CLAUDE.md:0-0
Timestamp: 2026-03-15T11:48:14.867Z
Learning: Applies to src/synthorg/providers/**/*.py : Rate limiter respects RateLimitError.retry_after from providers — automatically pauses future requests.
Applied to files:
tests/unit/providers/resilience/test_rate_limiter.py
📚 Learning: 2026-03-15T11:48:14.867Z
Learnt from: CR
Repo: Aureliolo/synthorg PR: 0
File: CLAUDE.md:0-0
Timestamp: 2026-03-15T11:48:14.867Z
Learning: Applies to src/synthorg/**/*.py : Every module with business logic MUST have: `from synthorg.observability import get_logger` then `logger = get_logger(__name__)`. Never use `import logging` / `logging.getLogger()` / `print()` in application code. Variable name: always `logger` (not `_logger`, not `log`).
Applied to files:
CLAUDE.md
📚 Learning: 2026-03-15T11:48:14.867Z
Learnt from: CR
Repo: Aureliolo/synthorg PR: 0
File: CLAUDE.md:0-0
Timestamp: 2026-03-15T11:48:14.867Z
Learning: Applies to src/synthorg/**/*.py : Structured kwargs in logging: always `logger.info(EVENT, key=value)` — never `logger.info('msg %s', val)`.
Applied to files:
CLAUDE.md
📚 Learning: 2026-03-15T12:05:56.884Z
Learnt from: CR
Repo: Aureliolo/synthorg PR: 0
File: CLAUDE.md:0-0
Timestamp: 2026-03-15T12:05:56.884Z
Learning: Applies to src/synthorg/**/*.py : Always log with structured kwargs: `logger.info(EVENT, key=value)` — never use old-style formatting `logger.info("msg %s", val)`
Applied to files:
CLAUDE.md
📚 Learning: 2026-03-15T12:05:56.884Z
Learnt from: CR
Repo: Aureliolo/synthorg PR: 0
File: CLAUDE.md:0-0
Timestamp: 2026-03-15T12:05:56.884Z
Learning: Applies to src/synthorg/**/*.py : Every module with business logic must import `from synthorg.observability import get_logger` and define `logger = get_logger(__name__)`
Applied to files:
CLAUDE.mdsrc/synthorg/api/middleware.py
📚 Learning: 2026-03-15T11:48:14.867Z
Learnt from: CR
Repo: Aureliolo/synthorg PR: 0
File: CLAUDE.md:0-0
Timestamp: 2026-03-15T11:48:14.867Z
Learning: Applies to src/synthorg/**/*.py : Event names: always use constants from domain-specific modules under synthorg.observability.events (e.g., PROVIDER_CALL_START from events.provider, BUDGET_RECORD_ADDED from events.budget, etc.). Import directly: `from synthorg.observability.events.<domain> import EVENT_CONSTANT`.
Applied to files:
CLAUDE.mdsrc/synthorg/api/middleware.py
📚 Learning: 2026-03-15T11:48:14.867Z
Learnt from: CR
Repo: Aureliolo/synthorg PR: 0
File: CLAUDE.md:0-0
Timestamp: 2026-03-15T11:48:14.867Z
Learning: Applies to src/synthorg/**/*.py : All error paths must log at WARNING or ERROR with context before raising.
Applied to files:
CLAUDE.md
📚 Learning: 2026-03-15T12:05:56.884Z
Learnt from: CR
Repo: Aureliolo/synthorg PR: 0
File: CLAUDE.md:0-0
Timestamp: 2026-03-15T12:05:56.884Z
Learning: Applies to src/synthorg/**/*.py : Always use event name constants from domain-specific modules under `synthorg.observability.events` (e.g., `PROVIDER_CALL_START` from `events.provider`); import directly: `from synthorg.observability.events.<domain> import EVENT_CONSTANT`
Applied to files:
CLAUDE.md
📚 Learning: 2026-03-15T11:48:14.867Z
Learnt from: CR
Repo: Aureliolo/synthorg PR: 0
File: CLAUDE.md:0-0
Timestamp: 2026-03-15T11:48:14.867Z
Learning: Applies to src/synthorg/**/*.py : All state transitions must log at INFO.
Applied to files:
CLAUDE.md
📚 Learning: 2026-03-15T12:05:56.884Z
Learnt from: CR
Repo: Aureliolo/synthorg PR: 0
File: CLAUDE.md:0-0
Timestamp: 2026-03-15T12:05:56.884Z
Learning: Applies to src/synthorg/**/*.py : All error paths must log at WARNING or ERROR with context before raising
Applied to files:
CLAUDE.md
📚 Learning: 2026-03-15T12:05:56.884Z
Learnt from: CR
Repo: Aureliolo/synthorg PR: 0
File: CLAUDE.md:0-0
Timestamp: 2026-03-15T12:05:56.884Z
Learning: Applies to src/synthorg/**/*.py : Never use `import logging`, `logging.getLogger()`, or `print()` in application code — use the synthorg logger instead
Applied to files:
CLAUDE.md
📚 Learning: 2026-03-15T12:05:56.884Z
Learnt from: CR
Repo: Aureliolo/synthorg PR: 0
File: CLAUDE.md:0-0
Timestamp: 2026-03-15T12:05:56.884Z
Learning: Applies to tests/**/*.py : Never use real vendor names (Anthropic, OpenAI, Claude, GPT, etc.) in project-owned code, docstrings, comments, tests, or config examples — use generic names like `example-provider`, `example-large-001`, `test-provider`, `test-small-001`, or size aliases (`large`/`medium`/`small`)
Applied to files:
CLAUDE.md
📚 Learning: 2026-03-15T11:48:14.867Z
Learnt from: CR
Repo: Aureliolo/synthorg PR: 0
File: CLAUDE.md:0-0
Timestamp: 2026-03-15T11:48:14.867Z
Learning: Applies to {src,tests,web,cli,site}/**/*.{py,ts,tsx,go,astro} : Vendor-agnostic everywhere: NEVER use real vendor names (Anthropic, OpenAI, Claude, GPT, etc.) in project-owned code, docstrings, comments, tests, or config examples. Use generic names: example-provider, example-large-001, example-medium-001, example-small-001. Vendor names may only appear in: (1) Operations design page provider list (docs/design/operations.md), (2) .claude/ skill/agent files, (3) third-party import paths/module names.
Applied to files:
CLAUDE.md
📚 Learning: 2026-03-15T11:48:14.867Z
Learnt from: CR
Repo: Aureliolo/synthorg PR: 0
File: CLAUDE.md:0-0
Timestamp: 2026-03-15T11:48:14.867Z
Learning: Applies to tests/**/*.py : Test markers: pytest.mark.unit, pytest.mark.integration, pytest.mark.e2e, pytest.mark.slow. Coverage: 80% minimum (enforced in CI).
Applied to files:
CLAUDE.md
📚 Learning: 2026-03-15T11:48:14.867Z
Learnt from: CR
Repo: Aureliolo/synthorg PR: 0
File: CLAUDE.md:0-0
Timestamp: 2026-03-15T11:48:14.867Z
Learning: Applies to tests/**/*.py : Tests must use test-provider, test-small-001, etc. for vendor-agnostic test data.
Applied to files:
CLAUDE.md
📚 Learning: 2026-03-15T11:48:14.867Z
Learnt from: CR
Repo: Aureliolo/synthorg PR: 0
File: CLAUDE.md:0-0
Timestamp: 2026-03-15T11:48:14.867Z
Learning: Property-based testing: Python uses Hypothesis (given + settings). Hypothesis profiles: ci (200 examples, default) and dev (1000 examples), controlled via HYPOTHESIS_PROFILE env var. Run dev profile: HYPOTHESIS_PROFILE=dev uv run python -m pytest tests/ -m unit -n auto -k properties.
Applied to files:
CLAUDE.md
📚 Learning: 2026-03-15T11:48:14.867Z
Learnt from: CR
Repo: Aureliolo/synthorg PR: 0
File: CLAUDE.md:0-0
Timestamp: 2026-03-15T11:48:14.867Z
Learning: Parallelism: pytest-xdist via `-n auto` — ALWAYS include `-n auto` when running pytest, never run tests sequentially.
Applied to files:
CLAUDE.md
📚 Learning: 2026-03-15T12:05:56.884Z
Learnt from: CR
Repo: Aureliolo/synthorg PR: 0
File: CLAUDE.md:0-0
Timestamp: 2026-03-15T12:05:56.884Z
Learning: Applies to cli/**/*.go : Never use real vendor names (Anthropic, OpenAI, Claude, GPT, etc.) in project-owned code, docstrings, comments, or config examples — use generic names
Applied to files:
CLAUDE.md
📚 Learning: 2026-03-15T12:05:56.884Z
Learnt from: CR
Repo: Aureliolo/synthorg PR: 0
File: CLAUDE.md:0-0
Timestamp: 2026-03-15T12:05:56.884Z
Learning: Applies to web/**/*.{ts,tsx,js,jsx,vue} : Never use real vendor names (Anthropic, OpenAI, Claude, GPT, etc.) in project-owned code, docstrings, comments, or config examples — use generic names like `example-provider`, `example-large-001`
Applied to files:
CLAUDE.md
📚 Learning: 2026-03-15T11:48:14.867Z
Learnt from: CR
Repo: Aureliolo/synthorg PR: 0
File: CLAUDE.md:0-0
Timestamp: 2026-03-15T11:48:14.867Z
Learning: Applies to tests/**/*.py : Parametrize: Prefer pytest.mark.parametrize for testing similar cases.
Applied to files:
CLAUDE.md
📚 Learning: 2026-03-15T12:05:56.884Z
Learnt from: CR
Repo: Aureliolo/synthorg PR: 0
File: CLAUDE.md:0-0
Timestamp: 2026-03-15T12:05:56.884Z
Learning: Applies to tests/**/*.py : Use Hypothesis for property-based testing with `given` + `settings` decorators; control profiles via `HYPOTHESIS_PROFILE` env var (`ci` for 200 examples, `dev` for 1000 examples)
Applied to files:
CLAUDE.md
📚 Learning: 2026-03-15T12:05:56.884Z
Learnt from: CR
Repo: Aureliolo/synthorg PR: 0
File: CLAUDE.md:0-0
Timestamp: 2026-03-15T12:05:56.884Z
Learning: Applies to src/synthorg/**/*.py : Use Pydantic v2 conventions: `BaseModel`, `model_validator`, `computed_field`, `ConfigDict`
Applied to files:
src/synthorg/api/dto.py
🧬 Code graph analysis (6)
tests/unit/api/test_errors.py (1)
src/synthorg/api/errors.py (9)
ApiError(86-117)ApiValidationError(131-139)ConflictError(142-150)ErrorCategory(14-27)ErrorCode(30-68)ForbiddenError(153-161)NotFoundError(120-128)ServiceUnavailableError(175-184)UnauthorizedError(164-172)
tests/unit/api/test_dto_properties.py (2)
src/synthorg/api/dto.py (1)
ErrorDetail(40-63)src/synthorg/api/errors.py (1)
ErrorCategory(14-27)
tests/unit/api/test_exception_handlers.py (2)
src/synthorg/api/errors.py (8)
ApiError(86-117)ApiValidationError(131-139)ConflictError(142-150)ErrorCategory(14-27)ErrorCode(30-68)ForbiddenError(153-161)ServiceUnavailableError(175-184)UnauthorizedError(164-172)src/synthorg/api/exception_handlers.py (3)
handle_unexpected(212-225)_get_instance_id(45-59)_category_for_status(106-114)
src/synthorg/api/exception_handlers.py (2)
src/synthorg/api/dto.py (2)
ApiResponse(69-89)ErrorDetail(40-63)src/synthorg/api/errors.py (3)
ApiError(86-117)ErrorCategory(14-27)ErrorCode(30-68)
tests/unit/api/test_dto.py (2)
src/synthorg/api/dto.py (6)
ErrorDetail(40-63)PaginatedResponse(108-130)PaginationMeta(92-105)ApiResponse(69-89)success(87-89)success(128-130)src/synthorg/api/errors.py (1)
ErrorCategory(14-27)
src/synthorg/api/dto.py (1)
src/synthorg/api/errors.py (1)
ErrorCategory(14-27)
🔇 Additional comments (27)
src/synthorg/api/middleware.py (1)
25-29: LGTM! Clean correlation ID lifecycle management.The correlation ID is properly bound at request start and defensively cleared in a
finallyblock, ensuring cleanup even on exceptions. The_log_request_completionhelper appropriately uses WARNING level when status is unknown (indicating an incomplete response flow).Also applies to: 173-173, 193-194
src/synthorg/api/errors.py (2)
106-117: Excellent use of__init_subclass__for compile-time validation.This ensures error code/category consistency is validated at class definition time rather than runtime, catching misconfigurations early. The validation logic correctly extracts the category prefix and fails fast with a clear
TypeError.
14-83: LGTM! Well-structured error taxonomy.The error codes are logically grouped by category prefix (1xxx-8xxx), and the
_CODE_CATEGORY_PREFIXmapping enables runtime validation. The enum values align with the RFC 9457 Phase 1 requirements.src/synthorg/api/dto.py (2)
40-63: LGTM! Well-designedErrorDetailmodel.The model correctly uses
frozen=Truefor immutability,NotBlankStrfor the correlation ID instance field, andge=0constraint onretry_after. Usingintforerror_code(rather thanErrorCodeenum) provides flexibility for edge cases while keeping the model self-contained.
83-83: LGTM! Backward-compatible envelope changes.Adding
error_detailas an optional field withNonedefault ensures existing clients are unaffected. The field is present on bothApiResponseandPaginatedResponsefor consistency.Also applies to: 123-123
src/synthorg/api/exception_handlers.py (3)
45-59: Defensive error handling is appropriate here.The silent
exceptwith pass at lines 56-58 is intentionally documented ("Exception handlers must never crash — silent fallback is intentional"). This is the correct pattern for code running inside exception handlers, which are the last line of defense. The fallback togenerate_correlation_id()ensures a valid instance ID is always returned.
62-114: LGTM! Clean centralized error response construction.The
_build_error_responsehelper and status-to-metadata mapping provide a single source of truth for structured error responses. The retryable flags are correctly set (429, 503 →True), and the fallback logic appropriately defaults to validation for 4xx and internal for 5xx.
188-209: LGTM!ApiErrorhandler correctly extracts class-level metadata.The handler properly accesses the class-level
error_code,error_category, andretryableattributes from the exception type, preserving the 5xx message scrubbing pattern (line 198 usestype(exc).default_messagefor server errors).docs/architecture/index.md (1)
40-40: LGTM! Documentation accurately reflects the new feature.The addition of "RFC 9457 structured error responses" to the API module description is concise and accurate.
tests/unit/api/test_dto_properties.py (1)
223-263: LGTM! Solid property-based tests forErrorDetail.The strategy correctly generates valid instances with appropriate constraints (error codes 1000-8999, non-blank instance via filter). The roundtrip test validates serialization integrity, and the field preservation test confirms retryable/category are maintained.
docs/design/operations.md (1)
978-996: LGTM! Comprehensive documentation for the new error format.The section clearly documents all
error_detailfields, error code groupings, and provides actionable guidance for agent consumers on usingretryable/retry_afterfor autonomous retry logic. This aligns well with the RFC 9457 Phase 1 objectives.CLAUDE.md (2)
115-115: LGTM! Accurate package structure update.The API module description now correctly reflects the RFC 9457 structured error additions.
217-217: LGTM! Valuable guidance on flaky test remediation.The directive to fix flaky tests with deterministic mocks (rather than widening timing margins or skipping) is a best practice that improves test reliability.
tests/unit/providers/resilience/test_rate_limiter.py (2)
340-353: LGTM!The logging tests correctly verify pause and throttle event emission. The use of
structlog.testing.capture_logs()is appropriate for validating log output.
98-139: LGTM!The refactored pause tests use well-structured mocking to achieve determinism. The
time_fnclosure correctly simulates time progression through the pause window, andfake_sleepcaptures the sleep duration for assertions.tests/unit/api/test_dto.py (3)
45-64: LGTM!The tests correctly verify that
error_detailisNoneon success and properly populated on error. The assertions cover both object attributes and serialized output.
66-116: LGTM!Comprehensive coverage of
ErrorDetailconstruction, defaults, immutability, and serialization. The frozen model behavior is correctly validated withpytest.raises(ValidationError).
118-140: LGTM!Tests correctly verify
error_detaildefaults toNoneand is properly accessible when provided onPaginatedResponse.tests/unit/api/test_errors.py (3)
1-18: LGTM!Well-structured test module for the RFC 9457 error taxonomy. The imports cover all error classes and the
pytestmarkcorrectly applies@pytest.mark.unitto all tests.
39-86: LGTM!Excellent use of
@pytest.mark.parametrizefor testing enum members. The prefix consistency test (lines 79-85) ensures all error codes follow the category-grouped convention documented in the enum.
139-145: Verify that the test class definition is complete.The
_BadErrorclass definition appears to end at line 145 without apassstatement or method body. In Python, a class body requires at least one statement. However, since theerror_codeassignment is present, this should work correctly. The test relies on theTypeErrorbeing raised during class creation (in__init_subclass__), which happens before the class body would need additional statements.tests/unit/api/test_exception_handlers.py (6)
34-36: LGTM!The
_UUID_REpattern correctly validates standard UUID format, and_assert_error_detailis an excellent helper that reduces duplication across all exception handler tests while ensuring consistent validation of the RFC 9457 error structure.Also applies to: 46-64
67-455: LGTM!Comprehensive updates to all exception handler tests with proper
_assert_error_detailvalidation. The tests correctly verify:
- Error code and category mappings for each exception type
- Retryable flags (True for 429, False for most others)
- 5xx message scrubbing (internal details not exposed)
551-566: Good security test coverage.The
test_5xx_scrubs_custom_messagetest correctly verifies that internal details (like IP addresses) are not leaked in 5xx error responses. This is critical for preventing information disclosure.
568-615: LGTM!Thorough unit tests for
_get_instance_idcovering:
- Happy path (context has valid request_id)
- Fallback when no context
- Edge cases (non-string, empty string)
The
try/finallycleanup pattern correctly prevents test pollution.
618-691: LGTM!Excellent parametrized tests for
_category_for_statuscovering all mapped statuses and both unmapped server/client error fallbacks. TheTestApiErrorInstantiationclass correctly validates default messages, status codes, and custom message precedence.
1-3: LGTM!The updated docstring accurately reflects the test file's expanded scope to cover RFC 9457 structured error responses.
CLAUDE.md
Outdated
| - **Never** use `import logging` / `logging.getLogger()` / `print()` in application code | ||
| - **Variable name**: always `logger` (not `_logger`, not `log`) | ||
| - **Event names**: always use constants from the domain-specific module under `synthorg.observability.events` (e.g., `PROVIDER_CALL_START` from `events.provider`, `BUDGET_RECORD_ADDED` from `events.budget`, `CFO_ANOMALY_DETECTED` from `events.cfo`, `CONFLICT_DETECTED` from `events.conflict`, `MEETING_STARTED` from `events.meeting`, `MEETING_SCHEDULER_STARTED` from `events.meeting`, `MEETING_SCHEDULER_ERROR` from `events.meeting`, `MEETING_SCHEDULER_STOPPED` from `events.meeting`, `MEETING_PERIODIC_TRIGGERED` from `events.meeting`, `MEETING_EVENT_TRIGGERED` from `events.meeting`, `MEETING_PARTICIPANTS_RESOLVED` from `events.meeting`, `MEETING_NO_PARTICIPANTS` from `events.meeting`, `MEETING_NOT_FOUND` from `events.meeting`, `CLASSIFICATION_START` from `events.classification`, `CONSOLIDATION_START` from `events.consolidation`, `ORG_MEMORY_QUERY_START` from `events.org_memory`, `API_REQUEST_STARTED` from `events.api`, `API_ROUTE_NOT_FOUND` from `events.api`, `API_COORDINATION_STARTED` from `events.api`, `API_COORDINATION_COMPLETED` from `events.api`, `API_COORDINATION_FAILED` from `events.api`, `API_COORDINATION_AGENT_RESOLVE_FAILED` from `events.api`, `CODE_RUNNER_EXECUTE_START` from `events.code_runner`, `DOCKER_EXECUTE_START` from `events.docker`, `MCP_INVOKE_START` from `events.mcp`, `SECURITY_EVALUATE_START` from `events.security`, `HR_HIRING_REQUEST_CREATED` from `events.hr`, `PERF_METRIC_RECORDED` from `events.performance`, `TRUST_EVALUATE_START` from `events.trust`, `PROMOTION_EVALUATE_START` from `events.promotion`, `PROMPT_BUILD_START` from `events.prompt`, `MEMORY_RETRIEVAL_START` from `events.memory`, `MEMORY_BACKEND_CONNECTED` from `events.memory`, `MEMORY_ENTRY_STORED` from `events.memory`, `MEMORY_BACKEND_SYSTEM_ERROR` from `events.memory`, `AUTONOMY_ACTION_AUTO_APPROVED` from `events.autonomy`, `TIMEOUT_POLICY_EVALUATED` from `events.timeout`, `PERSISTENCE_AUDIT_ENTRY_SAVED` from `events.persistence`, `TASK_ENGINE_STARTED` from `events.task_engine`, `COORDINATION_STARTED` from `events.coordination`, `COORDINATION_FACTORY_BUILT` from `events.coordination`, `COMMUNICATION_DISPATCH_START` from `events.communication`, `COMPANY_STARTED` from `events.company`, `CONFIG_LOADED` from `events.config`, `CORRELATION_ID_CREATED` from `events.correlation`, `DECOMPOSITION_STARTED` from `events.decomposition`, `DELEGATION_STARTED` from `events.delegation`, `EXECUTION_LOOP_START` from `events.execution`, `CHECKPOINT_SAVED` from `events.checkpoint`, `PERSISTENCE_CHECKPOINT_SAVED` from `events.persistence`, `GIT_OPERATION_START` from `events.git`, `PARALLEL_GROUP_START` from `events.parallel`, `PERSONALITY_LOADED` from `events.personality`, `QUOTA_CHECKED` from `events.quota`, `ROLE_ASSIGNED` from `events.role`, `ROUTING_STARTED` from `events.routing`, `SANDBOX_EXECUTE_START` from `events.sandbox`, `TASK_CREATED` from `events.task`, `TASK_ASSIGNMENT_STARTED` from `events.task_assignment`, `TASK_ROUTING_STARTED` from `events.task_routing`, `TEMPLATE_LOADED` from `events.template`, `TOOL_INVOKE_START` from `events.tool`, `TOOL_OUTPUT_WITHHELD` from `events.tool`, `WORKSPACE_CREATED` from `events.workspace`, `APPROVAL_GATE_ESCALATION_DETECTED` from `events.approval_gate`, `APPROVAL_GATE_ESCALATION_FAILED` from `events.approval_gate`, `APPROVAL_GATE_INITIALIZED` from `events.approval_gate`, `APPROVAL_GATE_RISK_CLASSIFIED` from `events.approval_gate`, `APPROVAL_GATE_RISK_CLASSIFY_FAILED` from `events.approval_gate`, `APPROVAL_GATE_CONTEXT_PARKED` from `events.approval_gate`, `APPROVAL_GATE_CONTEXT_PARK_FAILED` from `events.approval_gate`, `APPROVAL_GATE_PARK_TASKLESS` from `events.approval_gate`, `APPROVAL_GATE_RESUME_STARTED` from `events.approval_gate`, `APPROVAL_GATE_CONTEXT_RESUMED` from `events.approval_gate`, `APPROVAL_GATE_RESUME_FAILED` from `events.approval_gate`, `APPROVAL_GATE_RESUME_DELETE_FAILED` from `events.approval_gate`, `APPROVAL_GATE_RESUME_TRIGGERED` from `events.approval_gate`, `APPROVAL_GATE_NO_PARKED_CONTEXT` from `events.approval_gate`, `APPROVAL_GATE_LOOP_WIRING_WARNING` from `events.approval_gate`). Import directly: `from synthorg.observability.events.<domain> import EVENT_CONSTANT` | ||
| - **Event names**: always use constants from the domain-specific module under `synthorg.observability.events` (e.g., `PROVIDER_CALL_START` from `events.provider`, `BUDGET_RECORD_ADDED` from `events.budget`, `CFO_ANOMALY_DETECTED` from `events.cfo`, `CONFLICT_DETECTED` from `events.conflict`, `MEETING_STARTED` from `events.meeting`, `MEETING_SCHEDULER_STARTED` from `events.meeting`, `MEETING_SCHEDULER_ERROR` from `events.meeting`, `MEETING_SCHEDULER_STOPPED` from `events.meeting`, `MEETING_PERIODIC_TRIGGERED` from `events.meeting`, `MEETING_EVENT_TRIGGERED` from `events.meeting`, `MEETING_PARTICIPANTS_RESOLVED` from `events.meeting`, `MEETING_NO_PARTICIPANTS` from `events.meeting`, `MEETING_NOT_FOUND` from `events.meeting`, `CLASSIFICATION_START` from `events.classification`, `CONSOLIDATION_START` from `events.consolidation`, `ORG_MEMORY_QUERY_START` from `events.org_memory`, `API_REQUEST_STARTED` from `events.api`, `API_REQUEST_ERROR` from `events.api`, `API_ROUTE_NOT_FOUND` from `events.api`, `API_COORDINATION_STARTED` from `events.api`, `API_COORDINATION_COMPLETED` from `events.api`, `API_COORDINATION_FAILED` from `events.api`, `API_COORDINATION_AGENT_RESOLVE_FAILED` from `events.api`, `CODE_RUNNER_EXECUTE_START` from `events.code_runner`, `DOCKER_EXECUTE_START` from `events.docker`, `MCP_INVOKE_START` from `events.mcp`, `SECURITY_EVALUATE_START` from `events.security`, `HR_HIRING_REQUEST_CREATED` from `events.hr`, `PERF_METRIC_RECORDED` from `events.performance`, `TRUST_EVALUATE_START` from `events.trust`, `PROMOTION_EVALUATE_START` from `events.promotion`, `PROMPT_BUILD_START` from `events.prompt`, `MEMORY_RETRIEVAL_START` from `events.memory`, `MEMORY_BACKEND_CONNECTED` from `events.memory`, `MEMORY_ENTRY_STORED` from `events.memory`, `MEMORY_BACKEND_SYSTEM_ERROR` from `events.memory`, `AUTONOMY_ACTION_AUTO_APPROVED` from `events.autonomy`, `TIMEOUT_POLICY_EVALUATED` from `events.timeout`, `PERSISTENCE_AUDIT_ENTRY_SAVED` from `events.persistence`, `TASK_ENGINE_STARTED` from `events.task_engine`, `COORDINATION_STARTED` from `events.coordination`, `COORDINATION_FACTORY_BUILT` from `events.coordination`, `COMMUNICATION_DISPATCH_START` from `events.communication`, `COMPANY_STARTED` from `events.company`, `CONFIG_LOADED` from `events.config`, `CORRELATION_ID_CREATED` from `events.correlation`, `DECOMPOSITION_STARTED` from `events.decomposition`, `DELEGATION_STARTED` from `events.delegation`, `EXECUTION_LOOP_START` from `events.execution`, `CHECKPOINT_SAVED` from `events.checkpoint`, `PERSISTENCE_CHECKPOINT_SAVED` from `events.persistence`, `GIT_OPERATION_START` from `events.git`, `PARALLEL_GROUP_START` from `events.parallel`, `PERSONALITY_LOADED` from `events.personality`, `QUOTA_CHECKED` from `events.quota`, `ROLE_ASSIGNED` from `events.role`, `ROUTING_STARTED` from `events.routing`, `SANDBOX_EXECUTE_START` from `events.sandbox`, `TASK_CREATED` from `events.task`, `TASK_ASSIGNMENT_STARTED` from `events.task_assignment`, `TASK_ROUTING_STARTED` from `events.task_routing`, `TEMPLATE_LOADED` from `events.template`, `TOOL_INVOKE_START` from `events.tool`, `TOOL_OUTPUT_WITHHELD` from `events.tool`, `WORKSPACE_CREATED` from `events.workspace`, `APPROVAL_GATE_ESCALATION_DETECTED` from `events.approval_gate`, `APPROVAL_GATE_ESCALATION_FAILED` from `events.approval_gate`, `APPROVAL_GATE_INITIALIZED` from `events.approval_gate`, `APPROVAL_GATE_RISK_CLASSIFIED` from `events.approval_gate`, `APPROVAL_GATE_RISK_CLASSIFY_FAILED` from `events.approval_gate`, `APPROVAL_GATE_CONTEXT_PARKED` from `events.approval_gate`, `APPROVAL_GATE_CONTEXT_PARK_FAILED` from `events.approval_gate`, `APPROVAL_GATE_PARK_TASKLESS` from `events.approval_gate`, `APPROVAL_GATE_RESUME_STARTED` from `events.approval_gate`, `APPROVAL_GATE_CONTEXT_RESUMED` from `events.approval_gate`, `APPROVAL_GATE_RESUME_FAILED` from `events.approval_gate`, `APPROVAL_GATE_RESUME_DELETE_FAILED` from `events.approval_gate`, `APPROVAL_GATE_RESUME_TRIGGERED` from `events.approval_gate`, `APPROVAL_GATE_NO_PARKED_CONTEXT` from `events.approval_gate`, `APPROVAL_GATE_LOOP_WIRING_WARNING` from `events.approval_gate`). Import directly: `from synthorg.observability.events.<domain> import EVENT_CONSTANT` |
There was a problem hiding this comment.
🧹 Nitpick | 🔵 Trivial
Consider condensing the event names list.
The exhaustive inline list of 60+ event constants is comprehensive but impacts readability. Consider either:
- Referencing the
synthorg.observability.eventsmodule structure directly - Moving the full list to a separate reference document
This is a minor readability concern; the current form is functionally correct.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@CLAUDE.md` at line 190, The long inline list of event constants (e.g.,
PROVIDER_CALL_START, BUDGET_RECORD_ADDED, CFO_ANOMALY_DETECTED, MEETING_STARTED,
CLASSIFICATION_START, MEMORY_RETRIEVAL_START, TASK_CREATED,
APPROVAL_GATE_ESCALATION_DETECTED, etc.) hurts readability; condense this by
replacing the exhaustive list with a short explanatory sentence pointing to
synthorg.observability.events and an example import pattern (e.g., "from
synthorg.observability.events.<domain> import EVENT_CONSTANT"), or move the full
list into a separate reference document and link to it from the guideline;
update the CLAUDE.md section that currently enumerates the constants so it
contains only the module reference and an example, and add the full enumeration
to a referenced doc for maintainability.
…, and Codecov Source fixes: - ErrorDetail: message uses NotBlankStr, retry_after/retryable cross-field validator, error_code docstring clarified as "by convention 4-digit" - ApiResponse/PaginatedResponse: model_validator rejects error_detail on success responses - ApiError docstring: distinguish ClassVar vs instance attributes - MappingProxyType wrapping on _CODE_CATEGORY_PREFIX and _STATUS_TO_ERROR_META - _get_instance_id: log at DEBUG on fallback instead of silent pass - _build_error_response: docstring documents instance auto-population - Dead code: remove unreachable `or type(exc).default_message` branch - Security: allow-list exc.headers (retry-after, www-authenticate, allow) - Security: 401/403 handlers use fixed strings instead of echoing exc.detail - Middleware: status_code=0 (int) instead of "unknown" (str) in structured logs; log WARNING on missing ASGI status key - _CODE_CATEGORY_PREFIX comment: remove test code reference Frontend type sync: - Add ErrorDetail, ErrorCategory, ErrorCode types to types.ts - Add error_detail field to ApiResponse/PaginatedResponse discriminated unions - Add ApiRequestError class to client.ts carrying ErrorDetail - Add getErrorDetail() helper to errors.ts Test improvements: - Module-level pytestmark in test_dto, test_exception_handlers, test_rate_limiter, test_middleware (consistency) - Hoist in-function imports to module level in test_exception_handlers and test_rate_limiter - New tests: retry_after negative rejection, retry_after/retryable consistency, error_detail on success rejected, _get_instance_id exception fallback, _build_error_response with retry_after, middleware correlation ID binding/clearing, _log_request_completion with status_code=None - Fix asyncio.sleep(0.05) in test_concurrent_limit to asyncio.sleep(0) - Fix auth handler test expectations for fixed strings Documentation: - CLAUDE.md: add API_REQUEST_COMPLETED to event constants list
Update client.property.test.ts to include error_detail field in mock ApiResponse/PaginatedResponse objects, matching the updated discriminated union types.
🤖 I have created a release *beep* *boop* --- ## [0.2.6](v0.2.5...v0.2.6) (2026-03-15) ### Features * add intra-loop stagnation detector ([#415](#415)) ([#458](#458)) ([8e9f34f](8e9f34f)) * add RFC 9457 structured error responses (Phase 1) ([#457](#457)) ([6612a99](6612a99)), closes [#419](#419) * implement AgentStateRepository for runtime state persistence ([#459](#459)) ([5009da7](5009da7)) * **site:** add SEO essentials, contact form, early-access banner ([#467](#467)) ([11b645e](11b645e)), closes [#466](#466) ### Bug Fixes * CLI improvements — config show, completion install, enhanced doctor, Sigstore verification ([#465](#465)) ([9e08cec](9e08cec)) * **site:** add reCAPTCHA v3, main landmark, and docs sitemap ([#469](#469)) ([fa6d35c](fa6d35c)) * use force-tag-creation instead of manual tag creation hack ([#462](#462)) ([2338004](2338004)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).
Summary
ErrorCategory(8-value StrEnum) andErrorCode(15-value IntEnum) with 4-digit category-grouped codesErrorDetailfrozen Pydantic model (message,error_code,error_category,retryable,retry_after,instance)error_detailfield onApiResponseandPaginatedResponseenvelopes (additive, backward-compatible)error_category/error_code/retryableonApiErrorhierarchy with__init_subclass__validationRequestLoggingMiddlewarevia structlog contextvarserror_detailvia centralized_build_error_response()time.monotonic()(deterministic)Review coverage
Pre-reviewed by 11 agents (code-reviewer, python-reviewer, test-analyzer, silent-failure-hunter, type-design-analyzer, logging-audit, resilience-audit, conventions-enforcer, security-reviewer, docs-consistency, issue-resolution-verifier). 20 findings addressed across source, tests, and documentation.
Key fixes from review:
PERSISTENCE_ERRORcode no longer exposed to clients (uses genericINTERNAL_ERROR)clear_correlation_ids()for defensive cleanup_get_instance_id()wrapped in try/except (exception handlers must never crash)ClassVarannotations +__init_subclass__enforces code/category consistencyNotBlankStroninstance,Field(ge=0)onretry_after, redundant overrides removed_get_instance_id,_category_for_status, 5xx scrubbing,ApiErrorinstantiationdocs/design/operations.md,docs/architecture/index.mdupdatedTest plan
uv run ruff check src/ tests/— lint passesuv run mypy src/ tests/— type-check passes (975 files)uv run python -m pytest tests/ -n auto --cov=synthorg --cov-fail-under=80— 8068 passed, 94.50% coverageuv run pre-commit run --all-files— all hooks passerror_detailappears in error responses via health check or manual testCloses #419