Summary

Implement a database-backed settings persistence layer that replaces direct YAML config reads. YAML files provide defaults only — all user customizations are stored in the database and take precedence. Every config value in the system flows through this layer.

Motivation

Currently all configuration is loaded from YAML at startup and is static for the lifetime of the process. Users must edit YAML files and restart to change anything. This is unacceptable for a production system — settings must be editable at runtime through the UI and API.

Architecture

Config Resolution Order (highest priority wins)

1. Database overrides — user-set values via API/UI
2. Environment variables — for deployment-specific secrets
3. YAML defaults — shipped defaults and template starting points
4. Code defaults — Pydantic model defaults

Core Components

SettingsRepository protocol:

- get(namespace: str, key: str) -> str | None
- get_all(namespace: str) -> dict[str, str]
- set(namespace: str, key: str, value: str) -> None
- delete(namespace: str, key: str) -> None
- list_namespaces() -> list[str]

SQLite implementation:

• Table: settings(namespace TEXT, key TEXT, value TEXT, updated_at TEXT, PRIMARY KEY(namespace, key))
• Namespaces: company, providers, memory, budget, security, coordination, observability, backup

SettingsService:

• Merges YAML defaults + env vars + DB overrides
• Exposes typed getters that return resolved Pydantic config models
• Cache layer with invalidation on write
• Change notification (publish to message bus so running components can hot-reload)

Settings Metadata Registry

Every setting must be registered with metadata:

@dataclass(frozen=True)
class SettingDefinition:
    namespace: str           # e.g. "providers", "budget"
    key: str                 # e.g. "max_daily_usd"
    type: SettingType        # str, int, float, bool, enum, json
    default: Any             # default value
    description: str         # human-readable description
    group: str               # UI grouping (e.g. "Budget Limits", "Security")
    level: SettingLevel      # BASIC or ADVANCED
    sensitive: bool          # if True, encrypt at rest, mask in UI
    restart_required: bool   # if True, change takes effect after restart
    validator: Callable | None  # optional custom validation

This registry is the single source of truth for what settings exist, their types, defaults, and UI presentation. The web dashboard reads this registry to dynamically render settings forms (see #NNN — dynamic settings UI).

API Endpoints

Encryption for Sensitive Settings

Settings marked sensitive: true (API keys, secrets) are encrypted at rest using Fernet symmetric encryption. The master key comes from an environment variable (SYNTHORG_SETTINGS_KEY), NOT stored in DB.

Affected Modules

• New: src/synthorg/settings/ — repository, service, registry, schema
• Modify: src/synthorg/config/ — loader becomes "defaults provider", not the sole config source
• Modify: src/synthorg/api/controllers/ — add settings endpoints
• Modify: src/synthorg/api/state.py — wire settings service into app state

Acceptance Criteria

• SettingsRepository protocol + SQLite implementation
• SettingsService merges YAML defaults + env vars + DB overrides
• Settings metadata registry with type, description, group, level, sensitive flag
• REST API for CRUD on settings
• GET /api/v1/settings/_schema returns full registry for dynamic UI
• Sensitive settings encrypted at rest
• Change notifications via message bus
• All existing config paths migrated to read through SettingsService
• Tests: unit + integration for resolution order, encryption, cache invalidation