Summary

Add retrieval quality observability following the existing observer pattern. A new RetrievalObserver tracks query metrics (result counts, scores, latency, rerank usage) and reports health via the observer API.

Problem Statement

The observer infrastructure covers queue, VikingDB, VLM, and transaction subsystems, but the retrieval pipeline has no observability beyond debug logs. When retrieval quality degrades, users have no metrics to diagnose the problem. The HierarchicalRetriever logs individual queries at DEBUG level, but there are no aggregate stats for hit rates, score distributions, or latency.

Evidence

The observer pattern is well-established in the codebase. This PR follows it exactly.

Changes

New files:

• openviking/retrieve/retrieval_stats.py (165 lines) - Thread-safe RetrievalStatsCollector singleton that accumulates per-query metrics
• openviking/storage/observers/retrieval_observer.py (99 lines) - RetrievalObserver implementing BaseObserver
• tests/unit/retrieve/test_retrieval_stats.py (175 lines) - 17 unit tests

Modified files:

• openviking/retrieve/hierarchical_retriever.py - Record stats after each retrieve() call (3 lines added)
• openviking/storage/observers/__init__.py - Export RetrievalObserver
• openviking/service/debug_service.py - Register retrieval in observer service + system health
• openviking/server/routers/observer.py - Add GET /api/v1/observer/retrieval endpoint

Metrics tracked

Health logic

• Healthy: zero-result rate < 50%
• Unhealthy: zero-result rate >= 50% (after at least 5 queries)
• No errors flagged with fewer than 5 queries

Testing

All 17 unit tests pass:

======================== 17 passed, 1 warning in 0.03s =========================

The RetrievalObserver uses a lazy import of get_stats_collector() to avoid a circular dependency with the storage module. This follows the same pattern as tabulate imports in other observers.

This contribution was developed with AI assistance (Claude Code).

Relates to #578