Skip to content

Add collector performance diagnostics to Lite UI #40

@erikdarlingdata

Description

@erikdarlingdata

Problem

Every collector in Lite tracks split timing via Stopwatch — SQL Server query time (sql_duration_ms) and DuckDB insert time (duckdb_duration_ms) — and logs both to the collection_log table and to disk (logs/lite_YYYYMMDD.log). But none of this timing data is surfaced in the UI.

When a collector starts timing out (like the XE ring buffer issue in #37), the only way to diagnose it today is to:

  • Read the log file manually from logs/ on disk
  • Query the collection_log DuckDB table directly
  • Use the MCP get_collection_health tool

There's no way for a user to see that a collector is getting slower over time, or whether the bottleneck is SQL Server or DuckDB.

Proposed Solution

Add a diagnostics view (tab, panel, or window) that shows:

  1. Collector timing grid — recent collection_log entries showing collector name, duration, SQL vs DuckDB split, rows collected, status
  2. Timing trend chart — per-collector duration over time to spot degradation
  3. Log viewer — display recent entries from the daily log file, or at minimum an "Open Log File" button

The collection_log table already has all the data needed:

  • collector_name, collection_time, duration_ms, sql_duration_ms, duckdb_duration_ms
  • rows_collected, status, error_message

Related Files

  • Lite/Database/Schema.cscollection_log table schema (line 37), split timing columns added in v4 migration
  • Lite/Services/RemoteCollectorService.cs_lastSqlMs / _lastDuckDbMs fields (line 89), LogCollectionAsync() (line 372)
  • Lite/Services/LocalDataService.CollectionHealth.csGetCollectionHealthAsync() query (line 21)
  • Lite/Services/AppLogger.cs — file-based logging implementation

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions