Skip to content

Investigate band/status storms, logging overhead, and pan/waterfall UX stutter #2471

Description

@jensenpat

Summary

While investigating panadapter/waterfall FPS behavior, we found a separate performance risk around band/status storms and logging overhead. The pan/waterfall FPS reconciliation bugs are being handled separately; this issue tracks the remaining question: when radio status bursts occur around startup, band changes, and tuning/scrolling across boundaries, how much of the visible UX stutter is caused by status volume, synchronous logging, log mutex contention, or file I/O?

The most striking observation so far: after turning off every debug category, the client became very fast and fluid. Pan stayed locked near 29 FPS, waterfall stayed near 25 FPS, and App CPU was about 5%. That suggests debug/status logging pressure may be a major amplifier during otherwise normal radio state storms.

What We Learned

  • Band/status storms appear most likely during startup and band changes, and may also happen during tuning/scrolling/jump operations that cross band or out-of-band boundaries.
  • These periods line up with places where the UI can feel loaded: pan stutter, waterfall FPS drops, sluggish drag response, and general main-thread pressure.
  • The new perf summaries are useful for correlating uiLagMaxMs, pan/waterfall age, update/render duration, UDP health, and S-history cost, but they are currently gated behind Performance logging.
  • The highest-value problem-report data is exactly the data users are least likely to have enabled ahead of time.
  • Making raw debug logs default-on is risky because of privacy/PII concerns and because the logging itself may be part of the performance problem.
  • Encrypting/redacting all raw logs may be a larger project. A better first step may be sanitized summary counters plus a low-overhead buffered logging path.

Working Hypothesis

Radio status storms generate many log lines and status updates in a short window. If debug categories are enabled, synchronous logging, shared log mutexes, formatting, regex/redaction, or disk writes can turn those bursts into visible UX stalls. This may be especially noticeable during pan drag or band-boundary movement because the UI is already doing render/update work.

Areas To Instrument

  • Radio status storm counters: status messages/sec, status bytes/sec, max burst size, and burst duration.
  • LogManager pressure: lines/sec, bytes/sec, per-category counts, max/p95 write duration, mutex wait time, dropped/coalesced line count if buffering is introduced.
  • Event-loop health: heartbeat lateness/max stall during storm windows.
  • Pan/waterfall correlation: FPS, frame age p95/max, update duration p95/max, render duration p95/max.
  • Network correlation: UDP batches, datagrams/sec, drain max, packet loss, incomplete frame resets.
  • Interaction correlation: drag-active state, mouse move gap max, wheel/drag update duration.

Proposed Next Steps

  1. Quantify the current cost: compare a normal build with debug categories enabled vs all debug categories disabled during the same startup, band-change, and pan-drag scenarios.
  2. Prototype buffered/asynchronous logging: move log file writes off hot paths, preserve ordering, and keep warnings/errors reliable.
  3. Add summary-only default telemetry for storms: sanitized one-line counters suitable for support bundles without requiring raw command/status logs.
  4. Measure LogManager contention explicitly: include queue depth, p95/max enqueue time, p95/max writer flush time, and dropped/coalesced debug lines.
  5. Decide privacy policy for support bundles: prefer PII-safe summaries by default, with opt-in raw debug logs when needed.
  6. Re-run perf scenarios after buffering: idle receive, startup, band change, pan drag, zoom drag, waterfall history scroll, Network Diagnostics open, spots/memories/jump/go-to workflows.

Acceptance Criteria

  • We can identify whether a bad UX window correlates with radio status storms, logging pressure, network delivery, UI backlog, rendering, or S-history work.
  • Support bundles include useful startup/band-boundary performance summaries by default without exposing raw sensitive radio/user data.
  • Debug logging no longer causes obvious pan/waterfall stutter during status-heavy windows.
  • Pan/waterfall visible behavior and radio command ordering remain unchanged.

Notes

This is separate from the pan FPS startup reconciliation and waterfall line-duration reconciliation work. Those fixes may remove FPS target drift, but this issue is about the broader UX impact of status storms and logging/log I/O pressure.

Metadata

Metadata

Assignees

No one assigned

    Labels

    GUIUser interfaceenhancementImprovement to existing featuremaintainer-reviewRequires maintainer review before any action is takenpriority: mediumMedium priorityprotocolSmartSDR protocol

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions