Summary

R&D proposal to give AetherSDR a first-class, agent-drivable test surface so we can do end-to-end UX testing of the panadapter/waterfall and applet controls across macOS/Windows/Linux. Today we ship features without complete E2E UX coverage across OSes; this issue captures the findings from a read-only investigation and proposes a phased path.

The core finding

AetherSDR is a Qt 6 Widgets native desktop app (no QML, no web layer). Our UX splits into two fundamentally different testing problems that need different tools:

There is no single right tool. Controls want structured introspection; the spectral display wants deterministic pixels.

Evaluation of the obvious options

• OS accessibility APIs (AT-SPI / UIA / NSAccessibility): right instinct for controls, wrong delivery — driving three different OS APIs cross-platform is flaky/slow, and still blind to the panadapter. The underlying QAccessible tree is gold; expose it in-process instead.
• Claude computer-use / VNC: works on the real app on any OS and can see the waterfall, but slow, non-deterministic, DPI/coordinate-fragile, expensive. Good for exploratory smoke + genuinely-visual checks; poor as a regression backbone.
• Browser tooling: N/A — nothing renders in a browser.

Proposal: an in-app, agent-first automation bridge

Add an in-process automation + introspection channel, gated behind a flag (e.g. --automation / AETHER_AUTOMATION=1, off in production). We already have every building block: QTcpServer/QWebSocket servers, Q_PROPERTY models, and a rich QLoggingCategory taxonomy. An agent then drives this socket — fast, deterministic, identical on all three OSes, CI-friendly — mirroring the snapshot→act→assert loop we already use for web work.

Four verbs:

1. dumpTree() → JSON — ARIA-style semantic snapshot of the widget hierarchy (objectName, accessibleName, role, value, enabled, geometry). Our "DOM snapshot" for controls. Finishing the a11y annotations the linter already flags makes this near-free.
2. invoke(objectName, action, value) — click a button / set a slider / pick a combo entry by name. Deterministic, no pixel hunting.
3. get(model, property) — read live truth from PanadapterModel (center MHz, zoom, min/max dBm), SliceModel (freq, mode, filter, NB/NR), RadioModel. Assert on state without screenshots.
4. grab(widget) → PNG — on-demand capture of any widget, including a GPU framebuffer readback of the panadapter. Pixels enter only when an assertion is inherently visual, and deterministically.

Recommendation: build a dedicated automation channel rather than overloading TCI. TCI has external protocol-compat constraints (eesdr-tci aborts on unknown commands), so test verbs must not leak into a radio-control protocol.

The panadapter problem specifically

A screenshot of a live radio spectrum is non-deterministic noise — un-golden-able. The unlock is deterministic data injection: a fixture/replay mode that feeds a recorded or synthetic VITA-49 FFT + meter stream so the waterfall shows a known signal every run. Then:

• Assert structurally via get() (e.g. after scroll-zoom: PanadapterModel.bandwidth == 48kHz, center unchanged) — fast, exact, no pixels.
• Assert visually via grab() + perceptual diff against per-OS golden images (QRhi backends/GPUs render legitimately differently, so exact-match won't work).

We appear to have no radio simulator in the tree today. Our lcConnection raw-command logging is already a recording substrate, so replay mode is the foundational prerequisite that makes all E2E deterministic without hardware.

Closing the cross-OS E2E gap

CI already builds on macOS/Windows/Linux but never runs the GUI. Add a per-OS job that:

1. Launches with QT_QPA_PLATFORM=offscreen + --automation + a recorded fixture (no hardware, no display).
2. Runs an agent-authored scenario over the bridge (snapshot → act → assert).
3. Captures semantic snapshots + grab() PNGs as artifacts; perceptual-diffs the panadapter against per-OS goldens; fails on control-state regressions.

Phased plan

• Phase 0 — Prove it (~1 day): dumpTree() + grab() on a QLocalServer behind AETHER_AUTOMATION. Drive live; produce one applet snapshot + a panadapter PNG.
• Phase 1 — Drive + assert: add invoke() + get(); clear the a11y-linter backlog (improves real accessibility and snapshot quality).
• Phase 2 — Determinism: replay/fixture mode so the panadapter is reproducible without a radio.
• Phase 3 — CI E2E matrix: offscreen + agent scenarios + perceptual golden diffs across all three OSes.
• Phase 4 — Computer-use as the exploratory tier: keep VNC/computer-use for occasional real-GPU/real-WM smoke + visual spot-checks, not the regression backbone.

Why this fits AetherSDR

We extend patterns already in the tree rather than bolting on a foreign framework: TciServer is already a WS command/event bus, Q_PROPERTY models are already introspectable, logging is already tailable, and the a11y linter already maps the annotation backlog.

Reference (read-only investigation)

• Rendering: src/gui/SpectrumWidget.h/.cpp (QRhi GPU path #ifdef AETHER_GPU_SPECTRUM, else paintEvent raster), src/gui/PanadapterApplet.*
• Applets/controls: src/gui/MeterApplet.*, RxApplet.*, PhoneApplet.* (QWidget subclasses); tools/check_a11y.py, .github/workflows/a11y-check.yml
• Models: src/models/{RadioModel,SliceModel,PanadapterModel}.* (Q_PROPERTY)
• Existing control surfaces to reuse patterns from: src/core/{TciServer,RigctlProtocol,SmartCatSession,CatPort}.*, src/core/tnc/KissTncServer.*
• Logging: src/core/LogManager.* (QLoggingCategory, tailable log)
• Entry/headless: src/main.cpp (no --test/headless flags today; AETHER_NO_GPU env exists; QT_QPA_PLATFORM=offscreen works)

💻 Generated with Claude Code (Opus 4.8) with architecture by @jensenpat