Skip to content

Multi-thread architecture: AudioEngine, PanadapterStream, QRhi GPU rendering #502

Description

@ten9876

Problem

The main thread consumes ~70-78% of one CPU core, running audio processing, VITA-49 packet parsing, and QPainter rendering in a single event loop. On lower-end machines, paintEvent starves the audio callback, causing RX/TX audio artifacts reported by multiple users across Windows, macOS, and Linux.

Solution: Three-Phase Thread Migration

Migrate from single-threaded to a three-thread architecture:

```
Current: Main Thread (78% CPU) = UDP parsing + audio + DSP + rendering + GUI

Target: Main Thread (~5-10% CPU) = QRhi GPU rendering + GUI events
Audio Thread (~5% CPU) = AudioEngine + NR2/RN2/BNR DSP
Network Thread (~3% CPU) = PanadapterStream + VITA-49 parsing
```


Phase 1: AudioEngine Worker Thread

Goal: Decouple audio processing from GUI rendering. Fixes audio artifacts.

Thread Safety Changes

Member Current Change
`m_muted`, `m_transmitting`, `m_daxTxMode` plain `bool` `std::atomic`
`m_rxVolume`, `m_pcMicGain` plain `float`/`int` `std::atomic`/`std::atomic`
`m_nr2Enabled`, `m_rn2Enabled`, `m_bnrEnabled` plain `bool` `std::atomic` + lifecycle mutex
`m_nr2`, `m_rn2`, `m_bnrFilter` `unique_ptr` Protected by `m_dspMutex` (create/destroy/process)
`m_radeMode` already `std::atomic` No change

Signal/Slot Rewiring

Signal From → To Current After
`PanadapterStream::audioDataReady` Main → Audio Direct Queued (cross-thread)
`AudioEngine::txPacketReady` Audio → Main Direct Queued
`AudioEngine::levelChanged` Audio → Main Direct Queued (VU meters)
`AudioEngine::pcMicLevelChanged` Audio → Main Direct Queued
UI → `setNr2Enabled()` Main → Audio Direct call `QMetaObject::invokeMethod`
UI → `setRn2Enabled()` Main → Audio Direct call `QMetaObject::invokeMethod`
UI → `setBnrEnabled()` Main → Audio Direct call `QMetaObject::invokeMethod`
UI → `setMuted()`, `setVolume()` Main → Audio Direct call Atomic write (no invoke needed)
`feedDaxTxAudio()` Main → Audio Direct call `QMetaObject::invokeMethod`
`feedDecodedSpeech()` RADE → Audio Already queued No change

Lifecycle

  • Create `QThread` in MainWindow, move AudioEngine to it before `startRxStream()`
  • `QAudioSink`/`QAudioSource` created via `QMetaObject::invokeMethod` on the worker thread
  • `CwDecoder` stays on main thread (feeds GUI text overlay directly)
  • On shutdown: stop audio thread before destroying AudioEngine members

Files Modified

File Changes
`AudioEngine.h` Add `m_dspMutex`, make flags atomic, add `Q_INVOKABLE` to DSP setters
`AudioEngine.cpp` Lock `m_dspMutex` in `feedAudioData()` DSP section and enable/disable methods
`MainWindow.h` Add `QThread* m_audioThread`
`MainWindow.cpp` Create thread, `moveToThread()`, rewire signals with `Qt::QueuedConnection`, invoke setters via `QMetaObject`

Testing

  • Verify audio plays without artifacts while resizing window (forces heavy paintEvent)
  • Verify NR2/RN2/BNR enable/disable doesn't crash (DSP mutex)
  • Verify TX audio timing (queued txPacketReady adds ~1ms latency)
  • Verify VU meters still update (queued levelChanged)
  • Run with TSAN (thread sanitizer) to catch data races

Estimated Effort: ~400 lines changed


Phase 2: PanadapterStream Worker Thread

Goal: Move UDP parsing and VITA-49 demux off the main thread.

Architecture

```
Network Thread (PanadapterStream):
QUdpSocket::readyRead
→ processDatagram() — parse headers, assemble FFT frames, decode waterfall tiles
→ emit spectrumReady() ──queued──→ Main Thread (SpectrumWidget)
→ emit waterfallRowReady() ──queued──→ Main Thread (SpectrumWidget)
→ emit meterDataReady() ──queued──→ Main Thread (MeterModel)
→ emit audioDataReady() ──queued──→ Audio Thread (AudioEngine)
→ emit daxAudioReady() ──queued──→ Main Thread (DaxBridge)
```

Key Considerations

  • `QUdpSocket` must be created on the network thread (owns the socket)
  • `sendToRadio()` called from AudioEngine (TX packets) — must be invoked on the network thread since it uses the socket
  • FFT frame assembly (`FrameAssembler`) and waterfall tile assembly are CPU work that moves off main thread
  • Multi-Flex stream filtering (`m_knownPanStreams`, `m_knownWfStreams`) written from main thread (RadioModel) — needs `QMutex` or atomic set swap
  • `setOwnedStreamIds()`, `setYPixels()`, `setDbmRange()` called from main thread — need thread-safe setters

Signal/Slot Rewiring

All PanadapterStream → MainWindow/RadioModel connections become automatically queued when PanadapterStream lives on a different thread. No explicit `Qt::QueuedConnection` needed if using standard `connect()`.

`txPacketReady` from AudioEngine → PanadapterStream becomes Audio Thread → Network Thread (queued).

Files Modified

File Changes
`PanadapterStream.h` Add mutex for stream ID sets, make setters thread-safe
`PanadapterStream.cpp` Move socket creation to `start()` on worker thread
`RadioModel.h` Add `QThread* m_networkThread`
`RadioModel.cpp` Create thread, move PanadapterStream, rewire
`MainWindow.cpp` Update wiring (signals auto-queue cross-thread)

Testing

  • Verify FFT/waterfall still render correctly (no dropped frames from queue)
  • Verify meter updates aren't delayed (queued signal latency)
  • Verify Multi-Flex filtering still works (thread-safe stream ID sets)
  • Verify TX audio packets still reach the radio (cross-thread sendToRadio)
  • Profile: main thread CPU should drop significantly (no more packet parsing)

Estimated Effort: ~300 lines changed


Phase 3: QRhi GPU-Accelerated Rendering

Goal: Offload spectrum and waterfall rendering from CPU to GPU.

Architecture

Replace `SpectrumWidget`'s `QPainter`-based `paintEvent()` with a `QRhiWidget` subclass that renders via GPU shaders.

Platform QRhi Backend Notes
Linux x64 OpenGL Mesa drivers
Linux ARM (Pi) OpenGL ES Pi 4/5 V3D
macOS Metal Apple deprecated OpenGL
Windows D3D11 Works in VMs via WARP

Waterfall (biggest CPU win)

  • Allocate GPU texture: width × history_rows (e.g., 1024 × 800)
  • New row: upload single texture row (~4KB) at ring buffer offset
  • Render as textured quad with UV offset for scrolling — zero CPU copy
  • Color mapping: fragment shader with intensity → color LUT texture
  • Current CPU cost: ~40% of paintEvent. GPU cost: ~0.1%

FFT Spectrum

  • Upload bin values as vertex buffer (~2000 floats per frame)
  • Draw as `GL_LINE_STRIP` with optional smoothing
  • Fill below the line: triangle strip to bottom of area
  • Grid lines: separate line batch (static, rarely changes)
  • Current CPU cost: ~15% of paintEvent

Overlays

  • Filter passband: colored quad with alpha blend
  • Slice markers: vertical line + triangle
  • TNF markers: semi-transparent rectangles
  • Spot labels: hybrid — QPainter text overlay on top of QRhi surface
  • Band plan: colored quads (static, regenerate on plan change)

Hybrid Rendering

Qt supports QPainter on top of QRhiWidget for text and complex UI elements. Strategy:

  1. GPU renders: waterfall texture, FFT polyline, grid, filter passband, colored overlays
  2. CPU renders (QPainter overlay): text labels (callsigns, frequencies, dBm values), VFO widgets (remain as child QWidgets)

This avoids implementing a full GPU text rendering pipeline.

Files Modified

File Changes
`SpectrumWidget.h/.cpp` Major rewrite: QWidget → QRhiWidget subclass, shader-based rendering
`resources/shaders/` NEW — GLSL vertex/fragment shaders for waterfall + FFT
`CMakeLists.txt` Add shader compilation, QRhi dependency
`MainWindow.cpp` Minor — SpectrumWidget API stays the same

Fallback

Keep QPainter rendering as a compile-time option (`-DUSE_QRHI=OFF`) for:

  • CI builds (headless, no GPU)
  • Raspberry Pi (may prefer software rendering)
  • Debugging

Requirements

  • Qt 6.6+ for stable QRhiWidget API
  • GPU with OpenGL 3.3+ / Metal / D3D11 (covers virtually all hardware from 2010+)
  • No external dependencies — QRhi is bundled with Qt6

Testing

  • Visual comparison: GPU vs CPU rendering should be pixel-identical for waterfall colors
  • Performance: measure main thread CPU before/after (target: 78% → <10%)
  • Platform testing: Linux (OpenGL), macOS (Metal), Windows (D3D11)
  • Edge cases: window resize, multi-pan layout changes, minimal mode toggle
  • Fallback: verify QPainter path still compiles and works

Estimated Effort: ~800-1000 lines (largest phase)


Risk Assessment

Phase Risk Mitigation
1 DSP use-after-free on enable/disable Mutex around DSP lifecycle + process
1 QAudioSink on wrong thread Create via invokeMethod on worker
1 TX audio latency from queued signal Measure — expect <1ms, acceptable
2 Dropped FFT frames from queue backpressure Qt queued signals are unbounded; monitor
2 Stream ID race (main writes, network reads) Mutex or atomic swap
3 Platform-specific GPU bugs QPainter fallback always available
3 QRhiWidget API instability Requires Qt 6.6+, well-tested by now

Implementation Order

Phase 1 → Phase 2 → Phase 3. Each phase is independently shippable and testable. Phase 1 alone fixes the reported audio artifacts. Phase 2 gives additional headroom. Phase 3 is the long-term solution.

Total estimated effort: ~1500-1700 lines across all three phases.

Metadata

Metadata

Assignees

No one assigned

    Labels

    GUIUser interfaceVITA-49VITA-49 UDP streaming: FFT, waterfall, audio, metersaudioAudio engine and streamingpriority: highHigh priority

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions