Skip to content

KiwiSDR: assigning any public receiver (full, free, or audio-failing) triggers escalating main-thread stalls that freeze the entire UI/desktop and repeatedly drop the Flex (v26.6.4) #3816

Description

@skerker

Summary

Assigning a public KiwiSDR as a slice's RX antenna can send the main/UI thread into a runaway: MainThreadWatchdog logs event-loop stalls that grow geometrically (≈0.6 s → 1.4 s → 3.9 s → 11 s → 31 s, then on auto-reconnect again to 41 s) until the whole desktop freezes. Each freeze starves the Flex TCP link until the radio drops (RadioConnection: socket error: "The remote host closed the connection"RadioModel: disconnectedRadioDiscovery: radio disconnected). The app auto-reconnects to the Flex and then freezes again — a destructive loop that only ends when the process is killed.

Key finding: the freeze is intrinsic to the KiwiSDR RX path — it does not depend on the receiver being full or on audio failing. It was reproduced from three independent triggers, all funneling into the same escalating main-thread runaway:

  1. Capacity rejection — a receiver at 4/4 users returns a clean "at capacity" rejection (no audio attempted) → freeze.
  2. Free receiver — a receiver with 1/8 users (plenty free) passes policy, opens the SND WebSocket, then begins the same escalation during connection setup (audio never arrives because the main thread is already stalling).
  3. No SND audio frames — the audio-handshake failure seen across multiple receivers yesterday.

That the free-receiver case freezes during setup (before any audio) shows this is not an error-path edge case — the Kiwi connect/preflight/WebSocket-setup work on the main thread is itself the problem.

Environment

  • AetherSDR v26.6.4 built from source (-DENABLE_BNR=ON -DAETHER_GPU_SPECTRUM=ON), Qt 6.8.3
  • Ubuntu 24.04, Linux 6.17, X11 (display froze entirely during the stall)
  • FlexRadio FLEX-8400, direct LAN (10.0.0.94:4992), 32 GB RAM (not resource-starved)
  • RTX 5060 Laptop; GPU waterfall on the iGPU

Steps to reproduce (today's minimal repro)

  1. Connect the Flex; have a slice on an HF freq.
  2. On the slice's RX-antenna control, assign a public KiwiSDR (here: Point Reyes KPH, kphsdr.com:8073).
  3. The receiver returns "at capacity (4/4 users)" — a normal rejection.
  4. Observe: the UI begins stalling and escalates to a full freeze; the Flex is dropped; on auto-reconnect it freezes again.

Reproduced immediately on the first attempt. (Yesterday the same runaway was reached via the "no SND audio frames" audio-handshake failure across 3 receivers, 6/7 attempts — same end state.)

Evidence (log aethersdr-20260625-112815.log)

Trigger — a clean capacity rejection (no audio attempted):

11:29:22 kiwisdr: Connecting Point Reyes KPH -> kphsdr.com:8073
11:29:22 kiwisdr: State -> 1 (Checking KiwiSDR access policy ...)
11:29:22 kiwisdr: status preflight http://kphsdr.com:8073/status
11:29:22 MainThreadWatchdog: event loop stalled ~608 ms        <- stall begins during preflight
11:29:23 kiwisdr: users=4 users_max=4 preempt=0
11:29:23 kiwisdr: State -> 3 (This KiwiSDR endpoint is at capacity (4/4 users)...)

Cycle 1 — escalation → Flex dropped:

11:29:24 stalled ~1357 ms
11:29:27 stalled ~3859 ms
11:29:38 stalled ~10869 ms
11:29:53 RadioConnection: socket error: "The remote host closed the connection"
11:30:08 AudioEngine: no audio data received for 15001 ms, restarting RX (#1411)
11:30:09 stalled ~30720 ms                                     <- ~31 s; UI frozen
11:30:15 RadioModel: disconnected ; RadioDiscovery: radio disconnected

Cycle 2 — auto-reconnect, then freezes again → Flex dropped again:

11:30:20 RadioModel: auto-reconnecting to ...94 ; RadioModel: connected
11:30:22 stalled ~708 ms
11:30:23 stalled ~1710 ms
11:30:29 stalled ~5178 ms
11:30:43 stalled ~14935 ms
11:30:59 RadioConnection: socket error: "The remote host closed the connection"
11:31:13 AudioEngine: no audio data received for 15010 ms, restarting RX (#1411)
         stalled ~40918 ms                                     <- ~41 s; whole desktop frozen
11:31:24 RadioModel: disconnected (loop continues until process killed)

Free receiver — same freeze during connection setup (log aethersdr-20260625-113546.log):

11:37:19 kiwisdr: Connecting San Jose (CA) -> sybil.yak.net:8073
11:37:19 kiwisdr: status preflight http://sybil.yak.net:8073/status
11:37:19 stalled ~664 ms
11:37:20 kiwisdr: users=1 users_max=8 preempt=1            <- FREE (7 of 8 slots open)
11:37:20 kiwisdr: State -> 1 (Connecting ...)
11:37:20 kiwisdr: SND URL ws://sybil.yak.net:8073/ws/kiwi/.../SND   <- audio WebSocket opened
11:37:21 stalled ~1449 ms
11:37:25 stalled ~4161 ms                                 <- escalation underway; killed here to spare the desktop

A receiver with 7 free slots still enters the runaway during setup, before audio could arrive — so availability is not the factor.

Intermittent success (~1 in 7) — the tell

The feature does occasionally work end-to-end. Yesterday audio arrived once in seven attempts (Point Reyes KPH, 18:03 — "Connected, audio arrived"); the operator reports ~3 lifetime successes total. This intermittency is the signature of a race: when SND frames happen to arrive quickly (fast/lightly-loaded receiver, good network moment) before the main-thread stall cascade dominates, the connection stabilizes and works; otherwise the runaway wins. That it ever succeeds shows the protocol/audio path is fundamentally sound — the defect is the main-thread work buildup, not the Kiwi handshake itself.

Why this looks main-thread-bound (not a network timeout)

  • KiwiSdrClient uses async I/O (QWebSocket signals, QNetworkAccessManager::get() for /status) — no waitForConnected/QEventLoop. So the stall is main-thread work buildup, not a blocking socket.
  • The escalation is geometric and survives the original trigger (it keeps growing after the capacity rejection / after the Flex drop), which points to an unbounded queue / retry storm / repaint cascade being serviced on the UI thread, feeding itself.
  • Waterfall frames kept flowing while this happened, so the IQ/waterfall path is fine — the problem is in the Kiwi connect/preflight/error + audio-restart handling on the main thread.

Suggested investigation

  • Move all Kiwi connection/preflight/error handling and stream decode off the main thread, and bound any backlog/retry queue.
  • The /status preflight + JSON parse and the State→Error transition should never block the UI; a rejection (capacity, policy, no-audio) must emit one clean error and tear down, never escalate.
  • The AudioEngine ... restarting RX (#1411) 15 s retry appears to interact with the freeze — confirm the restart isn't being re-queued on the main thread during a stall.

Related

  • Some Kiwis Fail to Connect #3790 (Some Kiwis fail to connect) — different: that's a connect/TLS failure on the new proxy pool. Here KPH connected fine to preflight and the freeze followed a normal capacity rejection.
  • [sanitizer] TSan weekly run failure #3462 ([sanitizer] TSan audio) — possibly related threading issue in the audio path.
  • ROADMAP "KiwiSDR follow-ups." Full logs available on request.

Metadata

Metadata

Assignees

Labels

GUIUser interfaceaudioAudio engine and streamingbugSomething isn't workingexternal devicesFlexControl, MIDI, serial port, USB peripheralsmaintainer-reviewRequires maintainer review before any action is takenpriority: highHigh priority

Type

No type

Fields

No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions