Skip to content

Add clean-room KiwiSDR receive integration#3668

Merged
ten9876 merged 3 commits into
aethersdr:mainfrom
rfoust:codex/kiwisdr-cleanroom-capture
Jun 20, 2026
Merged

Add clean-room KiwiSDR receive integration#3668
ten9876 merged 3 commits into
aethersdr:mainfrom
rfoust:codex/kiwisdr-cleanroom-capture

Conversation

@rfoust

@rfoust rfoust commented Jun 19, 2026

Copy link
Copy Markdown
Collaborator

Summary

KiwiSDR receiver waterfall/FFT preview

Adds receive-only KiwiSDR integration as client-owned virtual RX antennas. KiwiSDR profiles can be configured in Settings -> Antennas, auto-connected, selected from existing slice/panadapter antenna controls, and used as alternate receive sources without transmitting or sending Flex antenna mutations. When a slice uses a Kiwi profile, AetherSDR routes Kiwi audio through the RX DSP/speaker path, keeps Flex and Kiwi audio sources mixable, shows Kiwi waterfall/FFT data for that panadapter, and overlays connection errors when the selected Kiwi receiver is unavailable.

The implementation keeps KiwiSDR isolated behind new KiwiSdrClient, KiwiSdrManager, and KiwiSdrProtocol components, adds an informational KiwiSDR applet, maps existing waterfall controls to Kiwi W/F cell/floor/rate while a Kiwi antenna is selected, and documents the clean-room protocol/design boundary in docs/kiwisdr-cleanroom-design.md.

Constitution principle honored

Principle IV — this is a clean-room receive-only integration. The implementation was based on user requirements, user-provided protocol/spec text, clean black-box receive observations against user-approved KiwiSDR endpoints, and clean upstream AetherSDR interfaces.

Clean-room provenance

Allowed sources used:

  • User requirements and iterative behavior reports from this clean-room thread.
  • User-provided KiwiSDR protocol/spec corrections pasted into this thread.
  • Clean black-box receive-only observations against user-approved public KiwiSDR endpoints.
  • Clean upstream AetherSDR main.
  • AetherSDR project governance and accessibility docs.

Sources not used:

  • No KiwiSDR AGPL source, WebSDR source, served JavaScript, server code, decoder code, protocol handlers, copied snippets, blogs, forums, or implementation-derived assets.
  • No PR feat(websdr): add optional listen-only WebSDR receive module #3612, prior WebSDR worktrees/threads, contaminated temp files, rollout summaries, or previous WebSDR/RX-DSP implementation context.

What changed

  • Adds KiwiSDR virtual RX antenna profiles with custom names, endpoint normalization, auto-connect, reconnect handling, and persisted client-side settings.
  • Allows slice/panadapter antenna menus to select Kiwi profiles without sending those names to the Flex radio.
  • Routes Kiwi audio through AetherSDR RX DSP and speaker output while preserving independent Flex and per-Kiwi NR2/resampler/FIFO state.
  • Mixes Flex audio and multiple Kiwi audio streams post-DSP so other slices remain audible.
  • Adds Kiwi waterfall/FFT rendering from received W/F rows only, with independent retained history per Kiwi profile and per-panadapter source switching.
  • Maps waterfall rate, gain/cell, floor/black-level, and auto behavior to Kiwi controls when a Kiwi RX antenna is active, and restores normal Flex behavior when it is not.
  • Adds Kiwi S-meter support through the existing Flex-style meter surfaces and MeterSmoother, with unsupported layouts reported honestly instead of guessed.
  • Adds connection overlays for selected Kiwi receivers when sound or waterfall sockets close.
  • Adds protocol tests and NR2 regression coverage for the Kiwi receive path.

Skipped / intentionally unsupported

  • No TX/PTT/MOX/tune behavior.
  • No password-protected Kiwi receiver support beyond public receive auth.
  • No compressed/ADPCM Kiwi audio decoder.
  • No Kiwi IQ stream support.
  • No Kiwi directory discovery or extension launch/control surface.
  • No calibrated dBm claim for Kiwi waterfall bytes; they are treated as raw display/bin intensity.
  • No Kiwi-only controls that do not already map to an existing AetherSDR/Flex receive UI concept.

Test plan

  • Local build passes (cmake --build build --parallel)
  • Focused Kiwi/NR2 tests pass (ctest --test-dir build -R "kiwi|spectral_nr" --output-on-failure)
  • Whitespace check passes (git diff --check --cached)
  • Accessibility scanner run (python3 tools/check_a11y.py; warning-only existing findings remain)
  • Manual receive testing during development against public KiwiSDR endpoints and existing AetherSDR panadapter/audio workflows
  • Existing tests pass in CI

Checklist

  • Commits are signed (docs/COMMIT-SIGNING.md)
  • No new flat-key AppSettings calls — Kiwi RX antenna profiles are stored under one feature-owned JSON key
  • Code is clean-room — not decompiled, disassembled, or reverse-engineered from proprietary or AGPL implementation source
  • All meter UI uses MeterSmoother
  • Documentation updated for user-visible behavior and clean-room provenance
  • Security-sensitive changes reference a GHSA if applicable: N/A

@rfoust rfoust marked this pull request as ready for review June 19, 2026 14:31
@rfoust rfoust requested review from a team as code owners June 19, 2026 14:31
Copilot AI review requested due to automatic review settings June 19, 2026 14:31

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a clean-room, receive-only KiwiSDR integration to AetherSDR, exposing Kiwi endpoints as client-owned virtual RX antennas that can be selected from existing antenna controls, routed through the RX DSP/speaker path, and rendered via the existing spectrum/waterfall surfaces.

Changes:

  • Introduces KiwiSdrProtocol, KiwiSdrClient, and KiwiSdrManager to encapsulate KiwiSDR receive behavior, persistence, reconnection, and per-slice assignment.
  • Wires Kiwi virtual antennas into existing UI flows (VFO/RX applet/pan overlay), adds a Kiwi status applet, and adds waterfall/meter handling when Kiwi is active.
  • Fixes an NR2 (SpectralNR) block-size regression (packet-burst vs hop cadence) and adds protocol + NR2 regression tests.

Reviewed changes

Copilot reviewed 37 out of 37 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
tests/spectral_nr_test.cpp Adds a regression test ensuring NR2 output is invariant to burst/block size (Kiwi packet cadence).
tests/kiwi_sdr_protocol_test.cpp New test coverage for Kiwi protocol parsing, waterfall mapping helpers, and meter extraction/fallback logic.
src/models/SliceModel.h Extends slice audio getters to support an external receive-audio replacement mode (Kiwi) while preserving Flex values.
src/models/SliceModel.cpp Implements external-audio replacement handling for gain/mute and status application behavior.
src/models/RadioModel.cpp Minor safety tweak: null-check slice pointers during lookup.
src/gui/VfoWidget.h Adds Kiwi manager hookup and support for displaying Kiwi-derived meter readings.
src/gui/VfoWidget.cpp Integrates Kiwi virtual antennas into RX antenna menus and updates meter display/accessible events for Kiwi readings.
src/gui/SpectrumWidget.h Adds Kiwi waterfall/overlay APIs and state needed to render Kiwi W/F rows and manage per-profile history.
src/gui/SpectrumOverlayMenu.h Adds Kiwi manager hookup and Kiwi-mode waterfall control signals (cell/floor/rate).
src/gui/SpectrumOverlayMenu.cpp Implements Kiwi-mode behavior for existing waterfall controls (labels/ranges/emits) and virtual antenna selection.
src/gui/SMeterWidget.h Adds an API for Kiwi meter readings and “unavailable” meter behavior.
src/gui/SMeterWidget.cpp Implements Kiwi meter display rules (only calibrated meters move the needle) and accessibility updates.
src/gui/RxApplet.h Adds Kiwi manager hookup and virtual antenna selection signals.
src/gui/RxApplet.cpp Integrates Kiwi virtual antennas into RX applet antenna selection UI.
src/gui/RadioSetupDialog.h Adds KiwiSdrManager* plumbing into the settings dialog.
src/gui/RadioSetupDialog.cpp Adds a KiwiSDR RX Antennas section to configure profiles (name/endpoint/autoconnect/connect/remove).
src/gui/MainWindow.h Adds Kiwi wiring/refresh slots and state for Kiwi routing/tracking/mute ownership.
src/gui/MainWindow.cpp Instantiates Kiwi manager, wires Kiwi integration, and ensures orderly teardown in the destructor.
src/gui/MainWindow_Wiring.cpp Wires Kiwi selection and pan/zoom events into Kiwi waterfall view updates; integrates Kiwi into meter and setup dialog wiring.
src/gui/MainWindow_Menus.cpp Passes KiwiSdrManager into RadioSetupDialog when opened from menus.
src/gui/MainWindow_KiwiSdr.cpp New: centralizes KiwiSDR wiring, slice assignment, UI sync, waterfall routing, and meter propagation.
src/gui/KiwiSdrApplet.h New: defines a Kiwi status applet model for receiver state + slice assignment display.
src/gui/KiwiSdrApplet.cpp New: implements a lightweight receiver-status list UI for configured Kiwi profiles.
src/gui/AppletPanel.h Exposes the Kiwi applet accessor.
src/gui/AppletPanel.cpp Adds the Kiwi applet to the drawer and default applet order.
src/core/SpectralNR.cpp Fixes packet/burst processing by internally chunking >hop calls to avoid OLA ring wrap aliasing.
src/core/KiwiSdrProtocol.h New: protocol helpers for parsing frames, sequence gap accounting, waterfall display mapping, and meter reading representations.
src/core/KiwiSdrProtocol.cpp New: implements protocol parsing + waterfall/meter helper logic used by the Kiwi client/manager and tests.
src/core/KiwiSdrManager.h New: manages Kiwi profiles, persistence, per-slice assignment, and exposes decoded audio/waterfall/meter signals.
src/core/KiwiSdrManager.cpp New: implements profile CRUD, settings JSON persistence, reconnect scheduling, and per-profile client orchestration.
src/core/KiwiSdrClient.h New: declares the Kiwi WebSocket client interface, state, telemetry, and decoded output signals.
src/core/AudioEngine.h Extends RX mixing/DSP plumbing to accept Kiwi audio sources (per-source gain/mute/enable) and separate NR2 state.
CMakeLists.txt Adds Kiwi core/gui sources and registers the new kiwi_sdr_protocol_test executable.

Comment thread src/core/KiwiSdrProtocol.cpp Outdated
@rfoust rfoust force-pushed the codex/kiwisdr-cleanroom-capture branch from 04ebdeb to 8463b85 Compare June 19, 2026 14:41
@aethersdr-agent

Copy link
Copy Markdown
Contributor

Thanks for this, @rfoust — this is a seriously thorough clean-room write-up, and the receive-only KiwiSDR integration is a great addition. 🙏 Let me help you get CI green.

What actually failed

On commit 04ebdeb, the jobs split like this:

Job Result
build (Linux) ❌ failed at the Build step (Configure + DFNR setup succeeded)
check-macos ✅ passed
check-windows 🟡 still running when I looked
analyze (cpp) (CodeQL) 🟡 still running

So this is compile/link failure on Linux only, while macOS built fine. (build job log)

A transparency note: the diagnostics I can pull surface the failing job and step, but not the raw compiler line, so I'm not going to quote an error I can't see — please open the Build step in the log above for the exact message. That said, the macOS-passes/Linux-fails pattern is a well-worn signature in this repo, and it's almost always one of two things:

1. A Qt API newer than the Linux CI's Qt floor (most common)

The Linux job builds against Qt 6.4.2, while the macOS/Windows runners use a newer Qt. Anything you reached for that landed in Qt 6.5+ will compile locally and on macOS but break the Linux job. Worth scanning the new KiwiSdr* and touched SpectrumWidget/AudioEngine/SMeterWidget code for any 6.5+ method (and double-checking against the Qt "since" annotations in the docs).

2. A header GCC needs but Clang pulled in transitively

macOS uses Clang/libc++; Linux uses GCC/libstdc++, which is stricter about explicit includes. For binary-frame parsing like this, the usual offenders are <cstdint>, <cstring> (std::memcpy), <algorithm> (std::clamp/std::nth_element), <cmath>, and <vector>. KiwiSdrProtocol.cpp looks clean here, but the larger KiwiSdrClient.cpp / KiwiSdrManager.cpp patches were truncated in the API response, so I couldn't verify them — those are the files I'd check first.

One extra thing worth confirming since this feature is WebSocket-based: your header guards QWebSocket behind #ifdef HAVE_WEBSOCKETS. Make sure no symbol outside that guard depends on it, in case the Linux CONFIG resolves HAVE_WEBSOCKETS/Qt6::WebSockets differently than your local box — that would compile on a machine that has the module and fail on one that doesn't.

How to reproduce locally

Easiest path is to build against the same Qt the Linux job uses (6.4.2) rather than your system Qt — that surfaces both classes of failure immediately. See CONTRIBUTING.md / AGENTS.md for the build setup.

On the Copilot comment

The Copilot note (KiwiSdrProtocol.cpp:50 — use kObservedExtendedSoundFrameBytes instead of repeating the literal 1034) is a good catch and worth doing, but it's a style cleanup, not the build break — repeating a magic number compiles fine. Folding it into the same push is a nice-to-have.

Once you post (or I can re-check) the exact Build error, I'm happy to point at the specific fix. Thanks again for volunteering the time on such a well-documented PR! 🚀


🤖 aethersdr-agent · cost: $5.5958 · model: claude-opus-4-8

@rfoust

rfoust commented Jun 19, 2026

Copy link
Copy Markdown
Collaborator Author

@AetherClaude review this

@rfoust rfoust force-pushed the codex/kiwisdr-cleanroom-capture branch from 8463b85 to c334f3b Compare June 19, 2026 18:50
Implements receive-only KiwiSDR virtual RX antennas, audio mixing through the RX DSP path, Kiwi waterfall/FFT rendering, S-meter integration, persisted profile settings, and the clean-room design note.
@rfoust rfoust force-pushed the codex/kiwisdr-cleanroom-capture branch from c334f3b to ab51b29 Compare June 19, 2026 19:15
@rfoust

rfoust commented Jun 19, 2026

Copy link
Copy Markdown
Collaborator Author

Updated the branch with the cleaned KiwiSDR waterfall/pan routing fix. This replaces the previous pushed attempt.

Fixes addressed:

  • Accept small server-side W/F start snapping and render rows using the frame-reported range, so valid Kiwi waterfall frames are not dropped when the server snaps the requested start by a tiny amount.
  • Route Kiwi waterfall/FFT rows by the profile assigned to the selected slice/pan, avoiding stale pan/profile checks when moving a Kiwi receiver between panadapters.
  • Resolve the antenna overlay target slice from the menu pan before emitting Kiwi/Flex antenna selection, so inactive/non-current panadapter menus target the correct slice.

Validation run:

  • cmake --build build --parallel
  • ctest --test-dir build -R "kiwi|spectral_nr" --output-on-failure
  • git diff --check
  • python3 tools/check_a11y.py (warning-only existing findings)

@aethersdr-agent aethersdr-agent Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this, @rfoust — this is a genuinely careful, well-structured contribution. The clean-room boundary is clearly documented, the protocol parsers are defensive about frame sizes, the audio/DSP integration follows the existing source-FIFO pattern, and the conventions I'd normally flag are all respected: receive-only RX antenna profiles persist under a single feature-owned JSON key via AppSettings (no flat keys), sockets are owned through deleteLater()/RAII with disconnect(this) before teardown, cross-thread flags use std::atomic, and auth passwords are redacted in command logs. Nice work.

A few small notes, none blocking:

1. Magic number — confirming the Copilot finding (valid).
src/core/KiwiSdrProtocol.cpp:50 hardcodes the literal 1034:

const bool observedExtendedFrame = frame.size() == 1034;

The constant kObservedExtendedSoundFrameBytes = 1034 is defined at line 11 and already used by extractMeterFromSndVerifiedLayout at line 155. Using the constant here keeps the header parser and the meter extractor from drifting if the observed frame size ever changes. (Note KiwiSdrClient.cpp:31 independently defines its own kObservedSoundFrameBytes = 1034 — three copies of the same observed value across two TUs; worth at least the one-line fix Copilot suggests, and ideally a shared constant.)

While there: the && frame.size() >= 8 on line 51 is dead — observedExtendedFrame already implies size == 1034. Harmless, but it reads as if a non-1034 path could reach the branch.

2. Minor consistency in the multi-source feed.
In AudioEngine::feedKiwiSdrAudioData(sourceId, ...) (AudioEngine.cpp:1917), the non-NR2 path does source->rxBuffer.append(...) directly, whereas the legacy single-source variant routes through processRxAudioData(). The buffer-append-without-lock matches the established Flex/legacy non-NR2 pattern (processRxAudioData:2147), so this isn't a new race — just calling out that the two Kiwi feed overloads take different routes to the same end, which is easy to miss when maintaining one and not the other.

On CI: the test plan leaves "Existing tests pass in CI" unchecked. I wasn't able to pull the check-run status for the head SHA through tooling, so I can't confirm it either way here — worth a glance before merge given the size of the AudioEngine/SpectrumWidget surface this touches.

Scope looks clean — the files not named in the description (VfoWidget, SliceModel, RadioModel, SMeterWidget) all line up with selecting Kiwi profiles from the slice/panadapter antenna controls and routing the meter/waterfall, so nothing reads as out-of-scope.

Overall this is in good shape. The only thing I'd actually ask you to change is the 1034 literal.


🤖 aethersdr-agent · cost: $5.1299 · model: claude-opus-4-8

…th (aethersdr#3668 review)

Two review fixes for PR aethersdr#3668:

1. Audio-source leak: removeKiwiSdrAudioSource() existed but had no caller, so
   removing a KiwiSDR profile only *disabled* its AudioEngine source — the
   per-source DSP state (NR2/resampler/FIFOs) stayed allocated in
   m_externalKiwiSources forever, growing across add/remove cycles. Add a
   KiwiSdrManager::audioSourceRemoved(id) signal, emit it from removeProfile()
   after the client is torn down (so no further audio is fed for that id), and
   wire it on the audio thread (QueuedConnection) to removeKiwiSdrAudioSource(),
   which erases the unique_ptr under m_dspMutex.

2. Strip a checked-in developer-specific absolute path
   (/Users/rfoust/Documents/source/AetherSDR) from the clean-room design doc.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
ten9876
ten9876 previously approved these changes Jun 20, 2026

@ten9876 ten9876 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Approved

Reviewed across Principle I (Flex protocol authority), Principle V (settings), audio-path thread-safety, and clean-room provenance — all pass. CI was green; the clean-room record in docs/kiwisdr-cleanroom-design.md is credible (no AGPL/copied snippets; constants framed as observed/experimental).

Highlights

  • Principle I — PASS: Kiwi profiles are namespaced KIWI: and guarded at every antenna-selection entry point (VfoWidget / RxApplet / SpectrumOverlayMenu) before any rxant= command. No path leaks a Kiwi name to the radio; the real Flex antenna value is preserved while a Kiwi antenna is active.
  • Audio — thread-safe & mixing-correct: single-thread producers → QueuedConnection → single-thread audio consumer; sums to float, scales by 1/N, clamps; per-source NR2/resampler/FIFO isolation. The SpectralNR overlap-add fix (with its block-size-invariance regression test) is a genuine correctness win.
  • Principle V — PASS: one nested-JSON key KiwiSdrRxAntennas, robust to malformed JSON, fields clamped.

Governance: covered by the now-approved RFC #3613 (broadened to cover KiwiSDR; the "virtual RX antenna" surface is the approved go-forward design for remote receivers).

Review fixes folded in (commit a82d38d, signed):

  1. Audio-source leak — removeKiwiSdrAudioSource() had no caller, so profile removal only disabled the source and leaked its DSP state. Added KiwiSdrManager::audioSourceRemoved → wired on the audio thread to free the entry.
  2. Stripped a checked-in developer-specific absolute path from the design doc.

Both Kiwi/NR2 tests pass locally; clean build. Excellent work @rfoust.

@ten9876 ten9876 enabled auto-merge (squash) June 20, 2026 01:41
…aethersdr#3668 review)

parseSoundFrameHeader() hard-coded the extended-frame size literal (1034) while
the meter-extraction check already uses kObservedExtendedSoundFrameBytes. Use the
named constant in both so the parser and meter checks can't drift if the observed
frame size ever changes. (Copilot review nit.)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

@ten9876 ten9876 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-approving after the Copilot review fix (6918050): parseSoundFrameHeader now uses kObservedExtendedSoundFrameBytes instead of the repeated 1034 literal, so the parser and meter-extraction size checks share one constant. Build + kiwi protocol test pass locally. Substance unchanged from the prior approval (Principle I guard, Principle V settings, thread-safe audio, clean-room provenance all verified).

@ten9876 ten9876 merged commit dff3a87 into aethersdr:main Jun 20, 2026
5 checks passed
ten9876 added a commit that referenced this pull request Jun 20, 2026
## Summary

Follow-up to #3668. KiwiSDR currently borrows the shared
`aether.protocol` logging category (so its messages can't be toggled
apart from SmartSDR protocol logging), and four of its five source files
log nothing at all. This adds two dedicated QLoggingCategory entries
that surface as checkboxes in **Help → Support → Diagnostic Logging**
(auto-built from `LogManager`'s category list).

## What changed

- **`aether.kiwisdr` — "KiwiSDR" (default ON):** the
connection/diagnosis story — connect/disconnect, handshake,
audio-rate/waterfall negotiation, reconnect scheduling, profile
add/remove, slice assignment, and the virtual-RX-antenna select/clear
(which logs that **no antenna command is sent to the radio — Principle
I**).
- **`aether.kiwisdr.audio` — "KiwiSDR Audio/DSP" (default OFF,
verbose):** per-source audio enable/disable/remove and frame-shape
diagnostics. Mirrors the existing `aether.audio` /
`aether.audio.summary` split so the high-rate decode chatter is
separately toggleable.
- **Migrated** the 14 existing `KiwiSdrClient` calls off
`aether.protocol` onto these categories, so KiwiSDR logging toggles
independently of SmartSDR protocol logging.
- **Added** lifecycle logging to the previously-silent `KiwiSdrManager`
and `MainWindow_KiwiSdr`.
- Warnings (unsupported stream shapes, invalid passband) always pass the
filter regardless of toggle.

## Notes

- No new flat `AppSettings` keys — the Support dialog auto-persists each
category under its existing `LogCategory_*` key; only the two
`m_categories` rows + `defaultOn` entry are added.
- Cross-platform: pure `QLoggingCategory` / `LogManager`, no
platform-specific code.

## Test plan

- [x] Builds clean on current `main`
- [x] `kiwi_sdr_protocol_test` + `spectral_nr_test` pass
- [x] Verified both categories register and appear in the Support dialog
(auto-built from the category list)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
ten9876 pushed a commit that referenced this pull request Jun 21, 2026
…ng (#3700) (#3701)

Closes #3700. #3668 introduced an unconditional native+kiwi double reproject on every pan step (no KiwiSDR guard), whose stream-state save/restore left m_waterfall COW-shared — forcing a full waterfall-image deep-copy per pan step (60-80 ms UI stall on wide/high-DPI displays), re-introducing the cost #3578 had removed. Reproject only the active stream; remap the other stream to the current frame when it next becomes visible (exact — each history row carries its own capture frame). AETHER_WF_KIWI_ALWAYS=1 restores the old double pass for A/B.

🤖 Generated with [Claude Code](https://claude.com/claude-code)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants