You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Add a radio-side display-stream inventory + leak-detection capability to the agent automation bridge (and, ideally, a standing in-app audit) so we can directly observe and prevent a whole class of bugs: leaked panadapter/waterfall streams left allocated on the radio, duplicate pans after reconnect/Multi-Flex races, and lingering streams from a crashed or botched teardown.
Today the client's view (get pans, m_panadapters) is cleaned up on the radio's removed echo, so it always looks clean even when the radio still holds a stream. We have no way to ask "what does the radio actually have allocated, and does it match what we own?" That blind spot is exactly what hid #3843, and it's the same blind spot behind the "leaving pans on the radio" and "duplicate pans during races" reports.
Background
Split out of #3855 (fixes #3843). While fixing the panafall close path I prototyped a streams bridge verb + a UDP-orphan detector and proved the close-path fix with it. I removed that prototype from the bug-fix PR to keep it minimal and low-risk — this issue tracks designing and landing it properly as its own feature.
Key empirical finding from that work (informs the whole design): on a live FLEX-8400M, firmware 4.2.18, the radio stops the waterfall UDP stream when the panadapter is removed — so a waterfall leaked by a missing display panafall remove is a resource left allocated on the radio, not continued UDP traffic. A packet-based detector therefore cannot see it on this firmware; we need a radio-authoritative resource query. (Continued-UDP leaks do happen on older 6000-series firmware — see #268 "waterfall continues to scroll".)
PanadapterStream records any display packet (PCC_FFT 0x8003 / PCC_WATERFALL 0x8004) whose stream id was ever registered this session AND is no longer registered — i.e. a stream the radio is still transmitting that we've let go of.
Bridge surface:
streams → { registeredPanStreams[], registeredWfStreams[], orphanStreams[], orphanCount }, where each orphan is { streamId, kind: "waterfall"|"panadapter", packets, age_ms }.
streams reset → clear the orphan tally to re-baseline a before/after measurement.
Read semantics: an orphan whose packets climbs across reads with small age_ms is a live leak; one that stops growing was a brief in-flight tail.
Design gotchas already discovered (adversarial review of the prototype) — bake these in:
Gate the orphan check on ever-registered ∧ not-currently-registered, not on "live known-set non-empty." The naive !isEmpty() gate has two bugs: (1) it goes blind after pan close all (live set empties → a real leak reads 0), and (2) it false-positives a freshly-created 2nd pan's waterfall during its sub-second registration-lag window.
Hot-path safety: only touch the orphan map on the (rare) orphan branch, under the existing m_streamMutex; bound the map (e.g. 32 entries); clear it (and the ever-registered sets) on disconnect.
Layer B — radio-authoritative display-resource query (catches resource-level lingering, #3843-class on modern firmware) — the missing piece
A verb that asks the radio for its authoritative set of display objects and diffs it against what the client owns:
Re-subscribe / refresh (sub display all) and collect the radio's display pan 0x… … / display waterfall 0x… … status echoes within a short window; and/or read capacity counters if the radio exposes them (available_panadapters / available_waterfalls).
Attribute each radio-side object by client_handle: ours, foreign (another Multi-Flex client), or orphan (no owner / not in our model) → an orphan = a leaked or crashed-session stream.
This is what actually detects the #3843 waterfall resource leak on fw 4.2.18, and it generalizes to detecting duplicate pans (radio has 2 where the client expects 1) and lingering pans from a prior crashed session.
Proposed surface: streams radio (or streams inventory).
Preventing the bug classes (the point of the feature)
Duplicate pans on reconnect / Multi-Flex races — Layer B run at the end of the reconnect sequence (ties into the [codex] Fix panadapter restore across reconnects #3429 stage/reclaim/prune flow): if the radio reports a pan we don't own / a duplicate, surface it and optionally auto-reconcile by pruning the orphan through the (now-correct) RadioModel::removePanadapter.
Lingering pans from a crashed prior session — Layer B on connect: any unowned radio-side display object is a candidate for cleanup.
Optional standing audit — a periodic in-app (debug/automation-gated) check that warns when radio-side display resources exceed client-owned: a continuous leak alarm rather than a manual probe.
No false positives during normal add → close cycles (registration-lag window) or after pan close all.
Reconnecting to a radio that already holds one of our pans surfaces it correctly (ours vs foreign vs orphan).
Docs in docs/automation-bridge.md; proof captured via the bridge.
Implementation notes / references
The aether.protocol log category (log set aether.protocol on, then log tail → events[]) and the on-disk session log at ~/Library/Preferences/AetherSDR/logs/aethersdr-<ts>.log are the proof channels for command-level verification (RadioModel::sendCommand: lines).
sub display all re-subscribe must not disrupt live rendering; collect within a bounded window and attribute by client_handle.
Summary
Add a radio-side display-stream inventory + leak-detection capability to the agent automation bridge (and, ideally, a standing in-app audit) so we can directly observe and prevent a whole class of bugs: leaked panadapter/waterfall streams left allocated on the radio, duplicate pans after reconnect/Multi-Flex races, and lingering streams from a crashed or botched teardown.
Today the client's view (
get pans,m_panadapters) is cleaned up on the radio'sremovedecho, so it always looks clean even when the radio still holds a stream. We have no way to ask "what does the radio actually have allocated, and does it match what we own?" That blind spot is exactly what hid #3843, and it's the same blind spot behind the "leaving pans on the radio" and "duplicate pans during races" reports.Background
Split out of #3855 (fixes #3843). While fixing the panafall close path I prototyped a
streamsbridge verb + a UDP-orphan detector and proved the close-path fix with it. I removed that prototype from the bug-fix PR to keep it minimal and low-risk — this issue tracks designing and landing it properly as its own feature.Key empirical finding from that work (informs the whole design): on a live FLEX-8400M, firmware 4.2.18, the radio stops the waterfall UDP stream when the panadapter is removed — so a waterfall leaked by a missing
display panafall removeis a resource left allocated on the radio, not continued UDP traffic. A packet-based detector therefore cannot see it on this firmware; we need a radio-authoritative resource query. (Continued-UDP leaks do happen on older 6000-series firmware — see #268 "waterfall continues to scroll".)Proposed feature — two complementary layers
Layer A — UDP-orphan detector +
streamsverb (catches continued-UDP leaks, #268-class)PanadapterStreamrecords any display packet (PCC_FFT 0x8003/PCC_WATERFALL 0x8004) whose stream id was ever registered this session AND is no longer registered — i.e. a stream the radio is still transmitting that we've let go of.Bridge surface:
streams→{ registeredPanStreams[], registeredWfStreams[], orphanStreams[], orphanCount }, where each orphan is{ streamId, kind: "waterfall"|"panadapter", packets, age_ms }.streams reset→ clear the orphan tally to re-baseline a before/after measurement.Read semantics: an orphan whose
packetsclimbs across reads with smallage_msis a live leak; one that stops growing was a brief in-flight tail.Design gotchas already discovered (adversarial review of the prototype) — bake these in:
!isEmpty()gate has two bugs: (1) it goes blind afterpan close all(live set empties → a real leak reads 0), and (2) it false-positives a freshly-created 2nd pan's waterfall during its sub-second registration-lag window.m_streamMutex; bound the map (e.g. 32 entries); clear it (and the ever-registered sets) on disconnect.Layer B — radio-authoritative display-resource query (catches resource-level lingering, #3843-class on modern firmware) — the missing piece
A verb that asks the radio for its authoritative set of display objects and diffs it against what the client owns:
sub display all) and collect the radio'sdisplay pan 0x… …/display waterfall 0x… …status echoes within a short window; and/or read capacity counters if the radio exposes them (available_panadapters/available_waterfalls).client_handle: ours, foreign (another Multi-Flex client), or orphan (no owner / not in our model) → an orphan = a leaked or crashed-session stream.{ radioPanCount, radioWfCount, ours[], foreign[], orphan[] }.This is what actually detects the #3843 waterfall resource leak on fw 4.2.18, and it generalizes to detecting duplicate pans (radio has 2 where the client expects 1) and lingering pans from a prior crashed session.
Proposed surface:
streams radio(orstreams inventory).Preventing the bug classes (the point of the feature)
RadioModel::removePanadapter.Acceptance criteria
streamsreports an orphan when the radio keeps streaming a removed pan's waterfall (validate on Multi-panadapter: 2nd panadapter loses FFT spectrum (waterfall continues), cannot be closed — persistent across reboot and sole-client reconnect #268-class firmware, or via a simulated/replayed stream).streams radioreports a radio-side waterfall/pan the client doesn't own after a deliberately botched teardown — i.e. it catches the Closing a panafall-created panadapter never removes the waterfall stream (display panafall remove missing); removePanadapter uses non-existent 'display pan close' #3843 resource leak on fw 4.2.18, which Layer A cannot.pan close all.docs/automation-bridge.md; proof captured via the bridge.Implementation notes / references
aether.protocollog category (log set aether.protocol on, thenlog tail→events[]) and the on-disk session log at~/Library/Preferences/AetherSDR/logs/aethersdr-<ts>.logare the proof channels for command-level verification (RadioModel::sendCommand:lines).sub display allre-subscribe must not disrupt live rendering; collect within a bounded window and attribute byclient_handle.Related
SliceRecreatePolicy).