Skip to content

fix(dax/tci): prevent re-assert storm when WSJT-X connects via TCI with DAX active#4010

Closed
NF0T wants to merge 1 commit into
aethersdr:mainfrom
NF0T:fix/dax-tci-reassert-storm
Closed

fix(dax/tci): prevent re-assert storm when WSJT-X connects via TCI with DAX active#4010
NF0T wants to merge 1 commit into
aethersdr:mainfrom
NF0T:fix/dax-tci-reassert-storm

Conversation

@NF0T

@NF0T NF0T commented Jul 4, 2026

Copy link
Copy Markdown
Collaborator

Summary

Fixes #4009

Stops the ~12–15 Hz slice set N dax=<ch> feedback loop that occurs when WSJT-X
connects via TCI while a DAX channel is already active on a slice. Three targeted
changes across two files:

  • onDaxChannelChanged() (MainWindow_DigitalModes.cpp): Do not update
    m_daxSliceLastCh when newCh == 0. The radio emits a transient dax=0 after
    every slice set dax=<ch> command; recording that 0 corrupts the tracker so the
    subsequent dax=<ch> recovery looks like a fresh 0→N transition and fires another
    re-assert. Leaving the tracker at the last real channel means the recovery sees
    oldCh == newCh and early-returns with no re-assert.

  • wireDaxSlice() (MainWindow_DigitalModes.cpp): Seed m_daxSliceLastCh
    with the slice's current channel before connecting the signal. Without this the
    tracker starts at 0 even when the slice already has a live DAX channel, so the
    first TCI re-assert triggers the transient broadcast and the dax=<ch> recovery
    immediately fires a spurious re-assert before the storm guard can stabilize.

  • ensureDaxForTci() (TciServer.cpp): Skip re-assert for channels already in
    m_tciDaxBorrowedChannels. When the DAX bridge already owns the stream,
    dax_clients is ≥ 1; re-asserting triggers the radio's transient dax=0/dax=
    broadcast, which seeds the storm through the bridge's active wireDaxSlice
    connection.

Affects Linux and macOS only — Windows has no DaxBridge type and the entire
reconciler block is absent from that build.

Constitution principle honored

Principle II — The Radio Is Authoritative On Live State. The storm was a
feedback loop between the client's command path (ensureDaxForTci re-assert) and
the radio's truth path (status echo). This fix breaks that loop by ensuring the
client treats the radio's dax=<ch> status echo as authoritative confirmation
rather than a trigger for another command.

Test plan

  • Build clean (ninja, no warnings in changed TUs) on Debian 13 / Qt 6.x
  • Verified against FLEX-8400 firmware v4.2.20.41343: single slice set 0 dax=1
    on WSJT-X TCI connect; no repeating storm; WSJT-X waterfall stable
  • Closing WSJT-X no longer leaves a residual spam loop in the log
  • CI (triggered on push)

Checklist

  • Commits are signed (SSH)
  • No new flat-key AppSettings entries (Principle V)
  • Clean-room implementation — no proprietary source referenced (Principle IV)
  • No MeterSmoother changes
  • No transmit path modified (Principle VI)
  • No documentation changes required (internal reconciler logic only)
  • No GHSA advisory needed (no security surface affected)

On all 8000-series radios, every `slice set dax=<ch>` command — including
the re-assert ensureDaxForTci() sends on audio_start — provokes a transient
broadcast sequence: `slice <id> dax=0` then `slice <id> dax=<ch>`. When the
DAX bridge is active (Linux/macOS), wireDaxSlice() has connected
onDaxChannelChanged() to the slice's daxChannelChanged signal. The transient
dax=0 fires that handler, which wrote m_daxSliceLastCh[sliceId]=0. The
subsequent dax=<ch> recovery then saw oldCh=0 vs newCh=<ch> — a "fresh
assignment" — re-triggered the re-assert, and the loop ran indefinitely at
~12 Hz. Symptom: continuous `slice set 0 dax=1` log spam, smeared WSJT-X
waterfall, FT8 signals displayed at inflated bandwidth (DAX VITA-49 sequence
reset every 80 ms). Closing WSJT-X did not stop the loop.

Root cause introduced by aethersdr#2895 (wireDaxSlice re-assert), not caught by the
aethersdr#3626 stream-create/remove storm fix because that fix's test environment had
no DAX HAL plugin, so m_daxBridge was always null and onDaxChannelChanged
never ran. Windows is immune: the entire bridge block is inside
`#if defined(Q_OS_MAC) || defined(HAVE_PIPEWIRE)`.

Fix (3 changes, 2 files):

1. onDaxChannelChanged: don't write 0 to m_daxSliceLastCh on the transient
   dax=0 broadcast. The tracker keeps the last real channel so the dax=<ch>
   recovery sees oldCh==newCh and early-returns with no re-assert.

2. wireDaxSlice: seed m_daxSliceLastCh with the slice's current channel
   before wiring the signal. Without this the tracker starts at 0 even
   when the slice has a pre-configured channel from the radio profile,
   allowing one extra loop iteration before fix 1 stabilises things.

3. ensureDaxForTci: skip the re-assert for borrowed DAX channels. When TCI
   borrows a bridge-created stream the slice binding is already correct and
   dax_clients >= 1 — re-asserting is unnecessary and is the opening shot
   that starts the chain. The aethersdr#1439 re-assert is preserved for TCI-owned
   (non-borrowed) streams where dax_clients would otherwise stay 0.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@NF0T NF0T requested a review from a team as a code owner July 4, 2026 00:55
@NF0T

NF0T commented Jul 4, 2026

Copy link
Copy Markdown
Collaborator Author

Author response to review

The review's findings are correct. After validating each point against the code and the wire captures, I'm acknowledging two high-severity regressions and recommending this PR be deferred in favour of #3305.


Regressions confirmed

Change 1 — Don't record newCh==0 in m_daxSliceLastCh:
Leaves the tracker stale at the old channel after a genuine user de-assign (slice dax=0). When the user later re-assigns the same channel, onDaxChannelChanged sees oldCh == newCh and early-returns without creating a stream. Repro: DAX 1 on slice A → set DAX Off → wait 2 s → set DAX 1 → silence. The fix cannot distinguish a transient dax=0 (re-assertion echo) from a genuine dax=0 (user action). High severity.

Change 2 — Seed tracker in wireDaxSlice():
The sliceAdded hook deliberately zeros the tracker before calling wireDaxSlice so that the subsequent explicit onDaxChannelChanged(s, ch) call at line 985 sees a 0→ch transition and creates the stream. Our seed inside wireDaxSlice overwrites that zero with the current channel, causing the explicit call to always early-return. On reconnect and profile restore — the normal case per the wire captures — slices arrive with DAX pre-assigned and get silenced. High severity.

Change 3 — Skip re-assert for borrowed channels:
Direction is correct; key is wrong. m_tciDaxBorrowedChannels can go stale relative to actual stream state. The guard should be stream-existence + dax_clients, not the borrow ledger. Medium severity.

Deleted comment:
The PR removed the comment at lines 1141–1146 explaining that SliceModel's equality guard is load-bearing for stopping same-value echoes from re-entering the reconciler. That architectural knowledge should survive whatever lands here.


Root cause stands; proposed fix direction is correct

The diagnosis in #4009 is accurate and the review confirms it via wire captures. The oscillator's gain element is the unconditional slice set %1 dax=%2 at onDaxChannelChanged lines 1138–1148: a command emitted in reaction to a status echo, violating Principle II. Every fix since #3270 has damped one edge of the oscillator without removing that element.

The review's proposed fix — delete the re-assert from onDaxChannelChanged entirely, revert tracker semantics, keep the sliceAdded 0-seed contract, and move the re-assert into ensureDaxForTci as a one-shot legacy fallback tied to stream creation — is the correct approach and cannot oscillate regardless of transient shape.


Recommendation

This PR should be closed or held in favour of #3305 (single DAX authority + refcounting), which implements the structural fix this code path has needed since #2895. #4010 as written would be an obstacle on top of the architecture #3305 needs to establish. I'll defer to the maintainer on timing, but I'm not requesting merge of this PR in its current state.

@aethersdr-agent aethersdr-agent Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this — the issue writeup (#4009) is exemplary: the log trace, the PR-history regression table, and the transient dax=0 root-cause are all traced precisely, and the fix targets the correct link in the feedback loop (Principle II, honored well). The TciServer borrowed-channel skip (change 3) is clearly correct: when the DAX bridge already owns the stream, dax_clients >= 1 and the re-assert only buys you the transient broadcast that seeds the storm. Skipping it is right, and the else if debug log is a nice touch.

Two concerns on the MainWindow_DigitalModes.cpp changes, one of which I think is a functional regression.

1. wireDaxSlice() seeding breaks the sliceAdded profile-restore path (likely blocker)

The new seed in wireDaxSlice():

m_daxSliceLastCh[slice->sliceId()] = slice->daxChannel();

collides with the deliberate = 0 in the sliceAdded handler a few lines up (startDax, ~L975–987):

m_daxSliceConns.append(connect(&m_radioModel, &RadioModel::sliceAdded,
                               this, [this](SliceModel* s) {
    if (!m_daxBridge || !s) return;
    // Let onDaxChannelChanged() see a 0 -> channel transition for slices
    // restored with DAX already assigned.
    m_daxSliceLastCh[s->sliceId()] = 0;   // deliberately seed 0
    wireDaxSlice(s);                        // <-- now overwrites the 0 with daxChannel()
    if (s->daxChannel() >= 1 && s->daxChannel() <= 4) {
        onDaxChannelChanged(s, s->daxChannel());  // oldCh now == newCh -> early return
    }
}));

That handler intentionally seeds 0 so the explicit onDaxChannelChanged(s, s->daxChannel()) sees a 0 -> N transition and issues the stream create. With the new seeding, wireDaxSlice() overwrites the 0 with daxChannel() before that call, so oldCh == newCh hits the early-return at L1125 and the DAX RX stream is never created for a slice that arrives already carrying a channel (radio-profile restore / reconnect prune-and-re-add). That's dax_clients=0 / silence — the exact #1439 symptom, on a common path.

Also worth noting: the stated motivation for this seed ("tracker starts at 0 even when the slice already has a live DAX channel") is already covered for the startDax path by the loop at L953–956 (m_daxSliceLastCh[s->sliceId()] = ch;), so this seed is redundant there and only new on the sliceAdded path — where it's harmful. I'd suggest dropping the wireDaxSlice() seed entirely and confirming the storm is still fixed by changes 1 + 3 alone; or, if it's genuinely needed, reorder so wireDaxSlice() runs before the = 0 seed in the sliceAdded lambda.

2. Change 1 leaves the tracker stale on a genuine de-assign

Not recording newCh == 0 is exactly right for the radio's transient. But the code can't distinguish the transient from a real user de-assign (dax=0), and after a genuine de-assign the tracker stays pinned at the old channel:

  • User sets dax=0: oldCh=1, newCh=0 → proceeds, tracker stays 1, scheduleDaxRxStreamRemoval(1) fires and (no rebroadcast) removes the stream.
  • User later re-assigns the same channel dax=1: oldCh=1 (stale), newCh=1oldCh == newCh early-return → no stream create, no re-assert → silence until something else nudges it.

Re-assigning a different channel recovers fine, so this is narrow, but it's a real behavior change. Is a genuine slice dax=0 reachable from the UI in your setup, or does disabling DAX go through stopDax() (which clear()s the tracker) instead? If the former, worth handling.

Everything else looks good — no QSettings/flat-key AppSettings, no transmit path, RAII/connection bookkeeping unchanged, and the platform scoping matches the issue. If you can confirm the profile-restore stream-create still fires with these changes (a stream create ... dax_channel=N on reconnect with a DAX-assigned slice), that would resolve the main concern.


🤖 aethersdr-agent · cost: $5.0699 · model: claude-opus-4-8

@NF0T

NF0T commented Jul 4, 2026

Copy link
Copy Markdown
Collaborator Author

Closing as superseded by #4017.

#4017 takes the structurally correct approach — centralized refcounted acquireDaxChannel/releaseDaxChannel ownership in PanadapterStream with a strict "no commands from status-echo edges" invariant — which eliminates the re-assert storm by construction rather than patching around it.

This PR's tracker patches (if (newCh != 0) guard and the wireDaxSlice seed) were confirmed to regress two paths independently: DAX off→on retoggle goes silent after a de-assign (stale tracker early-returns), and slices arriving with DAX pre-assigned (profile restore / reconnect replay) never get a stream (seeded tracker defeats the sliceAdded 0→ch transition). Those regressions are documented in the #4010 review discussion and in #4017's PR body.

Tracking continues in #4017.

@NF0T NF0T closed this Jul 4, 2026
@NF0T NF0T deleted the fix/dax-tci-reassert-storm branch July 4, 2026 13:04
ten9876 pushed a commit to jensenpat/AetherSDR that referenced this pull request Jul 4, 2026
…hersdr#3305)

Replace three hand-rolled, mutually-peeking DAX ownership trackers (DAX
bridge m_daxSliceLastCh teardown logic, TCI m_tciDaxStreamIds/
m_tciDaxBorrowedChannels, RADE m_radeDaxStreamId) with refcounted
acquire/releaseDaxChannel(channel, consumer) in PanadapterStream — the
single decider of when a radio-side dax_rx stream exists. RadioModel is
the command plane (stream create/remove + a one-shot aethersdr#1439 dax_clients
nudge tied to the create) and the single, client_handle-filtered stream
status registrar (RADE's old private hook had no handle filter and could
adopt a foreign client's broadcast stream status).

Structural consequences:
- No commands are ever emitted from status-echo edges. The radio answers
  a re-assert of a live dax binding with a transient unbind/rebind
  dax=0/dax=<ch> pair (flex-protocol/state-machines.md §7.4); the old
  unconditional re-asserts in onDaxChannelChanged/ensureDaxForTci turned
  that into the ~12-15 Hz slice-set storm of aethersdr#4009 and the create/remove
  churn of aethersdr#3626. Both are now impossible by construction.
- Last-holder removal runs behind a 1.5 s grace window (absorbs the
  transient pairs; re-acquire cancels), subsuming aethersdr#3626's deferred
  removal and aethersdr#2895's cross-consumer guards.
- Radio-side stream death while held (profile load slice teardown)
  auto-recreates after a backoff — the aethersdr#3476 re-arm, now universal.
- Disconnect resets the table with no commands (radio reaps its own
  streams); slice removal releases the bridge's hold (orphan fix).

Live-proven against FLEX-8400M fw 4.2.18 as a second MultiFlex station:
TCI audio_start -> exactly one stream create + one-shot re-assert, 0
storm commands; DAX bridge toggled on+off across a live TCI stream ->
co-hold/release with no stream remove and uninterrupted audio (the
aethersdr#3363/aethersdr#2886 class); last holder release -> grace -> single stream
remove; clean session release on exit.

Closes aethersdr#3305. Fixes aethersdr#4009. Supersedes the tracker-patch approach of
PR aethersdr#4010 (which traded the storm for off->on retoggle and restored-slice
silence regressions).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
ten9876 added a commit that referenced this pull request Jul 5, 2026
…) — kills the #4009 re-assert storm class (#4017)

## Summary

Implements #3305: replaces the three hand-rolled, mutually-peeking DAX
RX ownership trackers (DAX bridge, TCI, RADE) with **refcounted
`acquireDaxChannel(ch, consumer)` / `releaseDaxChannel(ch, consumer)` in
`PanadapterStream`** — the single decider of when a radio-side `dax_rx`
stream exists. `RadioModel` becomes the command plane (`stream
create`/`stream remove` + a one-shot #1439 `dax_clients` nudge tied to
the create) and the single, `client_handle`-filtered stream-status
registrar.

**Closes #3305. Fixes #4009. Fixes #3669.** Supersedes #4010.

- #4009 — the re-assert storm: eliminated by construction (details
below); live-proven.
- #3669 — TCI audio not starting on connect / not recovering on DAX
toggle / dying when another client connects: every implicated mechanism
(stale `m_tciDaxStreamIds` guard wedging `stream create`, live-binding
re-asserts silencing via the radio's unbind/rebind transient,
toggle-blind channel tracking) is rebuilt by this PR — acquire is
idempotent, radio-side stream death auto-recreates, and no command is
ever echo-triggered. **@reporter verification on 26.7.x requested** — if
the FLEX-6400/Win11 repro survives, please reopen.

Related, intentionally left open: #3715 (slice-lifecycle
single-authority RFC — the slice-side sibling of this stream-side
change) and #3366 (previously diagnosed as a WSJT-X-side watchdog/format
contract issue, not stream ownership).

## Root cause this eliminates

The radio answers a re-assert of a **live** dax binding with a transient
unbind/rebind `dax=0`/`dax=<ch>` status pair. The old code sent
unconditional `slice set <n> dax=<ch>` re-asserts *from status-echo
edges* (`onDaxChannelChanged`, `ensureDaxForTci`) — turning that
transient pair into a self-sustaining oscillator: the ~12–15 Hz `slice
set dax` storm of #4009, and the create/remove churn of #3626 before it.
The invariant this PR enforces: **no commands are ever emitted from
status-echo edges.** Acquire/release are idempotent, and the manager's
removal grace window (1.5 s, cancelled by re-acquire) absorbs the
transient pairs, so the loop is impossible by construction — not patched
around.

Why not #4010's tracker patches: they break the loop but leave the
echo-triggered re-assert in place and regress two paths — DAX off→on
retoggle goes silent (stale tracker early-returns after the stream was
removed), and slices arriving with DAX pre-assigned (profile restore /
reconnect replay) never get a stream (the seeded tracker defeats the
`sliceAdded` 0→ch transition). Details in the #4010 review discussion.

## Design

| Event | Manager behavior |
|---|---|
| First holder acquires a channel | `daxStreamCreateNeeded(ch)` →
RadioModel sends `stream create type=dax_rx dax_channel=<ch>` + one-shot
`slice set <id> dax=<ch>` (#1439, command-initiated, cannot oscillate) |
| Last holder releases | Deferred removal after 1.5 s grace (subsumes
#3626's defer+revalidate); re-acquire cancels |
| Radio destroys a held stream (profile load / slice teardown) | Auto
re-create after 500 ms backoff (#3476's re-arm, now universal —
`rearmDaxForProfileLoad` no longer hand-removes streams) |
| TCP disconnect | Table reset, **no commands** — the radio reaps all of
a client's streams itself; consumers re-acquire on their reconnect
re-arm paths |
| Slice removed with DAX active | Bridge releases the channel
(previously leaked a silent radio-side stream until stopDax) |

Deleted: the unconditional re-assert in `onDaxChannelChanged` (the #4009
gain element), `scheduleDaxRxStreamRemoval`, every
`tciUsing`/`radeUsing` cross-consumer peek (`stopDax`, `deactivateRADE`,
the profile recovery pass), `TciServer::ownsDaxChannel` +
`m_tciDaxStreamIds` + `m_tciDaxBorrowedChannels`, `m_radeDaxStreamId`,
and RADE's private stream-status hook — which had **no `client_handle`
filter** and could adopt a foreign client's broadcast stream status,
silently shadowing the channel. Ownership coordination goes from
O(consumers²) peeking to one enum value per consumer.

## New automation bridge verbs (second commit)

This PR also adds the test surface that proves it, so the
storm/co-hold/grace assertions are reproducible by any agent without
log-grepping:

- **`get dax`** — snapshot of the ownership table: per-channel `holders`
(`bridge`/`tci`/`rade`), `streamId`, `createPending`, plus each slice's
`dax=` assignment.
- **`tci start|status|stop [abrupt]`** — in-process TCI client simulator
speaking the WSJT-X dialect (init burst → `ready;` → `audio_samplerate`
+ `audio_start`, binary-frame counting). `stop abrupt` reproduces the
WSJT-X watchdog-reconnect shape for teardown/reap tests.

Documented in `docs/automation-bridge.md`; `tools/automation_probe.py`
gained the `tci` command.

## Live proof (FLEX-8400M, fw 4.2.18, as a second MultiFlex station)

| Scenario | Result |
|---|---|
| TCI `audio_start` (#4009 trigger) | Exactly **1** `stream create` +
**1** one-shot re-assert; **0** storm commands in the window (pre-fix:
12–15/s) |
| DAX bridge enabled during a live TCI stream | `DAX ch 1 acquired by
bridge holders=0x3` — co-hold, no duplicate create, no re-assert |
| DAX bridge disabled during a live TCI stream | `released by bridge
holders=0x2` — **no stream remove**, TCI audio uninterrupted (2106
frames, steady ~47/s) — the #3363/#2886 failure class |
| TCI client killed (last holder) | grace → exactly **1** `stream
remove` |
| Verb-driven re-run (`tci start` → `get dax` → `tci stop abrupt` → `get
dax`) | `holders:["tci"]` with live stream id → after abrupt stop:
debounce → release → grace → single `stream remove` → channel table
drains to `[]` |
| App exit | Radio session fully released; discovery clean |

Log excerpt:

```
19:32:52 INF PanadapterStream: DAX ch 1 acquired by bridge holders=0x3
19:33:00 INF PanadapterStream: DAX ch 1 released by bridge holders=0x2   ← no stream remove
19:33:26 INF PanadapterStream: DAX ch 1 released by tci holders=0x0
19:33:28 INF PanadapterStream: DAX ch 1 unheld past grace — removing stream 4000008 (#3305)
```

Note: code comments cite
`docs/architecture/flex-protocol/state-machines.md` (the wire-level
contract, including captures of the transient unbind/rebind pair); that
guide lands in a separate docs PR.

## Constitution principles honored

**Principle II — The Radio Is Authoritative On Live State.** Status
echoes are treated purely as state carriers; the client never re-asserts
state in response to them.

## Test plan

- [x] Clean build (ninja, macOS, RelWithDebInfo); no new warnings in
changed TUs
- [x] Full ctest: 53/54 pass; `theme_manager_test` failure is
pre-existing/environmental (reads the developer's live settings file;
unrelated to this change)
- [x] Live-tested against FLEX-8400M via the agent automation bridge as
a second MultiFlex station (table above), including full session cleanup
- [x] New `get dax` + `tci` verbs live-proven end-to-end (start → frames
flowing → abrupt stop → grace removal → table drained)
- [ ] CI (on push)
- [ ] Reporter validation welcome: #4009 hardware (Linux/PipeWire) —
@NF0T; #3669 (FLEX-6400, Win11)

## Checklist

- [x] Commits are signed (SSH)
- [x] No new flat-key `AppSettings` entries (Principle V)
- [x] Clean-room implementation — no proprietary source referenced
(Principle IV)
- [x] No `MeterSmoother` changes
- [x] No transmit path modified (Principle VI) — dax_tx handling
untouched
- [x] Automation docs updated for the new verbs
(`docs/automation-bridge.md`)
- [x] No GHSA advisory needed

💻 Generated with Claude Code (claude-fable-5 7/1/26) with architecture
by @jensenpat

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Jeremy [KK7GWY] <kk7gwy@aethersdr.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

DAX + TCI re-assert storm: continuous "slice set N dax=<ch>" spam when WSJT-X connects via TCI with DAX active (Linux/macOS)

1 participant