Skip to content

[dax] Fix macOS DAX RX DT bias from stale audio backlog#1822

Merged
ten9876 merged 1 commit into
aethersdr:mainfrom
jensenpat:aether/dax-dt-live-edge-clamp
Apr 21, 2026
Merged

[dax] Fix macOS DAX RX DT bias from stale audio backlog#1822
ten9876 merged 1 commit into
aethersdr:mainfrom
jensenpat:aether/dax-dt-live-edge-clamp

Conversation

@jensenpat

@jensenpat jensenpat commented Apr 21, 2026

Copy link
Copy Markdown
Collaborator
drift

Summary

This change fixes a long-standing FT8 receive timing problem on macOS DAX where WSJT-X and JTDX could show a consistent positive DT offset even when the radio connection itself had very low network latency.

The fix adds live-edge protection to the macOS DAX RX shared-memory ring so stale audio cannot build up and get delivered to digital-mode clients as if it were current audio. It also keeps lightweight timing counters in the bridge for support and verification, while gating the periodic summary diagnostics behind the existing DAX logging category so normal production logs stay quiet.

Problem

Users reported that FT8 decodes over local Ethernet and Wi-Fi were consistently late by roughly +1.5 s to +1.6 s in both JTDX and WSJT-X. The symptom was severe enough to cause missed QSOs even on otherwise healthy local networks.

A few characteristics made the issue especially suspicious:

  • the problem reproduced in both JTDX and WSJT-X
  • ICMP and normal network timing looked healthy, often in the 1 ms to 12 ms range
  • the DT values were commonly shifted in the same positive direction across many stations
  • the offset was large enough to look like buffered or stale audio rather than transport latency

That pattern strongly suggested a local audio-queueing problem instead of RF propagation, remote-station clock drift, or ordinary LAN/Wi-Fi latency.

Root Cause

On macOS, DAX RX audio is written into a POSIX shared-memory ring in VirtualAudioBridge. That ring is intentionally large enough to hold about two seconds of 24 kHz stereo float samples so the CoreAudio/HAL side can read from it asynchronously.

The bug was that the macOS RX path had no live-edge protection at all. If the consumer side fell behind even briefly, the ring could accumulate a large backlog and continue serving old samples from the past rather than current audio from the radio.

That created a fixed positive FT8 DT bias because the decoder was not hearing the current symbol boundary. It was hearing delayed audio from the shared-memory queue.

This also explains why network diagnostics did not match the observed FT8 delay:

  • ping measured network transport latency
  • the FT8 DT error was caused after packets had already arrived, inside the local DAX RX buffering path

The existing TX path already had stale-backlog protection. The RX path did not. That asymmetry was the key indicator.

What Changed

1. Add RX live-edge backlog clamp on macOS

When DAX RX backlog exceeds a bounded threshold, the bridge now advances the reader to keep only a small recent slice of audio near the writer head.

Current thresholds:

  • target backlog: about 40 ms
  • clamp threshold: about 200 ms

This prevents digital-mode clients from decoding stale audio if the shared-memory reader falls behind.

2. Add lightweight RX timing counters

The bridge now tracks per-channel RX timing state:

  • written sample time
  • trimmed sample time
  • trim event count
  • overrun event count
  • peak backlog within the reporting window

These counters are intentionally small and local to the macOS DAX bridge.

3. Keep diagnostics available without making production logs noisy

Two forms of diagnostics are available:

  • clamp events when the bridge has to jump back to the live edge
  • once-per-second summary timing lines

The summary logs are gated behind the existing aether.dax debug category, and clamp messages are only emitted when DAX info logging is enabled. That keeps the production default quiet while preserving support visibility when troubleshooting is needed.

Why This Is Safe

This change is scoped to the macOS DAX implementation only.

  • VirtualAudioBridge.cpp is compiled on Apple builds
  • Linux continues to use PipeWireAudioBridge.cpp
  • Windows does not use this macOS shared-memory DAX bridge path

The fix does not change demodulation, sample-rate negotiation, rig control, or radio transport. It only prevents stale RX audio from remaining queued in the macOS DAX bridge long enough to skew FT8 timing.

Investigation Notes

During investigation we also checked the application side to make sure the delay was not coming from a hidden client-specific RX ring.

  • WSJT-X showed normal Qt audio buffering behavior with no built-in multi-second queue that explained the symptom
  • JTDX uses a similar QAudioInput-based RX path and did not reveal a client-side RX ring matching the observed DT shift

That helped narrow the failure path back to Aether's DAX RX bridge.

We also added timing summaries temporarily and observed the following before the logging cleanup:

  • RX backlog could briefly grow into the hundreds of milliseconds
  • when clamped, backlog was pulled back toward the live edge immediately
  • after the fix, the large common positive DT bias disappeared

Validation

Build validation

  • cmake -S . -B build -G Ninja
  • cmake --build build --parallel 10

On-air validation

  • reproduced the original symptom before the fix with positive FT8 DT bias across both JTDX and WSJT-X
  • observed the backlog clamp and per-second DAX timing summaries during troubleshooting
  • reduced the shared positive DT bias from roughly +1.5 s / +1.6 s down into the normal few-tenths-of-a-second range
  • ran for about four hours with stable DT and no observed drift
  • completed a clean FT8 QSO at 0.0 delta time with no message retries using the final build

That final on-air behavior strongly supports that the stale-audio backlog was the actual root cause.

Files Changed

  • src/core/VirtualAudioBridge.cpp
  • src/core/VirtualAudioBridge.h

👨🏼‍💻 Generated with OpenAI Codex (GPT-5.4 Pro) and tested by @jensenpat

@jensenpat jensenpat changed the title Fix macOS DAX RX DT bias from stale audio backlog [dax] Fix macOS DAX RX DT bias from stale audio backlog Apr 21, 2026
@jensenpat jensenpat marked this pull request as ready for review April 21, 2026 20:03
@jensenpat jensenpat requested a review from ten9876 as a code owner April 21, 2026 20:03

@aethersdr-agent aethersdr-agent Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @jensenpat — this is a well-scoped fix with excellent investigation notes. The root cause analysis (RX path missing the live-edge protection that TX already had) is convincing, and the on-air validation is thorough.

What looks good

  • Scope is tight: only VirtualAudioBridge.cpp / .h, macOS-only code path, no impact on demod/rig control/transport.
  • The clamp thresholds (40 ms target, 200 ms trigger) are reasonable — generous enough to avoid false trims from normal jitter, tight enough to prevent the ~1.5 s DT bias.
  • Diagnostics are properly gated behind lcDax() debug/info levels — production logs stay clean.
  • The RxTimingStats struct uses value-initialized members and QElapsedTimer::isValid() guard — no uninitialized state risk.
  • The anonymous namespace constants with named units (kRxTargetBacklogFrames, kSamplesPerStereoFrame) are clear.

One concern: TOCTOU race on readPos store

In feedDaxAudio, the clamp logic does:

uint32_t currentRp = block->readPos.load(std::memory_order_acquire);
// ...
if (backlogSamples > kRxMaxBacklogSamples) {
    const uint32_t newRp = wp - kRxTargetBacklogSamples;
    if (newRp > currentRp) {
        block->readPos.store(newRp, std::memory_order_release);

Between the load and the store, the HAL plugin (on the CoreAudio thread) may have already advanced readPos past newRp. The unconditional store would then move readPos backward, causing the HAL to re-read samples it already consumed — a brief audio glitch/repeat.

A compare_exchange_strong would close this:

if (newRp > currentRp) {
    if (block->readPos.compare_exchange_strong(currentRp, newRp,
            std::memory_order_release, std::memory_order_relaxed)) {
        // trim succeeded
        const uint32_t trimmedSamples = newRp - currentRp;
        // ...
    }
    // else: HAL already caught up past our snapshot — no trim needed
}

In practice the race window is tiny and the consequence is minor (a brief glitch in an already-degraded scenario), so this isn't a blocker — but it would be a clean improvement if you want to tighten it up.

Minor notes

  • The two samplesToMs overloads (for uint32_t and uint64_t) could be a single template, but it's fine as-is for two call sites.
  • The static int meterCount[NUM_CHANNELS]{} on line 269 of the base file (not part of this PR) is a pre-existing file-scope concern — not yours to fix here.

Verdict

The fix is correct and well-contained. The CAS suggestion above is the only substantive item. Nice work tracking this down — the TX-had-it-but-RX-didn't asymmetry is exactly the kind of thing that hides for a long time. 73!

@ten9876 ten9876 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verified the fix is correctly scoped: macOS is the only platform with an in-process ring buffer that can accumulate stale audio. Linux uses O_NONBLOCK named pipes (drops at syscall), Windows routes through TCI (TCP backpressure). Atomic ordering is correct, clamp math is sound, logging properly gated. Thanks for the thorough investigation and on-air validation. LGTM.

@ten9876 ten9876 merged commit 196fc88 into aethersdr:main Apr 21, 2026
5 checks passed
@jensenpat jensenpat deleted the aether/dax-dt-live-edge-clamp branch April 22, 2026 11:56
ten9876 added a commit that referenced this pull request Apr 22, 2026
)

Cuts a community-contribution-heavy point release. Eight community PRs
landed plus three AetherClaude crash fixes. Highlights:

Community:
- Fix RADE RX not decoding on Windows (#1820, NF0T)
- Clamp stale DAX RX backlog on macOS — fixes +1.5s FT8 DT bias (#1822, jensenpat)
- Fix TCI DAX resampler crosstalk between slices (#1815, Chaosuk97)
- Fix RX applet pan slider with NR active (#1799, Chaosuk97)
- Fix TCI RX gain default 1.0 → 0.5 to match applet (#1811, NF0T)
- Fix HAVE_SERIALPORT guard (#1812, NF0T)
- Seamless ADIF logbook auto-reload (#1801, Chaosuk97)
- RAC Canada band-plan corrections (#1817, VE3NEM)

AetherClaude crash fixes:
- Applet reorder with floating containers (#1745#1746)
- Lazy-build RadioSetupDialog tabs, dodges Wayland FFmpeg/VDPAU crash (#1776#1777)
- PanAdapter float-freeze — show-after-reset + direct reparent (#1668#1669)

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants