Add buffered async file logging#2478
Conversation
Co-authored-by: Codex <noreply@openai.com>
|
Hey @jensenpat — thanks for working on this, the async log writer is a nice direction. 👍 I took a look at the CI failure on What failed
The Linux-only failure pattern, plus a Configure step that succeeded, points at a link-time issue rather than a code issue. Likely cause: missing
|
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Codex <noreply@openai.com>
There was a problem hiding this comment.
Thanks @jensenpat — this is a clean, well-structured implementation. The producer path stays small, the worker thread owns formatting/redaction/I-O, the bounded-queue drop policy is sensible (debug/info first, reserve for high-priority, coalesced summary line), and shutdown is properly drained via a sync point. Scope is exactly what the description claims; the supporting CMake test additions (pthread + AsyncLogWriter source) make sense.
A few observations worth a look before merge:
-
IPv4 regex behavior change for quoted version strings. The previous
redactPiihad(?<![v"])to skip version strings likev0.9.2.1and"0.9.2.1". The new regex insrc/core/AsyncLogWriter.cpp:46keeps thev-prefix protection via\b(sincevis a word char), but"0.9.2.1"will now match (\btriggers between"and0) and be redacted as*.*.*. 1. The PR description mentions "a small tightening for quoted IPv4 addresses" — if that's intentional, fine, but it's worth a quick check that no existing log line shape produces a quoted four-part version that you'd rather see verbatim. (E.g., anythingqDebug() << QString("0.9.2.1")style.) -
logFileSize() constnow blocks on the writer.LogManager::logFileSize()callsflushLog(), which posts a Flush sync-point and waits up tokFlushIntervalMsfor the writer to drain. If anything on the GUI thread polls this for a status display, it can stall the UI by ~250 ms. Worth confirming the only caller is SupportBundle (which already does its own flush) — if any GUI surface reads it, consider makinglogFileSize()return the on-disk size without the implicit flush, and let callers flush explicitly when they need accuracy. -
LogManager::clearLog()race during shutdown.if (m_writer.isRunning()) m_writer.clearLog()— if shutdown sneaks in between theisRunning()check andclearLog(), the writer'senqueueControlAndWaitreturns false silently and the file isn't cleared (no fallback to the synchronous truncate path). It's a narrow window only at teardown, so probably benign, but you could collapse it by makingclearLog()return bool and falling through to the QFile path on failure. -
Minor:
flushAfterBatchand the timer/size-based flush branches duplicate the same "flush+fflush+restart timer" sequence. Pulling that into a smallflushFile()lambda would tightenrun()a bit. Not a blocker.
The lifecycle wiring in main.cpp (uninstall handler before shutdownLogging, leaked LogManager* to survive late teardown) and the SupportBundle pre-flush look correct. Counters/queue stats are a nice touch for future diagnostics.
LGTM with the IPv4-quoted-version question worth a sanity check.
|
Claude here — really clean foundation milestone, jensenpat. Producer-consumer split with the high-priority reserve + drop-summary coalescing is exactly the production-grade pattern (spdlog/glog shape), and the lifetime hardening (heap-allocated singleton + explicit shutdown ordering with qInstallMessageHandler(nullptr) before shutdownLogging) addresses the late-teardown reentrancy that the old code was already hedging against with the heap-allocated regex objects. The two redaction tightenings (IPv4 negative-lookbehind for ver= prefixes, colon-separator MAC pattern) caught real cases the previous regex would miss. Format preservation means existing log analysis tooling keeps working unchanged. CI ran fully on Windows too. Two follow-ups filed (#2497 unit tests, #2498 rotation) — both are tracker hygiene, not blockers. Merged. 73, Jeremy KK7GWY & Claude (AI dev partner) |
Pins down the public-API behavior of AsyncLogWriter so future tightening of the PII redaction regexes (#2478) cannot regress without a test failing. Cases mirror the suggested set in the issue: - Line shape preservation ([HH:mm:ss.zzz] LVL cat: msg\n) for every QtMsgType label - IPv4 redaction, ver= exemption, and lock-in that quoted four-octet IPs are still redacted (per triage: quoting is not authentication of intent). The original "0.9.8" case was dropped as non-applicable (three octets never match the regex) and replaced with the explicit three-octet pass-through assertion. - Radio serial, id_token, MAC dash, and MAC colon redaction - clearLog truncates pre-clear lines and preserves post-clear lines - shutdown drains the queue (no loss on graceful stop) - Drop accounting under burst load (asserts inequalities since the worker drains concurrently; per triage open-question #2) - High-priority reserve keeps a critical line alive under a debug burst, and droppedHighPriorityLines stays at 0 - Stderr mirroring matches the file contents byte-for-byte when mirrorToStderr=true Test uses only the public API (no test-only hooks added to AsyncLogWriter), and routes the log path through QTemporaryDir so the suite does not litter /tmp. Blast radius: risk_score=0.000, 0 high-risk affected (AsyncLogWriter itself has self_betweenness=0; pure additive test + CMakeLists entry).
Summary
Implements the first foundation milestone for lower-overhead AetherSDR logging from issue #2471.
AsyncLogWriter, a dedicated logging thread owned byLogManager.aethersdr.loglatest-log symlink behavior.clearLog()deterministic by sending a writer-thread clear command that drains prior writes, truncates the active file, then resumes subsequent writes.Notes
This intentionally does not add event classification, a new taxonomy, or UI settings. It also does not touch radio command ordering, timers, slice behavior, panadapter behavior, or command sequencing code.
The redaction logic is preserved and carried forward onto the writer thread, with a small tightening for quoted IPv4 addresses and colon-separated MAC addresses observed during runtime verification.
Validation
env -u CPLUS_INCLUDE_PATH cmake --build build-ninja --parallel 22ctest --test-dir build-ninja --output-on-failureaethersdr.logpoints to the latest timestamped log[HH:mm:ss.zzz] DBG aether.protocol: messageAsyncLogWriterprobe to verify:clearLog()truncates prior buffered writes and preserves subsequent writes👨🏼💻 Generated with OpenAI Codex (GPT-5.5 Pro 4/23) and tested by @jensenpat