Skip to content

Add Advanced Spot Coloring (DXCC status, LoTW activity, per-callsign worked status) #2809

Description

@TnxQSO-Admin

Request preparation

  • I used an AI assistant to help structure this request
  • I checked for existing issues covering the same feature

What would you like?

Epic issue. This describes a coherent multi-dimensional spot-coloring system that can reasonably be implemented in phases. The visual model is designed as one piece so that the three information dimensions complement rather than conflict with each other. Real-time logbook lookup against external loggers (e.g. Wavelog API) is out of scope and will be filed as a separate follow-up RFE.

What

Extend AetherSDR's existing DXCC Coloring with an opt-in Advanced Coloring mode that conveys three orthogonal information dimensions about each spot at a glance, using three different visual channels:

Dimension Visual channel Levels
DXCC status (New / New Band / New Mode / Worked) Background color 4
LoTW activity of the spotted operator Text color 2
Per-callsign worked status Opacity 2 (full / reduced)
Per-callsign confirmed (LoTW QSL received) Small colored dot on the spot pill 2 (shown / not shown)

Plus a fifth, top-priority override:

| My own callsign (from RadioModel::callsign()) | Configurable background + text color, no opacity change, no dot | 1 |

Existing "DXCC Coloring" remains as Basic Coloring. When the user enables Advanced Coloring, the existing Basic settings are visually greyed out and the new Advanced section drives the rendering. Disabling Advanced restores the existing behavior bit-for-bit. No migration of existing settings is performed; new keys are additive.

Why

Today's DXCC Coloring answers exactly one question: "is this DXCC entity needed, and at which granularity?" That is genuinely useful, but it leaves a lot of operationally interesting information on the table:

  • LoTW activity of the spotted station. Even an otherwise-attractive spot is less valuable to a confirmation-focused operator if the station is not active in LoTW. Knowing this up front saves time. Many operators want a printable, sortable, exportable confirmation; if the other side never uploads to LoTW, a worked QSO will likely never be confirmed.
  • Have I worked this exact callsign before? DXCC-level worked status answers "have I worked the entity?", but not "have I worked W1AW specifically?". An operator chasing a new callsign on a band+mode where the entity is otherwise worked needs the per-callsign view.
  • Have I confirmed this exact callsign in LoTW? Different question again, and separately informative.

Wave-Flex Integrator already proves the value of overlaying these dimensions on the panadapter. Operators who use AetherSDR plus WFI today get part of this benefit through WFI's spot sender, but AetherSDR cannot drive the rendering itself without WFI. Bringing this native into AetherSDR gives users full control through the existing SpotHub UX and removes the dependency on an external tool.

How Other Clients Do It

  • Wave-Flex Integrator (WFI). Configurable per status: separate background color, accent/text color, and opacity sliders for DXCC-needed status (Global, Band, Band & Mode), plus separate opacity blocks for "Call Confirmed" and "Worked Before" at the same three granularity levels. Spots painted on the FlexRadio panadapter via the FlexLib spot protocol. Tooltips spell out the status in plain language (e.g. "DXCC needed for band. New callsign. LoTW inactive.").
  • SmartSDR for Windows. Background-only coloring tied to DXCC need status; no LoTW awareness, no per-callsign awareness. Roughly equivalent to today's AetherSDR Basic Coloring.
  • N1MM Logger / Log4OM. Per-callsign worked status is shown in their bandmaps with different row coloring/icons, not in a panadapter overlay. They also display LoTW user-activity awareness when the user imports the ARRL list.
  • HRD / Logger32. Bandmap rows colored by DXCC need; some support an external LoTW user list for "LoTW user" highlighting.

The proposal here is closest to WFI's model, adapted to AetherSDR's existing SpotHub architecture and rendering pipeline.

Suggested Behavior

Coloring Mode toggle

Add a Coloring Mode selector to Settings > SpotHub > Display:

  • Basic (default, preserves current behavior): the existing DXCC Coloring block drives all spot colors. No changes to today's user experience.
  • Advanced: the new Advanced Coloring block (described below) drives all spot colors. The Basic block is visually greyed out (controls disabled) but its values are preserved in settings.

The toggle has an explicit Reset Advanced to Defaults button below the Advanced controls.

Dimension 1: DXCC status (background color)

Four background colors, configurable, with sensible defaults:

  • New DXCC (not worked on any band, any mode)
  • New Band (entity worked, but not on this band)
  • New Mode (entity worked on this band, but not in this modeGroup)
  • Worked (entity worked on this band and modeGroup)

Priority is fixed top-to-bottom: a spot that qualifies as New DXCC is colored as such, regardless of any "lower" classification.

modeGroup classification uses AetherSDR's existing 3-bucket normalization (CW / PHONE / DATA) in AdifParser::normaliseMode() and DxccColorProvider::normaliseMode(). See Known Limitations below.

Dimension 2: LoTW activity (text color)

Two text colors, configurable, with sensible defaults:

  • LoTW Active: the spotted callsign is present in the ARRL LoTW user activity list AND its last-upload date is within the configured staleness window.
  • LoTW Inactive or Unknown: the spotted callsign is either absent from the list OR present but with last-upload older than the staleness window.

The staleness window is configurable (default 365 days). The two negative cases (absent vs stale) are visually identical but the tooltip distinguishes them with different wording.

A separate Enable LoTW Coloring checkbox controls whether this dimension is active. When disabled, all spots use the default text color regardless of LoTW status.

Dimension 3: Per-callsign worked status (opacity)

Two opacity levels, configurable:

  • Full opacity: this exact callsign has not been logged on this band and modeGroup.
  • Reduced opacity (configurable percentage, e.g. 50%): this exact callsign has been logged on this band and modeGroup.

Sub-states (worked on another band, worked in another mode, worked but not on this band+mode) are not separate opacity levels because three or four opacity tiers are too hard to distinguish at a glance. Those finer distinctions are surfaced in the tooltip.

Callsign normalization for the lookup strips portable suffixes (/P, /M, /MM, /AM) before comparison, matching the existing convention in CtyDatParser::resolvePrimaryPrefix(). Rationale: if the operator has logged W1ABC and a spot for W1ABC/P arrives, treating them as the same callsign is more useful than treating them as different.

Dimension 4: Confirmed indicator (colored dot)

A small filled circle (e.g. 6 to 8 pixels diameter) drawn in the lower-right corner of the spot pill when:

  • The exact callsign appears in the user's ADIF with LOTW_QSL_RCVD=Y on this band AND in this modeGroup.

Less specific confirmations (confirmed on another band, confirmed in another mode) are not shown as a dot but are mentioned in the tooltip. The dot is only displayed when Advanced Coloring is enabled. Its color is configurable; default suggested: bright green. There is no equivalent for paper-QSL confirmations in this RFE.

Top-priority override: my own callsign

When the spotted callsign equals RadioModel::callsign() (case-insensitive, after stripping portable suffixes), the spot is rendered with a dedicated My Callsign background and text color. No opacity reduction, no confirmed dot. This override applies regardless of all other dimensions. If RadioModel::callsign() is empty (radio disconnected), the override is inactive.

Tooltip

The existing spot tooltip is extended to explain the visual state. Example tooltip lines (one or more depending on the spot):

  • DXCC needed. / DXCC needed for band. / `DXCC needed for band and mode.``
  • LoTW inactive, Not in LoTW.
  • New callsign. / Worked before.
  • LoTW Confirmed
  • Me.

This matches the WFI tooltip style shown in the original discussion.

Configuration UI

New Advanced Coloring subsection in Settings > SpotHub > Display, placed in the left column under the existing DXCC Coloring block, following the existing pattern (header label with border-top: 1px solid #304050, then a QGridLayout). Layout inspired by WFI's color/opacity dialog: rows for each status, color swatch buttons, opacity slider for the two opacity levels, a Reset to Defaults button.

A LoTW Database subsection contains:

  • Status line: Last updated: YYYY-MM-DD HH:MM / Never downloaded, with callsign count when known.
  • Update Now button.
  • Update frequency dropdown: Manual / Weekly (default) / Monthly.
  • Staleness threshold numeric input in days (default 365).
  • An info note: LoTW user activity is downloaded from https://lotw.arrl.org/lotw-user-activity.csv so the user understands the data source.

Re-coloring triggers

Today there is no explicit "recompute all colors" trigger; spots only re-color on the next SpotModel event. This is acceptable for the existing Basic flow but is insufficient when the user is configuring Advanced Coloring and expects to see changes immediately. The implementation must:

  • Emit a colorsChanged (or similar) signal from the new provider when any Advanced setting changes.
  • Connect it in MainWindow to SpotModel::refresh() so that spotsRefreshed fires and rebuildSpots re-evaluates every visible spot.

The same trigger fires on ADIF re-import and on completion of a LoTW database update.

Settings keys (new, all under AppSettings)

All keys use existing PascalCase / True-False conventions. None of these collide with the current Basic keys (IsDxccColoringEnabled, DxccColor*, DxccAdifFilePath).

Key Type Default
AdvancedSpotColoringEnabled String "True"/"False" "False"
AdvSpotBgNewDxcc hex color TBD by maintainer
AdvSpotBgNewBand hex color TBD
AdvSpotBgNewMode hex color TBD
AdvSpotBgWorked hex color TBD
AdvSpotTextLotwActive hex color TBD
AdvSpotTextLotwInactive hex color TBD
AdvSpotOpacityFull int (percent) 100
AdvSpotOpacityReduced int (percent) 50
AdvSpotConfirmedDotColor hex color "#00C853" or similar
AdvSpotMyCallsignBg hex color TBD
AdvSpotMyCallsignText hex color TBD
LotwColoringEnabled String "True"/"False" "False"
LotwStalenessThresholdDays int 365
LotwUpdateFrequency string enum "Weekly"
LotwLastUpdated ISO 8601 timestamp ""

Final default colors should be chosen by the maintainer for visual harmony with the existing dark theme.

Protocol Hints

No SmartSDR protocol changes are needed. This is entirely client-side based on data the client already receives or downloads.

Data sources:

  • ARRL LoTW user activity CSV: https://lotw.arrl.org/lotw-user-activity.csv. ~230 000 rows currently. Format per row appears to be callsign,YYYY-MM-DD,HH,MM,SS representing the last upload timestamp. Worth confirming column meanings during implementation.
  • ADIF file already loaded by the existing DxccAdifFilePath setting and parsed by AdifParser.

Existing infrastructure to reuse (file references confirmed by code inspection):

  • ADIF parsing on a worker thread: src/core/AdifParser.{h,cpp} (Q_INVOKABLE + moveToThread + Qt::QueuedConnection back to GUI thread). Same pattern is the right home for a new LotwCsvFetcher worker.
  • DXCC entity lookup: src/core/CtyDatParser.{h,cpp} with resolvePrimaryPrefix() that already strips /P, /M, /MM, /AM. Reuse for portable-suffix normalization across all three lookups.
  • Color provider: src/core/DxccColorProvider.{h,cpp} owns the parse thread and emits importFinished. A peer AdvancedSpotColorProvider is one option; extending DxccColorProvider is another. Decision left to the implementer.
  • Network downloads: QNetworkAccessManager already used in FirmwareStager, PropForecastClient, SpotCollectorClient, SmartLinkClient, WsjtxClient. No shared singleton — each feature owns its own. Follow the FirmwareStager or PropForecastClient pattern. No new dependency, no new Qt module.
  • Cache directory: QStandardPaths::writableLocation(QStandardPaths::GenericCacheLocation) + "/AetherSDR/" for lotw-users.bin.
  • Spot rendering: src/gui/SpectrumWidget.{h,cpp}, SpotMarker struct around lines 352-365. Opacity is already first-class (existing QPainter::setOpacity() use). Adding new fields requires updating spotMarkersVisuallyEqual() at lines 73-96 so that visual changes actually trigger a repaint.
  • Spot tooltip: spotMarkerTooltip() around lines 6074-6087, returns HTML. Trivial to extend.
  • Settings dialog: src/gui/DxClusterDialog::buildDisplayTab() around lines 1941-2405. New subsection follows the existing header-label + grid pattern.
  • Operator callsign: RadioModel::callsign() returns m_callsign, empty when disconnected. No existing normalization helper; convention is trimmed().toUpper() plus suffix stripping where applicable.

New classes proposed:

  • CallsignWorkedStatus — per-callsign, per-(band, modeGroup) flags for worked and confirmed. Fed from the same AdifParser::finished(QVector<QsoRecord>) signal that DxccWorkedStatus::load() consumes today, via fan-out in DxccColorProvider::onParseFinished(). The QsoRecord struct must be extended to carry the LOTW_QSL_RCVD field (currently dropped).
  • LotwUserRegistry — in-memory QHash<QString, QDate> (callsign → last-upload date), serialized to a small binary cache file at the GenericCacheLocation path above. Estimated ~5 MB RAM and ~3-4 MB on disk for the full ~230 000-entry list. Binary format with a version header so future schema changes can invalidate the cache cleanly.
  • LotwCsvFetcher — worker class with a Q_INVOKABLE downloadAndParse() slot, moved to a thread, emits finished(QHash<QString, QDate>) via QueuedConnection. Modeled on AdifParser and FirmwareStager.

Why no SQLite: AetherSDR currently has no SQL module linkage (grep for QSqlDatabase, Qt6::Sql, etc. returns no hits across the source tree and CMakeLists.txt). Introducing the Qt SQL module solely for one read-mostly lookup table would add a build dependency, a runtime dependency, and review surface that an in-memory QHash plus a small binary cache file does not. If a future feature warrants SQL more broadly, this can be revisited then.

Acceptance Criteria

  1. A new Coloring Mode selector in Settings > SpotHub > Display switches between Basic and Advanced. Switching is non-destructive: both setting groups persist independently and can be returned to.
  2. With Advanced enabled and no ADIF loaded and no LoTW database downloaded, every spot is rendered with a sensible default appearance (a configurable default background and text color); the panadapter does not break.
  3. With Advanced enabled, an ADIF loaded, and a LoTW database downloaded, each spot is rendered with the four dimensions described, correctly reflecting the user's logbook and the LoTW list. The operator's own callsign uses the My-Callsign override.
  4. The Update Now button downloads the LoTW CSV, parses it, persists the binary cache, and triggers a re-color of all visible spots without restarting the application. Cache survives application restart.
  5. Changing any color, opacity, or threshold setting in the Advanced UI triggers an immediate re-color of all visible spots (no need to wait for a new cluster spot to arrive).
  6. The spot tooltip explains, in plain language, why the spot is colored as it is, including the LoTW status (active / inactive / not in LoTW), DXCC status, callsign-level worked status, and confirmation status. WFI-style wording is acceptable.
  7. The Reset to Defaults button restores all Advanced settings to the maintainer-chosen defaults and triggers re-color.
  8. Disabling Advanced (returning to Basic) restores exactly the pre-existing rendering behavior with no visual artifacts left over.
  9. Performance: with 50 visible spots, the additional per-spot lookups (per-callsign worked status, LoTW registry) do not measurably impact the panadapter frame rate. All new lookups are O(1) hash operations.
  10. No use of QSettings. All new settings go through AppSettings with PascalCase keys and True/False string convention.

Known Limitations (intentional, to be documented)

  • modeGroup granularity is coarse. AetherSDR's existing mode normalization buckets all digital modes under DATA and all phone modes under PHONE. So "FT8 needed" cannot be distinguished from "RTTY needed" by background color. Operators chasing band-mode milestones at a finer granularity (e.g. FT8-only) will see false-negatives on New Mode. Improving this requires extending DxccWorkedStatus/CallsignWorkedStatus to store sub-mode information and is intentionally deferred.
  • cty.dat is bundled as a Qt resource and never updated at runtime. Newly-issued DXCC entities (e.g. recently-activated rare prefixes) will not resolve correctly until the bundled file is refreshed in a release. Auto-update of cty.dat is out of scope.
  • LoTW "not in list" vs "in list but stale" use the same text color. Only the tooltip differentiates. Three text colors plus default would push the visual budget past comfortable readability on a busy panadapter.
  • Portable suffix handling is "strip and match". W1ABC/P will be treated as W1ABC for both the worked-status lookup and the LoTW lookup. This is more useful than treating them as distinct for the common case (same operator from a portable location) but is technically lossy.
  • No paper-QSL awareness. Only LOTW_QSL_RCVD=Y drives the confirmed dot. Paper QSLs (QSL_RCVD=Y) are ignored because they cannot be cross-checked proactively before working the station.

Future Work (separate issues to be filed later)

  • Phase 4: real-time logbook lookup. Plugin-style live queries against an operator's logger via API (e.g. Wavelog already exposes a usable API). This would replace or complement the current ADIF-snapshot approach for worked/confirmed status. Out of scope here.
  • Hunt mode framework. This RFE assumes DXCC is the user's "thing being hunted". A configurable Hunt Mode (DXCC, WAS, IOTA, CQ Zone, etc.) would generalize the coloring system. The Advanced Coloring data model should be designed so that adding additional hunt modes is straightforward, but the actual feature is deferred.
  • Submode granularity. Distinguish FT8 from other DATA modes, etc. Requires schema changes in DxccWorkedStatus / CallsignWorkedStatus.
  • cty.dat self-update. Periodic refresh from an upstream source.

Suggested Labels

enhancement, New Feature, GUI, spectrum, maintainer-review

Metadata

Metadata

Assignees

No one assigned

    Labels

    GUIUser interfaceNew FeatureNew feature requestenhancementImprovement to existing featuremaintainer-reviewRequires maintainer review before any action is takenpriority: lowLow priorityspectrumPanadapter and waterfall

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions