Skip to content

fix(mqtt): stop lastHeard-refresh upserts from clobbering NodeInfo#3456

Merged
Yeraze merged 1 commit into
mainfrom
fix/mqtt-nodeinfo-clobber
Jun 14, 2026
Merged

fix(mqtt): stop lastHeard-refresh upserts from clobbering NodeInfo#3456
Yeraze merged 1 commit into
mainfrom
fix/mqtt-nodeinfo-clobber

Conversation

@Yeraze

@Yeraze Yeraze commented Jun 14, 2026

Copy link
Copy Markdown
Owner

Problem

Many nodes seen on MQTT had no name (longName/shortName) and hwModel: 0, despite their NodeInfo being received. Investigation showed this was not a decoding failure and not a de-dup failure — node de-dup on (nodeNum, sourceId) is correct and produces no duplicate rows. NodeInfo was being decoded and saved fine, then wiped microseconds later.

Root cause

The MQTT POSITION / TEXT / TELEMETRY / PAXCOUNTER / Store-&-Forward-heartbeat handlers each upsert the sender node only to refresh lastHeard, but hardcoded:

longName: '',
shortName: '',
hwModel: 0,

The upsert merge treats an empty string / 0 as a provided value, not "absent":

  • SQLite (upsertNodeSqlite): setIfProvided guards only on !== null && !== undefined, so ''/0 get written.
  • PG/MySQL: nodeData.longName ?? existing'' is not nullish, so it passes through.

A node broadcasts NodeInfo only every few hours but sends position/telemetry constantly, so the name was overwritten back to empty almost immediately after every NodeInfo packet. Net effect: MQTT nodes appear nameless nearly all the time.

This is unique to the MQTT ingest path — the TCP path omits the name fields on non-NodeInfo updates, so its ??/setIfProvided merges already preserve them.

Fix

Omit longName/shortName/hwModel from the five lastHeard-refresh upserts so the merge preserves whatever NodeInfo already exists. The NODEINFO_APP handler — the one upsert that legitimately carries real values — is untouched.

Tests

Adds a regression suite (lastHeard refresh must not clobber NodeInfo):

  • POSITION / TEXT / TELEMETRY / PAXCOUNTER / S&F-heartbeat refresh upserts now have longName/shortName/hwModel undefined (not ''/0) while still setting lastHeard.

  • Positive control: NODEINFO_APP still saves the real name/shortName/hwModel.

  • mqttIngestion.test.ts: 40/40 pass

  • Full suite: 6403 pass, 0 fail, 0 suite failures

  • tsc --noEmit: clean

🤖 Generated with Claude Code

The MQTT POSITION/TEXT/TELEMETRY/PAXCOUNTER/S&F-heartbeat handlers each
upsert the sender node to bump lastHeard, and previously hardcoded
longName: '', shortName: '', hwModel: 0. The upsert merge treats an empty
string / 0 as a provided value (setIfProvided on SQLite, ?? on PG/MySQL),
so the next position/telemetry packet after a NODEINFO_APP packet wiped
the saved name. Since nodes broadcast NodeInfo only every few hours but
position/telemetry constantly, MQTT nodes appeared nameless almost always.

Omit longName/shortName/hwModel from these refresh upserts so the merge
preserves existing NodeInfo. The NODEINFO_APP handler (the one upsert that
legitimately carries real values) is untouched. Node dedup on
(nodeNum, sourceId) was already correct — this was a clobber, not a dup.

Adds a regression suite asserting refresh upserts no longer carry name
fields while NODEINFO_APP still saves them.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@github-actions

github-actions Bot commented Jun 14, 2026

Copy link
Copy Markdown

Claude finished @Yeraze's task —— View job


Code Review Complete ✅

Reviewing PR #3456: "fix(mqtt): stop lastHeard-refresh upserts from clobbering NodeInfo"

Review Checklist:

  • Read repository guidelines (CLAUDE.md)
  • Analyze the main implementation changes (mqttIngestion.ts)
  • Review the test coverage (mqttIngestion.test.ts)
  • Evaluate code quality and best practices
  • Check for potential bugs or issues
  • Assess performance considerations
  • Review security concerns
  • Provide comprehensive feedback

Summary

This is an excellent fix for a subtle but significant bug in MQTT ingestion. The problem was well-diagnosed and the solution is both minimal and correct.

Code Quality & Best Practices ✅

Strengths:

  • Clear problem identification: The issue was correctly identified as hardcoded empty values being treated as "provided" values in upserts
  • Surgical fix: Changes are minimal and targeted, affecting only the problematic fields
  • Consistent application: The fix is applied consistently across all affected handlers (POSITION, TEXT, TELEMETRY, PAXCOUNTER, Store & Forward heartbeat)
  • Preserves intended behavior: NODEINFO_APP handler remains untouched as it should provide real values
  • Good commenting: Clear inline comments explain why fields are omitted (lines 255-259, 340-344, 393-397, 764-766, 855-857)

Potential Issues ✅

No significant issues found:

  • Database compatibility: The change from explicit empty values to undefined is safe across all three database backends (SQLite/PostgreSQL/MySQL)
  • Backward compatibility: No breaking changes to existing data or APIs
  • Type safety: Changes maintain TypeScript type safety with Partial<DbNode>

Performance Considerations ✅

Positive impact:

  • Reduced database writes: By omitting unchanged fields, the upsert operations become more efficient
  • Better database query planning: Undefined fields won't trigger unnecessary column updates
  • No performance regressions: The fix doesn't add any new database operations or complex logic

Security Concerns ✅

No security issues:

  • No data exposure: Fix only affects how node metadata is preserved, no sensitive information handling changes
  • Input validation preserved: All existing input validation and sanitization remains intact
  • Per-source scoping maintained: The fix doesn't affect the sourceId scoping that prevents data leakage

Test Coverage ✅

Comprehensive test suite:

  • Regression test suite: Lines 758-835 specifically test that lastHeard refresh upserts don't clobber NodeInfo
  • Complete coverage: Tests all affected packet types (POSITION, TEXT, TELEMETRY, PAXCOUNTER, S&F heartbeat)
  • Positive control: Test verifies NODEINFO_APP still saves real values correctly
  • Clear assertions: Tests verify fields are undefined (not empty strings/zero) to ensure proper merge behavior

Test quality highlights:

  • expectNoNameClobber() helper function clearly validates the fix requirements (lines 768-777)
  • Tests cover both the fix and the unchanged behavior
  • Good use of mocking to isolate the specific behavior being tested

Architecture Alignment ✅

Follows repository guidelines:

  • Multi-source architecture: Maintains proper sourceId scoping
  • Database abstraction: Uses repository pattern correctly
  • Consistent with existing patterns: Follows the same upsert patterns used elsewhere in the codebase

Minor Suggestions

  1. Documentation: Consider adding a brief comment in the NODEINFO_APP handler explaining why it's the exception that provides real values, for symmetry with the other handlers.

  2. Code organization: The fix is well-implemented, but you might consider extracting the common lastHeard-only upsert pattern into a helper function to reduce duplication (though the current approach is also fine for clarity).

Verdict: APPROVED

This is a high-quality fix that:

  • Correctly solves a real user-facing problem
  • Uses minimal, surgical changes
  • Includes comprehensive regression tests
  • Maintains all existing functionality
  • Follows project conventions and patterns
  • Has no negative performance or security implications

The problem analysis was thorough, the solution is elegant, and the test coverage ensures this regression won't reoccur. Excellent work!


@Yeraze Yeraze merged commit 199beb2 into main Jun 14, 2026
19 checks passed
@Yeraze Yeraze deleted the fix/mqtt-nodeinfo-clobber branch June 14, 2026 16:11
@Yeraze Yeraze mentioned this pull request Jun 14, 2026
Yeraze added a commit that referenced this pull request Jun 14, 2026
Bump version to 4.10.3 across package.json, package-lock.json,
helm/meshmonitor/Chart.yaml, desktop/package.json, and
desktop/src-tauri/tauri.conf.json.

Finalize the CHANGELOG: promote [Unreleased] to [4.10.3] and add the
entries merged after the section was last written — MQTT NodeInfo
clobber fix (#3456), Traffic Management/StatusMessage firmware-version
gate (#3457), local-node module config storage (#3460), phantom channel
swap fix (#3453), and the TX power help-text clarification (#3459).

Add a release blog post highlighting the Traffic Management detection
fix and the bug-fix round.

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Yeraze added a commit that referenced this pull request Jun 17, 2026
…3512)

The INSERT … ON CONFLICT DO UPDATE race-fallback (upsertSet) still used the
bare ?? null pattern, letting ''/0 persist and reintroducing the #3456/#3505
clobber under a concurrent-insert race. Normalize blank strings / hwModel 0
to NULL there (no existing row is fetched on the insert branch, so the
nameOrExisting helper can't apply — blank→null is the consistent fix).

Also add async-path (PG/MySQL-compatible) guard tests to the shared
nodes.test.ts suite, per review feedback, mirroring the SQLite guard tests.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Yeraze added a commit that referenced this pull request Jun 17, 2026
…3505) (#3512)

* fix(nodes): don't clobber learned name/macaddr/hwModel with blanks (#3505)

The node upsert merge guard treated '' and 0 as "provided", so a future caller
passing a blank name or hwModel 0 would overwrite a stored value — the latent
hole behind #3456 (which was fixed only at the mqttIngestion call site, not the
guard). This is the durable form of that fix.

Per-column policy (not a blanket skip-empty, since 0 is legitimate for most
numerics):
- SQLite upsertNodeSqlite: longName/shortName/macaddr now use a setIfNonBlank
  guard ('' preserves existing); hwModel skips 0 (HardwareModel.UNSET).
- Async PG/MySQL update: same via a nameOrExisting() helper + hwModel-0 guard.
- snr/rssi/battery/voltage/lat-lon/role etc. still write 0 (genuine zeros).

Adds nodes.upsertGuard.test.ts: a blank re-upsert preserves name/macaddr/hwModel,
while snr:0/batteryLevel:0 still write. (New file rather than editing nodes.test.ts
to avoid colliding with the in-flight #3501 test-DB migration.)

Closes #3505.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(nodes): guard upsertSet race-fallback + cover async path (review #3512)

The INSERT … ON CONFLICT DO UPDATE race-fallback (upsertSet) still used the
bare ?? null pattern, letting ''/0 persist and reintroducing the #3456/#3505
clobber under a concurrent-insert race. Normalize blank strings / hwModel 0
to NULL there (no existing row is fetched on the insert branch, so the
nameOrExisting helper can't apply — blank→null is the consistent fix).

Also add async-path (PG/MySQL-compatible) guard tests to the shared
nodes.test.ts suite, per review feedback, mirroring the SQLite guard tests.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(nodes): use blankToNull for macaddr in upsertSet too (review #3512)

Consistency: macaddr in the ON CONFLICT DO UPDATE block still used ?? null
while longName/shortName already use blankToNull. Align it so a blank ''
macaddr normalizes to null on the conflict-update path like the other
identity fields.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant