fix(mqtt): store ingested publicKey as base64 (not hex), backfill via migration 069 by Yeraze · Pull Request #3133 · Yeraze/meshmonitor

Yeraze · 2026-05-21T21:48:41Z

Summary

Fixes a long-standing false-positive in the key-mismatch detector caused by an encoding disagreement between the MQTT ingest path and every other publicKey write site.

The bug

Path	File:line	Encoding
Direct serial/TCP NodeInfo	`meshtasticManager.ts:5546`	base64
Device security-config handshake	`meshtasticManager.ts:3594`	base64
MQTT NodeInfo ingest	`mqttIngestion.ts:224`	hex ← inconsistent

When a node arrived via MQTT first, its key was stored hex-encoded. When the same node later sent NodeInfo over the direct radio link, the comparison at `meshtasticManager.ts:5572`

```ts
if (existingNode && existingNode.publicKey && nodeData.publicKey && existingNode.publicKey !== nodeData.publicKey)
```

compared `"1ec4c0f23909..."` (hex) against `"HsTA8jkJ..."` (base64) — never equal, even though the 32 underlying bytes were identical. The user-visible symptom: repeated `🔐 Key mismatch detected` log warnings for nodes whose keys had never actually changed. The upsert path then "self-healed" by overwriting hex with base64, but the next MQTT-side update flipped it back, and the cycle repeated.

Diagnosed by decoding the live log fragments side-by-side:
```
stored=1ec4c0f2... → hex → 0x1e 0xc4 0xc0 0xf2 ...
mesh=HsTA8jkJ... → base64 → 0x1e 0xc4 0xc0 0xf2 0x39 0x09 ...
```
Identical bytes.

Fix

`src/server/mqttIngestion.ts` now encodes `user.publicKey` (and the snake-case fallback) as base64, matching all other paths.

Cleanup migration

`src/server/migrations/069_normalize_node_public_keys_to_base64.ts` scans `nodes.publicKey` for lowercase 32-byte hex (`^[0-9a-f]{64}$`) and converts each to its base64 equivalent. Idempotent across SQLite / PostgreSQL / MySQL — base64 output is 44 chars and contains uppercase / `+` / `/` / `=`, so a second run finds nothing to match.

After this lands and the migration runs, any node whose key was stored hex by historical MQTT ingest will be converted in-place. No more spurious mismatch warnings; the key-mismatch detector only fires on actual key changes.

Test plan

`npm test` — 10,287 vitest tests pass
`npx tsc --noEmit` clean
New `069_normalize_node_public_keys_to_base64.test.ts` covers: hex → base64 conversion, base64 left alone, idempotent on second run, NULL-skip, 64-char-but-non-hex-skip, mixed-row scenarios
Migration test count bumped to 69 with last-migration assertion updated
After deploy: tail the dev container for `🔐 Key mismatch detected` warnings — expect zero for the previously-flapping nodes (Yeraze StationG2, KE4JQ G2 Base, LANTANA G2, Base 01, Julie Portable)

🤖 Generated with Claude Code

The MQTT NodeInfo ingest path at src/server/mqttIngestion.ts:224 was writing nodes.publicKey as hex (bytesToHex(user.publicKey)) while every other write path stored base64 — direct serial/TCP NodeInfo (meshtasticManager.ts:5546) and the device's security-config handshake (meshtasticManager.ts:3594) both use Buffer.toString('base64'). Net effect: any node first seen via MQTT was stored hex-encoded; when that same node later sent NodeInfo over the direct radio link, the key-mismatch detector at meshtasticManager.ts:5572 (string-equality compare on existingNode.publicKey vs nodeData.publicKey) fired as a false positive on every packet — the underlying 32 bytes were identical, only the string encoding differed (e.g. stored "1ec4c0f23909..." vs mesh "HsTA8jkJ..."). The "self-heal" upsert path then overwrote hex with base64, so subsequent packets came back as match — until the next user-initiated MQTT ingest flipped it back. Fix: encode the bytes as base64 in the MQTT path, matching every other write site. Migration 069 backfills existing hex-encoded rows by scanning `nodes.publicKey` for `^[0-9a-f]{64}$` (lowercase 32-byte hex) and converting each to its base64 equivalent in JS. Idempotent — base64 output is 44 chars and includes uppercase / `+/=`, so a second run finds nothing to convert. Implemented for SQLite / PostgreSQL / MySQL. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

github-actions · 2026-05-21T21:48:54Z

Claude finished @Yeraze's task —— View job

PR Review: MQTT PublicKey Encoding Fix ✅

Todo List

Review migration 069 implementation and safety
Examine MQTT ingestion fix
Assess test coverage
Check for security and performance concerns
Provide comprehensive feedback

Summary

This is a high-quality fix for a subtle but impactful encoding inconsistency bug. The implementation is production-ready with excellent attention to detail.

✅ Code Quality & Best Practices

Migration 069:

Excellent idempotency design using regex predicate /^[0-9a-f]{64}$/
Proper cross-database support with appropriate SQL dialects
Safe error handling and connection management (especially MySQL)
Good logging with conversion counts for observability

MQTT Ingestion Fix:

Root cause properly addressed - changed from implicit hex to explicit base64
Comprehensive comments explaining the consistency requirement
Fallback handling for both publicKey and public_key variants

✅ No Security Concerns

Migration uses parameterized queries (PostgreSQL/MySQL) preventing SQL injection
Base64 encoding is standard and safe for publicKey storage
No credential exposure or cryptographic weaknesses introduced

✅ Performance Considerations

One-time migration cost - acceptable for data cleanup
Efficient queries using length + regex predicates to minimize processing
In-memory conversion avoids expensive database function calls
Proper connection pooling in MySQL implementation

⚠️ Minor Test Gap (Non-blocking)

While migration 069 has comprehensive test coverage, the MQTT ingestion path lacks a specific test for publicKey encoding. Consider adding:

it('stores publicKey as base64 from MQTT NodeInfo', async () => {
  // Test that MQTT-ingested publicKey values are base64-encoded
});

✅ Architecture Alignment

Follows codebase patterns - consistent with existing Buffer.from().toString('base64') usage
Proper migration numbering and registry integration
Cross-database compatibility maintained

Impact Assessment

Before: False-positive "🔐 Key mismatch detected" warnings for nodes first seen via MQTT
After: Clean key comparison logic, spurious warnings eliminated

This fix resolves a user-visible reliability issue and should significantly improve the operator experience.

Recommendation: ✅ APPROVE

The fix is technically sound, well-tested, and addresses the root cause effectively. The migration is safe for production deployment.

Job run

…ey after purge (#3140) When key-repair detects a mismatch and (via processKeyRepairs or the immediate-purge path) calls sendRemoveNode, the firmware purges the destination's entry — including its public key — from its NodeDB. Our local nodes.publicKey column still holds the (now-stale) key, so sendTextMessage was still setting pkiEncrypted=true. The firmware, with no key to use, silently drops the outbound DM. Net effect: DMs to a node with an active keyMismatchDetected appear to "send" from MeshMonitor's side but never reach the destination. This was masked by — and the symptom surfaced through — #3133 (MQTT publicKey hex vs base64). MQTT-first-seen nodes had their key written as hex; every subsequent direct-radio NodeInfo string-compared unequal against the base64 form, triggering a false mismatch, triggering the auto-purge, leaving the firmware keyless for that destination. Once the hex/base64 normalization landed (2430152), the false positives stopped and DMs flowed again. The underlying "PKI=true with no key → silent drop" problem remains for genuine key changes, hence this guard. The fix consults targetNode.keyMismatchDetected before requesting PKI. When the flag is set, we skip pkiEncrypted=true and fall back to channel encryption — matching what the post-purge node-info-exchange code already does (lines 2174-2175, 5604-5605). Regression test: src/server/meshtasticManager.test.ts adds a "DM PKI guard" describe block covering the five publicKey/keyMismatchDetected combinations, mirroring the production decision logic so future refactors of that branch fail loudly. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Yeraze merged commit 2430152 into main May 21, 2026
19 checks passed

Yeraze mentioned this pull request May 22, 2026

fix(dm): skip PKI flag when keyMismatchDetected to prevent silent drop after firmware purge #3140

Merged

4 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

fix(mqtt): store ingested publicKey as base64 (not hex), backfill via migration 069#3133

fix(mqtt): store ingested publicKey as base64 (not hex), backfill via migration 069#3133
Yeraze merged 1 commit into
mainfrom
fix/mqtt-publickey-encoding

Yeraze commented May 21, 2026

Uh oh!

github-actions Bot commented May 21, 2026 •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Conversation

Yeraze commented May 21, 2026

Summary

The bug

Fix

Cleanup migration

Test plan

Uh oh!

github-actions Bot commented May 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR Review: MQTT PublicKey Encoding Fix ✅

Todo List

Summary

✅ Code Quality & Best Practices

✅ No Security Concerns

✅ Performance Considerations

⚠️ Minor Test Gap (Non-blocking)

✅ Architecture Alignment

Impact Assessment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

github-actions Bot commented May 21, 2026 •

edited

Loading