Skip to content

fix(messages): exclude traceroute rows from UI message window (#2741)#2745

Merged
Yeraze merged 2 commits into
mainfrom
fix/traceroute-evicts-messages
Apr 20, 2026
Merged

fix(messages): exclude traceroute rows from UI message window (#2741)#2745
Yeraze merged 2 commits into
mainfrom
fix/traceroute-evicts-messages

Conversation

@Yeraze

@Yeraze Yeraze commented Apr 20, 2026

Copy link
Copy Markdown
Owner

Summary

  • Fixes [BUG] Successfull traceroute with node makes last message in chat with that node dissapeared #2741 — a successful traceroute made the last DM with that node disappear on reload.
  • Root cause: traceroute responses are saved to the messages table (portnum=70), but the UI's DM view filters them out (portnum === 1). Every UI feed is capped at 100 rows, so each traceroute displaced one real DM out of the window. Repeating drained all DMs.
  • Fix: add an optional excludePortnums arg to MessagesRepository.getMessages and pass [TRACEROUTE_APP] from every UI-facing caller (poll, /api/sources/:id/messages, getRecentMessages, unified legacy fetch). NULL-portnum rows are preserved for legacy data. Public v1 API, cache warming, and export paths are unchanged.

Changed callers

  • src/server/server.ts/api/poll message window
  • src/server/meshtasticManager.tsgetRecentMessages (feeds /api/messages)
  • src/server/routes/sourceRoutes.ts/api/sources/:id/messages
  • src/server/routes/unifiedRoutes.ts — legacy no-channel unified fetch

Test plan

  • New regression test file messages.exclude-portnums.test.ts — 4 tests including the explicit [BUG] Successfull traceroute with node makes last message in chat with that node dissapeared #2741 scenario (traceroute arriving causes oldest DM to fall off the capped window unless filter is applied).
  • Full messages-repo suite green (35 tests).
  • Server/route suites unaffected (131 tests green: sourceRoutes, messageRoutes, meshtasticManager).
  • Manually verify in dev container: send DMs to a node, run a successful traceroute, reload — DMs should remain.

🤖 Generated with Claude Code

Traceroute responses were saved to the messages table with portnum=70 and
counted toward the 100-row cap that UI endpoints fetch, but the client
always filters them out of chat lists (MessagesTab displays portnum=1).
A successful traceroute therefore displaced a real DM from the capped
window, making it vanish on reload.

Adds an optional excludePortnums arg to MessagesRepository.getMessages
and passes [TRACEROUTE_APP] from every UI-facing caller (poll, source
messages, recent messages, unified legacy fetch). NULL portnum rows are
retained to preserve legacy data.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Apr 20, 2026

Copy link
Copy Markdown

Claude finished @Yeraze's task —— View job


Code Review Complete ✅

I've thoroughly reviewed this pull request implementing the traceroute message exclusion fix. Here's my comprehensive assessment:

🎯 Summary

This is an excellent solution to issue #2741. The fix correctly prevents traceroute responses from displacing DMs in the UI by adding an optional excludePortnums parameter to filter them out at the database layer.

Strengths

1. Surgical and Targeted Fix

  • Minimal, focused change that addresses the root cause
  • Only affects UI-facing data paths, preserving public APIs and export functionality
  • Clean parameter addition without breaking existing functionality

2. Robust Implementation

  • Excellent NULL handling: or(isNull(messages.portnum), notInArray(messages.portnum, excludePortnums)) correctly preserves legacy data where portnum is NULL
  • Consistent application: All 4 UI-facing callers correctly pass [PortNum.TRACEROUTE_APP]
  • Uses shared constants: Leverages PortNum.TRACEROUTE_APP from src/server/constants/meshtastic.ts instead of magic numbers

3. Comprehensive Test Coverage

🔧 Code Quality Assessment

Repository Guidelines Compliance: ✅

  • Follows multi-database async patterns correctly
  • Uses Drizzle ORM query builders (no raw SQL)
  • Maintains consistent Async naming convention
  • Preserves database-agnostic design

Implementation Quality: ✅

  • Clean parameter signature: excludePortnums?: number[]
  • Proper conditional logic with length checks
  • Maintains existing ordering and pagination behavior
  • Consistent documentation across all modified callers

🔒 Security & Performance

Security: ✅ No concerns

  • No user input validation needed (internal system filtering)
  • No SQL injection risk (uses parameterized Drizzle queries)
  • No privilege escalation concerns

Performance: ✅ Optimized approach

  • Efficient database filtering: Exclusion happens at DB level, not in application code
  • Query optimization: notInArray with OR condition is index-friendly
  • No additional queries: Single query per caller, same as before
  • Minimal overhead: Empty array check prevents unnecessary SQL conditions

🎯 Changed Files Review

Core Implementation (messages.ts:101-113):

const whereClause = (excludePortnums && excludePortnums.length > 0)
  ? and(
      this.withSourceScope(messages, sourceId),
      or(isNull(messages.portnum), notInArray(messages.portnum, excludePortnums)),
    )
  : this.withSourceScope(messages, sourceId);
  • Perfect implementation with proper NULL handling
  • Efficient early-exit for empty arrays

Caller Updates: All properly implemented

  • server.ts:poll - ✅ Filters UI message window
  • meshtasticManager.ts:getRecentMessages - ✅ Filters recent messages API
  • sourceRoutes.ts:messages - ✅ Filters source-specific messages
  • unifiedRoutes.ts:messages - ✅ Filters unified legacy API

🧪 Test Coverage Analysis

The test suite is exceptionally thorough:

  1. Basic functionality: Verifies traceroute exclusion works
  2. Legacy compatibility: Confirms NULL portnum rows are preserved
  3. Bug scenario reproduction: Direct test of the [BUG] Successfull traceroute with node makes last message in chat with that node dissapeared #2741 issue
  4. Edge cases: Empty arrays and omitted parameters

Each test has clear, descriptive names and excellent documentation.

📋 Minor Observations

Consistency Note: The async version doesn't have a corresponding sync getMessagesSqlite update with the same parameter. However, this appears intentional since:

  • UI callers use async methods
  • Sync methods are primarily for legacy/internal use
  • The fix targets UI data paths specifically

🎯 Verdict

Excellent implementation ⭐⭐⭐⭐⭐

This PR demonstrates:

  • ✅ Clear understanding of the problem
  • ✅ Minimal, surgical fix approach
  • ✅ Comprehensive test coverage
  • ✅ Proper database patterns
  • ✅ Excellent code documentation
  • ✅ No security or performance concerns

Recommendation: Approve and merge. This is a well-crafted fix that solves the issue cleanly without introducing technical debt or breaking changes.


…s arg

The traceroute-exclusion fix added a 4th arg to getMessages calls in
unifiedRoutes, breaking this strict-arg assertion. Update the expected
call signature to include [70] (PortNum.TRACEROUTE_APP).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Apr 20, 2026

Copy link
Copy Markdown

Claude finished @Yeraze's task —— View job


PR Review: Traceroute Message Exclusion Fix

Reviewing PR #2745 that fixes issue #2741 where traceroute responses were displacing DMs in the UI message window.

Todo List:

  • Read and analyze all changed files
  • Review implementation against repository architecture guidelines
  • Evaluate code quality and best practices
  • Check for potential bugs and security issues
  • Assess performance considerations
  • Review test coverage adequacy
  • Provide comprehensive feedback

🎯 Summary

This is an excellent solution to issue #2741. The fix correctly prevents traceroute responses from displacing DMs in the UI by adding an optional excludePortnums parameter to filter them out at the database layer.

Strengths

1. Surgical and Targeted Fix

  • Minimal, focused change that addresses the root cause
  • Only affects UI-facing data paths, preserving public APIs and export functionality
  • Clean parameter addition without breaking existing functionality

2. Robust Implementation

  • Excellent NULL handling: or(isNull(messages.portnum), notInArray(messages.portnum, excludePortnums)) correctly preserves legacy data where portnum is NULL
  • Consistent application: All 4 UI-facing callers correctly pass [PortNum.TRACEROUTE_APP]
  • Uses shared constants: Leverages PortNum.TRACEROUTE_APP from src/server/constants/meshtastic.ts instead of magic numbers

3. Comprehensive Test Coverage

🔧 Code Quality Assessment

Repository Guidelines Compliance: ✅

  • Follows multi-database async patterns correctly
  • Uses Drizzle ORM query builders (no raw SQL)
  • Maintains consistent Async naming convention
  • Preserves database-agnostic design

Implementation Quality: ✅

  • Clean parameter signature: excludePortnums?: number[]
  • Proper conditional logic with length checks
  • Maintains existing ordering and pagination behavior
  • Consistent documentation across all modified callers

🔒 Security & Performance

Security: ✅ No concerns

  • No user input validation needed (internal system filtering)
  • No SQL injection risk (uses parameterized Drizzle queries)
  • No privilege escalation concerns

Performance: ✅ Optimized approach

  • Efficient database filtering: Exclusion happens at DB level, not in application code
  • Query optimization: notInArray with OR condition is index-friendly
  • No additional queries: Single query per caller, same as before
  • Minimal overhead: Empty array check prevents unnecessary SQL conditions

🎯 Changed Files Review

Core Implementation (messages.ts:101-113):

const whereClause = (excludePortnums && excludePortnums.length > 0)
  ? and(
      this.withSourceScope(messages, sourceId),
      or(isNull(messages.portnum), notInArray(messages.portnum, excludePortnums)),
    )
  : this.withSourceScope(messages, sourceId);
  • Perfect implementation with proper NULL handling
  • Efficient early-exit for empty arrays

Caller Updates: All properly implemented

  • server.ts:poll - ✅ Filters UI message window
  • meshtasticManager.ts:getRecentMessages - ✅ Filters recent messages API
  • sourceRoutes.ts:messages - ✅ Filters source-specific messages
  • unifiedRoutes.ts:messages - ✅ Filters unified legacy API

🧪 Test Coverage Analysis

The test suite is exceptionally thorough:

  1. Basic functionality: Verifies traceroute exclusion works
  2. Legacy compatibility: Confirms NULL portnum rows are preserved
  3. Bug scenario reproduction: Direct test of the [BUG] Successfull traceroute with node makes last message in chat with that node dissapeared #2741 issue
  4. Edge cases: Empty arrays and omitted parameters

Each test has clear, descriptive names and excellent documentation.

📋 Minor Observations

Consistency Note: The async version doesn't have a corresponding sync getMessagesSqlite update with the same parameter. However, this appears intentional since:

  • UI callers use async methods
  • Sync methods are primarily for legacy/internal use
  • The fix targets UI data paths specifically

🎯 Verdict

Excellent implementation ⭐⭐⭐⭐⭐

This PR demonstrates:

  • ✅ Clear understanding of the problem
  • ✅ Minimal, surgical fix approach
  • ✅ Comprehensive test coverage
  • ✅ Proper database patterns
  • ✅ Excellent code documentation
  • ✅ No security or performance concerns

Recommendation: Approve and merge. This is a well-crafted fix that solves the issue cleanly without introducing technical debt or breaking changes.


@github-actions

Copy link
Copy Markdown
System Test Results

MeshMonitor System Test Results

Test Run: 2026-04-20 14:22:58 EDT

Test Summary

Test Suite Result
Configuration Import ✅ PASSED
Quick Start Test ✅ PASSED
Security Test ✅ PASSED
V1 API Test ✅ PASSED
Reverse Proxy Test ✅ PASSED
Reverse Proxy + OIDC ✅ PASSED
Virtual Node CLI Test ✅ PASSED
Backup & Restore Test ✅ PASSED
Database Migration Test ✅ PASSED
DB Backing Consistency ❌ FAILED
API Exercise (3 DBs) ❌ FAILED

❌ Overall Result: FAILED

Some tests failed. Please review the failures above and fix before creating/updating PR.

Failed Tests

  • DB Backing Consistency: Database backing consistency test failed

@github-actions

Copy link
Copy Markdown
System Test Results

MeshMonitor System Test Results

Test Run: 2026-04-20 14:41:18 EDT

Test Summary

Test Suite Result
Configuration Import ✅ PASSED
Quick Start Test ✅ PASSED
Security Test ✅ PASSED
V1 API Test ✅ PASSED
Reverse Proxy Test ✅ PASSED
Reverse Proxy + OIDC ✅ PASSED
Virtual Node CLI Test ✅ PASSED
Backup & Restore Test ✅ PASSED
Database Migration Test ✅ PASSED
DB Backing Consistency ✅ PASSED
API Exercise (3 DBs) ✅ PASSED

✅ Overall Result: PASSED

All deployment configurations are working correctly!

Test Details

Configuration Import:

  • Tests configuration import and device reboot cycle
  • Verifies channel roles, PSKs, and LoRa configuration
  • Note: Channel name verification skipped due to architectural limitation

Quick Start Test:

  • Zero-config deployment (no SESSION_SECRET or COOKIE_SECURE required)
  • HTTP access without HSTS
  • Auto-generated admin user with default credentials
  • Session cookies work over HTTP
  • Meshtastic node connection and message exchange verified

Security Test:

  • Verifies Node IP address hidden from anonymous users in API responses
  • Verifies MQTT configuration hidden from anonymous users
  • Verifies Node IP address visible to authenticated users
  • Verifies MQTT configuration visible to authenticated users
  • Verifies protected endpoints require authentication

V1 API Test:

  • Tests v1 REST API endpoints with Bearer token authentication
  • Verifies Bearer token requests bypass CSRF protection
  • Verifies POST/PUT/DELETE work without CSRF token when using Bearer auth
  • Verifies session-based requests still require CSRF token

Reverse Proxy Test:

  • Production deployment with COOKIE_SECURE=true
  • HTTPS-ready configuration
  • Trust proxy enabled for reverse proxy compatibility
  • CORS configured for HTTPS domain
  • Meshtastic node connection and message exchange verified

Reverse Proxy + OIDC Test:

  • OIDC authentication integration
  • Mock OIDC provider health checks
  • Authorization flow and session creation
  • Hybrid mode (OIDC + local auth)
  • Meshtastic node connection verified

Virtual Node CLI Test:

  • Virtual Node Server enabled on TCP port 4404
  • Meshtastic Python client successfully connects
  • Node data download and synchronization verified
  • Test message sent on gauntlet channel (index 3)
  • Message delivery confirmed via Web UI API
  • Virtual Node Server connection logging verified

Backup & Restore Test:

  • System backup created from running dev container
  • New container spun up with RESTORE_FROM_BACKUP env var
  • Data integrity verified (node count, message count, settings)
  • Restore event logged in audit log
  • Dev container unaffected by restore test

Database Migration Test:

  • SQLite to PostgreSQL migration verified
  • SQLite to MySQL migration verified
  • Data integrity confirmed for both target databases
  • Row counts match between source and target

DB Backing Consistency Test:

  • SQLite, PostgreSQL, and MySQL backends tested with same device
  • Node counts within ±10 across all three backends
  • Favorite counts identical across all backends
  • Key station verified as favorite on all backends

@Yeraze Yeraze merged commit d488a6e into main Apr 20, 2026
20 of 21 checks passed
@Yeraze Yeraze deleted the fix/traceroute-evicts-messages branch April 20, 2026 19:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] Successfull traceroute with node makes last message in chat with that node dissapeared

1 participant