Skip to content

Persistent nodedb#780

Merged
danditomaso merged 22 commits into
meshtastic:mainfrom
philon-:persistent-nodedb
Aug 24, 2025
Merged

Persistent nodedb#780
danditomaso merged 22 commits into
meshtastic:mainfrom
philon-:persistent-nodedb

Conversation

@philon-

@philon- philon- commented Aug 12, 2025

Copy link
Copy Markdown
Contributor

Description

This is a PR that adds persistence of node database. This is the first step towards safe persistence of messages, telemetry and so on.

More testing is needed before merging

Future work:

  • Update PKI regen dialog to notify user when nodes have mismatching keys, currently nodeError is set but no dialog to prompt the user.
  • CommandPalette option to disconnect & wipe storage

Related Issues

Changes Made

Major:

  • Refactored deviceStore by breaking the monolith apart and bringing all nodeDB methods into its own nodeDBStore
  • Adjusted the DeviceContext so that instead of passing the Device object along, we only pass the nodenum and get the correct Device and NodeDB objects from their respective hooks useDevice and useNodeDB
  • Rewrote the JSON serializer previously used for the messageStore to allow more special handlers, since more datatypes need specific (de)serialization.
  • Defined NODEDB_RETENTION_NUM (currently 10) which defined how many separate devices' nodeDBs should be saved before cycling them out.
  • Tests for nodeDB store, indexDB handler

Minor:

  • Updated useFavoriteNode and useIgnoreNode to use the separate device and nodedb stores.
  • Updated existing tests
  • All nodes except self are removed when resetting nodedb, all nodes are removed when performing factory reset.

Testing Done

Screenshots (if applicable)

Checklist

  • Code follows project style guidelines
  • Documentation has been updated or added
  • Tests have been added or updated
  • All i18n translation labels have been added (read
    CONTRIBUTING_I18N_DEVELOPER_GUIDE.md for more details)

Copilot AI review requested due to automatic review settings August 12, 2025 21:01
@vercel

vercel Bot commented Aug 12, 2025

Copy link
Copy Markdown

@philon- is attempting to deploy a commit to the Meshtastic Team on Vercel.

A member of the Team first needs to authorize it.

@philon- philon- marked this pull request as draft August 12, 2025 21:01

This comment was marked as outdated.

@philon- philon- requested a review from Copilot August 15, 2025 18:07

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR introduces persistent storage for the node database, separating it from the device store as a preparatory step for comprehensive persistence of messages and telemetry. The refactoring creates a dedicated NodeDB store with proper IndexedDB persistence while maintaining a clean separation of concerns between device management and node data.

Key Changes

  • Refactored device store: Extracted all node-related functionality into a separate, persistent NodeDB store
  • Enhanced persistence layer: Rewrote JSON serialization to support Maps, Uint8Arrays, and other complex data types with extensible handlers
  • Improved device context: Replaced direct Device object passing with device ID-based context for better data flow management

Reviewed Changes

Copilot reviewed 42 out of 42 changed files in this pull request and generated 5 comments.

Show a summary per file
File Description
packages/web/src/core/stores/nodeDBStore/ New persistent NodeDB store with comprehensive node management and error handling
packages/web/src/core/stores/utils/indexDB.ts Enhanced serialization framework supporting multiple data types with extensible handlers
packages/web/src/core/stores/utils/useDeviceContext.ts New context system for device ID-based data access
packages/web/src/core/stores/deviceStore/index.ts Removed node-related methods, focused on device-specific functionality
packages/web/src/core/subscriptions.ts Updated to work with separate NodeDB store and enhanced PKI validation
Multiple component files Updated imports and method calls to use new NodeDB store
Comments suppressed due to low confidence (1)

Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

Comment thread packages/web/src/core/subscriptions.ts
Comment thread packages/web/src/core/stores/nodeDBStore/index.ts Outdated
Comment thread packages/web/src/core/stores/utils/indexDB.ts
Comment thread packages/web/src/core/subscriptions.ts
Comment thread packages/web/src/core/stores/nodeDBStore/index.ts
@philon- philon- requested a review from danditomaso August 15, 2025 22:09
@philon- philon- marked this pull request as ready for review August 18, 2025 11:49
@vercel

vercel Bot commented Aug 19, 2025

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Preview Comments Updated (UTC)
web-test Ready Ready Preview Comment Aug 23, 2025 2:11am

@jkrauska

Copy link
Copy Markdown

really like this idea -- will help a lot with initial connection

@danditomaso danditomaso left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good! LGTM

@danditomaso

Copy link
Copy Markdown
Collaborator

We can merge this now, but it looks like two tests are failing. Its up to you if you want to fix these tests first then merge or merge and fix them on main.

@philon-

philon- commented Aug 23, 2025

Copy link
Copy Markdown
Contributor Author

We can merge this now, but it looks like two tests are failing. Its up to you if you want to fix these tests first then merge or merge and fix them on main.

Of course... forgot to mock the featureFlag in tests

@danditomaso danditomaso merged commit 68ec7ee into meshtastic:main Aug 24, 2025
2 of 3 checks passed
@philon- philon- deleted the persistent-nodedb branch August 24, 2025 20:11
@shadowkora shadowkora mentioned this pull request Aug 25, 2025
4 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants