Skip to content

#2147: implement dynamic wotValidationConfig endpoints#2179

Merged
thjaeckle merged 14 commits intoeclipse-ditto:masterfrom
beyonnex-io:add-wot-validation-config-APIs-2
Jun 20, 2025
Merged

#2147: implement dynamic wotValidationConfig endpoints#2179
thjaeckle merged 14 commits intoeclipse-ditto:masterfrom
beyonnex-io:add-wot-validation-config-APIs-2

Conversation

@hu-ahmed
Copy link
Contributor

@hu-ahmed hu-ahmed commented May 31, 2025

Resolves: #2147

Purpose
This PR implements dynamic WoT (Web of Things) validation configuration endpoints for Ditto, enabling runtime CRUD operations for validation settings via a new DevOps REST API. This allows for flexible, real-time updates to validation rules, especially useful during model migrations or deployment transitions, without requiring service restarts.

Key Features

  1. New DevOps REST Endpoints
    All endpoints are under /devops/wot/config:
    GET /devops/wot/config
    List all dynamic validation configs (by scope).
    GET /devops/wot/config/{scopeId}
    Retrieve a specific dynamic validation config section.
    PUT /devops/wot/config/{scopeId}
    Create or replace a dynamic validation config section.
    PATCH /devops/wot/config/{scopeId}
    Partially update a dynamic validation config section.
    DELETE /devops/wot/config/{scopeId}
    Delete a dynamic validation config section.
    GET /devops/wot/config/merged
    Retrieve the merged (static + dynamic) effective validation config.
  2. Distributed Persistence and Synchronization
    MongoDB is used for persistent storage of dynamic validation configs.
    Pekko Distributed Data (DData) is used to distribute dynamic config state across all Ditto instances in the cluster.
    When a config is updated via the API:
    The change is persisted to MongoDB.
    DData is updated, which automatically synchronizes the new state across all nodes.
    All Ditto instances immediately see the latest config, ensuring strong consistency and real-time updates.
  3. Startup and Merge Logic
    On startup, each instance loads the static config from things.conf.
    Dynamic configs are loaded from MongoDB and distributed via DData.
    The effective validation config is computed by merging static and dynamic configs, with dynamic overrides taking precedence for their scopes.

Benefits
Real-time, cluster-wide updates: Instantly change validation rules across all Ditto instances, with no restarts required.
Safe migrations: Temporarily relax or tighten validation for specific models or scopes.
Consistent distributed state: DData ensures all nodes have the latest config, with no manual cache management.
Separation of static and dynamic: Static config remains as a fallback; dynamic config is layered on top.

@hu-ahmed hu-ahmed force-pushed the add-wot-validation-config-APIs-2 branch 4 times, most recently from 88a54d1 to c3c30b5 Compare June 2, 2025 12:13
Copy link
Member

@thjaeckle thjaeckle left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks a lot for this (again) huge PR @hu-ahmed :) 👍

I left some first remarks inline .. which I spotted right away.
But I probably need some additional views/rounds ..

@thjaeckle thjaeckle added this to the 3.8.0 milestone Jun 4, 2025
@hu-ahmed hu-ahmed force-pushed the add-wot-validation-config-APIs-2 branch 4 times, most recently from 7865d63 to 18a09dd Compare June 7, 2025 14:00
@hu-ahmed hu-ahmed force-pushed the add-wot-validation-config-APIs-2 branch from 18a09dd to f8da4b5 Compare June 10, 2025 16:26
@hu-ahmed hu-ahmed force-pushed the add-wot-validation-config-APIs-2 branch from 8c37fe6 to 89f26d2 Compare June 11, 2025 12:35
hu-ahmed and others added 2 commits June 11, 2025 15:04
* javadocs, @nullable use, appendPayload use of predicate, hashCode/equals/toString, formatting
@thjaeckle thjaeckle moved this to In Progress in Ditto Planning Jun 13, 2025
@thjaeckle
Copy link
Member

@hu-ahmed thanks for providing the documentation, looks good.

Just 2 things:

  • Maybe we can move the technical details (DData, persistence) in a section "Technical details" or so
  • It would be nice to describe a use case for when this feature is useful - so for migrating things to new WoT model versions while still accepting "invalid" (according to the new model) updates to the things temporarily

@thjaeckle thjaeckle moved this from In Progress to Waiting for Approval in Ditto Planning Jun 20, 2025
Copy link
Member

@thjaeckle thjaeckle left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@hu-ahmed thanks a lot for this great addition

I left some minor review commits, cleaning up the codebase a little to be more "Ditto like".
Functionality-wise it works well 👍

System tests also passed - so this is good to be merged.

@thjaeckle thjaeckle merged commit f22370f into eclipse-ditto:master Jun 20, 2025
3 checks passed
@github-project-automation github-project-automation bot moved this from Waiting for Approval to Done in Ditto Planning Jun 20, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

Add DevOps REST API for dynamic WoT validation configuration

2 participants