Skip to content

fix: do not overwrite latest twin data from inventory.json values on mapper restart#3742

Merged
albinsuresh merged 1 commit intothin-edge:mainfrom
albinsuresh:fix/3734/twin-update-conflict
Jul 29, 2025
Merged

fix: do not overwrite latest twin data from inventory.json values on mapper restart#3742
albinsuresh merged 1 commit intothin-edge:mainfrom
albinsuresh:fix/3734/twin-update-conflict

Conversation

@albinsuresh
Copy link
Copy Markdown
Contributor

@albinsuresh albinsuresh commented Jul 24, 2025

Proposed changes

  • move inventory.json processing from tedge-mapper-c8y to tedge-agent
  • Publish the contents of inventory.json file to twin topics only after all the existing retained twin messages are processed by the agent, so that already published twin keys are skipped (to avoid overwriting them with old values in the file).

Types of changes

  • Bugfix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Improvement (general improvements like code refactoring that doesn't explicitly fix a bug or add any new functionality)
  • Documentation Update (if none of the other choices apply)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)

Paste Link to the issue

Checklist

  • I have read the CONTRIBUTING doc
  • I have signed the CLA (in all commits with git commit -s. You can activate automatic signing by running just prepare-dev once)
  • I ran just format as mentioned in CODING_GUIDELINES
  • I used just check as mentioned in CODING_GUIDELINES
  • I have added tests that prove my fix is effective or that my feature works
  • I have added necessary documentation (if appropriate)

Further comments

@codecov
Copy link
Copy Markdown

codecov bot commented Jul 24, 2025

Codecov Report

❌ Patch coverage is 81.32530% with 31 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
crates/core/tedge_agent/src/agent.rs 0.00% 13 Missing ⚠️
...ates/core/tedge_agent/src/entity_manager/server.rs 73.91% 9 Missing and 3 partials ⚠️
crates/extensions/c8y_mapper_ext/src/inventory.rs 40.00% 0 Missing and 3 partials ⚠️
...rates/core/tedge_agent/src/entity_manager/tests.rs 97.75% 2 Missing ⚠️
crates/extensions/c8y_mapper_ext/src/converter.rs 50.00% 0 Missing and 1 partial ⚠️

📢 Thoughts on this report? Let us know!

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Comment on lines -33 to -34
map.entry("name").or_insert(json!(self.device_name));
map.entry("type").or_insert(json!(self.device_type));
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The mapper publishing the device name and type from the tedge config as twin data on startup was already wrong, as this would have lead to one mapper overwriting the name overridden by another mapper in the config (since different cloud mappers can have their own device ids). The agent shouldn't be doing this either, as the impact would be the same and hence this logic was simply dropped and not ported to the agent either.

if self
.entity_store
.get_twin_fragment(&main_device, &key)
.is_none()
Copy link
Copy Markdown
Contributor Author

@albinsuresh albinsuresh Jul 24, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The side effect of this guard is that the user can not use the inventory.json file to update any twin values after the initial startup. They must use the twin/ topics for any updates. But, new values can be added to the file, which will be processed on subsequent restarts.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Jul 24, 2025

Robot Results

✅ Passed ❌ Failed ⏭️ Skipped Total Pass % ⏱️ Duration
667 0 3 667 100 1h50m35.90318s

@albinsuresh albinsuresh force-pushed the fix/3734/twin-update-conflict branch from 3c250aa to 0ff2887 Compare July 25, 2025 07:29
@albinsuresh albinsuresh temporarily deployed to Test Pull Request July 25, 2025 07:29 — with GitHub Actions Inactive
@albinsuresh albinsuresh marked this pull request as ready for review July 25, 2025 08:06
Copy link
Copy Markdown
Contributor

@didier-wenzek didier-wenzek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved (from a Rust perspective). I let @reubenmiller judge if the simplified behavior is okay as some device/twon metadata are no pushed to C8y on startup.

pub struct C8yMapperConfig {
pub device_id: String,
pub device_topic_id: EntityTopicId,
pub device_type: String,
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice to see this property gone. This has been a source of troubles from the start.

@albinsuresh albinsuresh force-pushed the fix/3734/twin-update-conflict branch from 0ff2887 to 95b791b Compare July 25, 2025 12:36
@albinsuresh albinsuresh temporarily deployed to Test Pull Request July 25, 2025 12:36 — with GitHub Actions Inactive
Copy link
Copy Markdown
Member

@rina23q rina23q left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The change looks very nice. Typo needs to be fixed before merge :)

Comment on lines +75 to +79
"c8y_Agent" : {
"name": "thin-edge.io",
"url": "https://thin-edge.io",
"version": version
}
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not asking to fix, but I found it interesting. In the c8y docs, c8y_Agent.version is marked mandatory. Though I know we can put anything via REST API or JSON over MQTT.

@albinsuresh albinsuresh changed the title fix: do not overwrite old inventory data on mapper restart fix: do not overwrite latest twin data from inventory.json values on mapper restart Jul 25, 2025
@albinsuresh albinsuresh force-pushed the fix/3734/twin-update-conflict branch from 95b791b to 799e662 Compare July 28, 2025 05:53
@albinsuresh albinsuresh temporarily deployed to Test Pull Request July 28, 2025 05:53 — with GitHub Actions Inactive
Copy link
Copy Markdown
Member

@rina23q rina23q left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Copy Markdown
Contributor

@reubenmiller reubenmiller left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since the reading of the inventory.json is now done by the tedge-agent service, we'll have to updated the docs to match as it used to guide users to restart the tedge-mapper-c8y service

To see the changes you need to restart the tedge-mapper.

It could also be worth mentioning that when removing values from the inventory.json file, the user will also have to clear older values in the entity store (like below), otherwise the values can be re-published again to the cloud when the tedge-mapper-c8y service is restarted.

tedge http delete /te/v1/entities/device/main///twin/type

@reubenmiller reubenmiller added the theme:telemetry Theme: Telemetry data label Jul 28, 2025
@albinsuresh albinsuresh force-pushed the fix/3734/twin-update-conflict branch from 799e662 to c3203d8 Compare July 29, 2025 12:14
@albinsuresh albinsuresh temporarily deployed to Test Pull Request July 29, 2025 12:14 — with GitHub Actions Inactive
@albinsuresh albinsuresh force-pushed the fix/3734/twin-update-conflict branch from c3203d8 to a8a48b2 Compare July 29, 2025 15:21
@albinsuresh albinsuresh temporarily deployed to Test Pull Request July 29, 2025 15:21 — with GitHub Actions Inactive
@albinsuresh albinsuresh added this pull request to the merge queue Jul 29, 2025
Merged via the queue into thin-edge:main with commit fbb9446 Jul 29, 2025
34 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

theme:telemetry Theme: Telemetry data

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants