feat: c8y-mapper sends all supported operations on-demand via signal channel by rina23q · Pull Request #3761 · thin-edge/thin-edge.io

rina23q · 2025-08-15T17:32:27Z

Proposed changes

This PR introduces signal, a new MQTT API channel for stateless requests.
This channel allows users to instruct tedge-mapper-c8y to republish all supported operations to the c8y tenant on-demand, without needing to restart any services.

Example usage:
Request:

tedge mqtt pub te/device/main/service/tedge-mapper-c8y/signal/sync '{}'

Response:

[c8y/s/us] 114,c8y_Command,c8y_DeviceProfile,c8y_DownloadConfigFile,c8y_LogfileRequest,c8y_RemoteAccessConnect,c8y_Restart,c8y_SoftwareUpdate,c8y_UploadConfigFile
[c8y/s/us/rina0010:device:child01] 114,c8y_LogfileRequest,c8y_Restart
[c8y/s/us/rina0010:device:child02] 114,c8y_Restart

Tasks:

Supported operations for child devices can be loaded from the operation directory (/etc/tedge/c8y/) and converted to 114.
Re-publishing 114 for main and child devices, which is triggered by the MQTT API new endpoint signal (signal).
Adding docs accordingly.
Adding a test.

Future plan

Expand the support for other signal types (sync_config, sync_log)

Types of changes

Bugfix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Improvement (general improvements like code refactoring that doesn't explicitly fix a bug or add any new functionality)
Documentation Update (if none of the other choices apply)
Breaking change (fix or feature that would cause existing functionality to not work as expected)

Paste Link to the issue

#3738

Checklist

I have read the CONTRIBUTING doc
I have signed the CLA (in all commits with git commit -s. You can activate automatic signing by running just prepare-dev once)
I ran just format as mentioned in CODING_GUIDELINES
I used just check as mentioned in CODING_GUIDELINES
I have added tests that prove my fix is effective or that my feature works
I have added necessary documentation (if appropriate)

Further comments

codecov · 2025-08-15T17:46:49Z

Codecov Report

❌ Patch coverage is 83.11688% with 26 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
crates/core/tedge_api/src/mqtt_topics.rs	51.21%	20 Missing ⚠️
crates/extensions/c8y_mapper_ext/src/config.rs	0.00%	2 Missing ⚠️
crates/extensions/c8y_mapper_ext/src/converter.rs	85.71%	0 Missing and 2 partials ⚠️
crates/extensions/c8y_mapper_ext/src/signals.rs	89.47%	2 Missing ⚠️

📢 Thoughts on this report? Let us know!

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

reubenmiller · 2025-08-18T07:21:07Z

I know this is still in draft, but the tedge-mapper c8y --no-cache might be difficult given that users generally don't start the mapper themselves, it is typically done via tedge reconnect c8y (or the service just starts). And given that generally you'd just want the mapper to ignore (delete?) the cache for once, it makes it impractical for users to coordinate.

docs/src/references/mqtt-api.md

albinsuresh · 2025-08-28T06:46:00Z

docs/src/references/mqtt-api.md

+| &lt;signal_type&gt; | Signal type. Each signal can define its own payload schema to allow signals to have parameters related to the signal's function. |
+| &lt;action&gt;      | Action for the signal type. e.g. `check`                                                                                         |


I'm wondering if two levels: signal_type and action are necessary or one would have been sufficient. From the examples listed below it makes sense. But I also feel that semantically config_type/check could have been modelled as config_check as well. But yeah, no harm keeping 2 levels and it might even open up further extension possibilities in future.

In a sense, the action is similar to a command's "id", e.g.

Type Example Topic

Command /cmd/firmware_update/c8y-1234

Signal /sig/config_type/check

Updated the design in 8f28b06. Now only one level after signal.

docs/src/references/mqtt-api.md

crates/core/tedge_api/src/mqtt_topics.rs

didier-wenzek · 2025-08-28T13:45:10Z

crates/extensions/c8y_mapper_ext/src/converter.rs

                Ok(vec![])
            }

+            Channel::Signal { signal_type } => self.process_signal_message(&source, signal_type),


Is the mapper the correct place to implement this?

When playing with the feature, I was expecting to see all the capabilities published on the te/# topics.

Then I noticed that sig/operations/check was sent to device/main/service/tedge-mapper-c8y and that only c8y/# capabilities were published. Okay, that can make senses. However, I would expect capabilities to be also published when the signal is sent to device/main// or device/main/service/tedge-agent.

Yes, I think publishing the capabilities locally makes sense, otherwise it will be very difficult for other clients to know what signals are supported by variables devices/services...though such capabilities don't necessarily have to be published to the cloud.

To focus on the initial scope (#3738), I keep the implementation only in tedge-mapper-c8y level. If we have concrete usecases that requires the signal channel support in tedge-agent, I'm happy to extend it.

docs/src/references/mqtt-api.md

github-actions · 2025-09-03T11:52:11Z

Robot Results

✅ Passed	❌ Failed	⏭️ Skipped	Total	Pass %	⏱️ Duration
693	0	3	693	100	2h7m37.405634s

didier-wenzek

I will be happy to approve this PR. I just have some questions and minor comments.

crates/core/tedge_api/src/mqtt_topics.rs

didier-wenzek · 2025-09-03T13:36:08Z

crates/extensions/c8y_mapper_ext/src/converter.rs

+        self.supported_operations
+            .load_all(external_id, &self.config.bridge_config)?;


Curious. Why do we need the bridge config to load supported operations?

It seems just to get c8y_prefix eventually. Since the usage is very nested, so if you want refactoring, better to do in a separated PR.

That's okay for now. However, this how unrelated things are slowly more and more coupled. The config should be extended with a method giving the operation directory. Being the scene, the c8y_prefix used by the bridge can be used to infer the name of that directory.

crates/extensions/c8y_mapper_ext/src/supported_operations/mod.rs

crates/extensions/c8y_mapper_ext/src/tests.rs

docs/src/operate/c8y/supported-operations.md

didier-wenzek

Approved with one question.

What is missing to complete thin-edge/thin-edge.io-roadmap#101 and easy tenant migration. Operations can be now resent to the tenant. Do we need to do the same for twin data? log types ? sw types ? config types?

Also:

The branch name is unrelated to the new PR. It would be nice to fix that.

reubenmiller · 2025-09-09T15:18:29Z

@rina23q I'm wondering if we really need the level of control over which parts of synced...For example, we could call the signal "sync" and just keep extending what parts are synced...otherwise if we have individual signals for sync_operations, sync_logs, sync_configs, sync_twin, then it becomes annoying to use, as the main use-case is when migrating a device between two Cumulocity tenants, and you want to make sure the new tenant receives all of the device data after the migration.

rina23q · 2025-09-10T09:43:53Z

@rina23q I'm wondering if we really need the level of control over which parts of synced...For example, we could call the signal "sync" and just keep extending what parts are synced...otherwise if we have individual signals for sync_operations, sync_logs, sync_configs, sync_twin, then it becomes annoying to use, as the main use-case is when migrating a device between two Cumulocity tenants, and you want to make sure the new tenant receives all of the device data after the migration.

@reubenmiller I can rename it. How about sync_twins? Then probably easier to extend the actions what mapper does in the future.

rina23q · 2025-09-10T13:24:57Z

Renamed to sync in 11082cd

...tFramework/tests/cumulocity/supported_operations/mapper-publishing-agent-supported-ops.robot

reubenmiller

Approved. I only pushed some very minor doc changes.

Once we extend the scope of the "sync" signal, we'll need to make sure we update the docs accordingly to say that it syncs more than just the supported operations to cumulocity (e.g. twin data etc.)

Signed-off-by: Rina Fujino <rina.fujino.23@gmail.com>

…channel Signed-off-by: Rina Fujino <rina.fujino.23@gmail.com>

Signed-off-by: Rina Fujino <rina.fujino.23@gmail.com>

rina23q had a problem deploying to Test Pull Request August 15, 2025 17:32 — with GitHub Actions Failure

rina23q had a problem deploying to Test Pull Request August 19, 2025 14:33 — with GitHub Actions Failure

rina23q had a problem deploying to Test Pull Request August 25, 2025 20:02 — with GitHub Actions Failure

rina23q had a problem deploying to Test Pull Request August 25, 2025 22:05 — with GitHub Actions Failure

rina23q had a problem deploying to Test Pull Request August 26, 2025 13:30 — with GitHub Actions Failure

rina23q had a problem deploying to Test Pull Request August 27, 2025 14:58 — with GitHub Actions Failure

albinsuresh reviewed Aug 28, 2025

View reviewed changes

rina23q had a problem deploying to Test Pull Request August 28, 2025 12:23 — with GitHub Actions Failure

rina23q had a problem deploying to Test Pull Request August 28, 2025 12:27 — with GitHub Actions Failure

rina23q had a problem deploying to Test Pull Request August 28, 2025 13:34 — with GitHub Actions Failure

didier-wenzek reviewed Aug 28, 2025

View reviewed changes

reubenmiller reviewed Aug 29, 2025

View reviewed changes

docs/src/references/mqtt-api.md Outdated Show resolved Hide resolved

rina23q had a problem deploying to Test Pull Request September 2, 2025 15:04 — with GitHub Actions Failure

didier-wenzek mentioned this pull request Sep 3, 2025

feat: extensible log management #3762

Merged

19 tasks

rina23q had a problem deploying to Test Pull Request September 3, 2025 10:49 — with GitHub Actions Failure

rina23q temporarily deployed to Test Pull Request September 3, 2025 11:22 — with GitHub Actions Inactive

rina23q changed the title ~~feat: c8y-mapper sends child supported operations from cache at startup~~ feat: c8y-mapper sends all supported operations on-demand via signal channel Sep 3, 2025

rina23q temporarily deployed to Test Auto September 3, 2025 11:34 — with GitHub Actions Inactive

rina23q marked this pull request as ready for review September 3, 2025 11:34

rina23q requested review from a team and jarhodes314 as code owners September 3, 2025 11:34

didier-wenzek reviewed Sep 3, 2025

View reviewed changes

rina23q had a problem deploying to Test Pull Request September 3, 2025 18:01 — with GitHub Actions Failure

rina23q temporarily deployed to Test Pull Request September 3, 2025 18:10 — with GitHub Actions Inactive

rina23q temporarily deployed to Test Auto September 3, 2025 18:19 — with GitHub Actions Inactive

didier-wenzek approved these changes Sep 4, 2025

View reviewed changes

rina23q closed this Sep 4, 2025

rina23q force-pushed the improve/3738/tedge-mapper-c8y-can-reload-supported-opetation-by-option branch from f593c4b to 34c8dbc Compare September 5, 2025 12:52

rina23q temporarily deployed to Test Pull Request September 5, 2025 12:52 — with GitHub Actions Inactive

rina23q temporarily deployed to Test Auto September 5, 2025 13:01 — with GitHub Actions Inactive

rina23q had a problem deploying to Test Pull Request September 10, 2025 13:20 — with GitHub Actions Failure

rina23q force-pushed the improve/3738/tedge-mapper-c8y-can-reload-supported-opetation-by-option branch from 86dbfeb to 11082cd Compare September 10, 2025 13:29

rina23q temporarily deployed to Test Pull Request September 10, 2025 13:29 — with GitHub Actions Inactive

rina23q temporarily deployed to Test Auto September 10, 2025 13:38 — with GitHub Actions Inactive

reubenmiller reviewed Sep 10, 2025

View reviewed changes

...tFramework/tests/cumulocity/supported_operations/mapper-publishing-agent-supported-ops.robot Outdated Show resolved Hide resolved

rina23q temporarily deployed to Test Pull Request September 10, 2025 15:58 — with GitHub Actions Inactive

rina23q temporarily deployed to Test Auto September 10, 2025 16:08 — with GitHub Actions Inactive

reubenmiller had a problem deploying to Test Pull Request September 16, 2025 07:55 — with GitHub Actions Failure

reubenmiller had a problem deploying to Test Pull Request September 16, 2025 07:57 — with GitHub Actions Failure

reubenmiller approved these changes Sep 16, 2025

View reviewed changes

reubenmiller added the theme:c8y Theme: Cumulocity related topics label Sep 16, 2025

rina23q force-pushed the improve/3738/tedge-mapper-c8y-can-reload-supported-opetation-by-option branch from 1ac3136 to ee20a69 Compare September 16, 2025 11:19

rina23q had a problem deploying to Test Pull Request September 16, 2025 11:19 — with GitHub Actions Failure

rina23q added 4 commits September 16, 2025 11:20

Add signal channel to MQTT API doc

7e5c5be

Signed-off-by: Rina Fujino <rina.fujino.23@gmail.com>

Add re-publishing supported operations system test

909fbe4

Signed-off-by: Rina Fujino <rina.fujino.23@gmail.com>

feat: c8y-mapper sends all supported operations on-demand via signal …

fe80cd2

…channel Signed-off-by: Rina Fujino <rina.fujino.23@gmail.com>

Rename sync_operations to sync

4cc7191

Signed-off-by: Rina Fujino <rina.fujino.23@gmail.com>

rina23q force-pushed the improve/3738/tedge-mapper-c8y-can-reload-supported-opetation-by-option branch from ee20a69 to 4cc7191 Compare September 16, 2025 11:20

rina23q temporarily deployed to Test Pull Request September 16, 2025 11:20 — with GitHub Actions Inactive

rina23q temporarily deployed to Test Auto September 16, 2025 11:36 — with GitHub Actions Inactive

rina23q added this pull request to the merge queue Sep 16, 2025

Merged via the queue into thin-edge:main with commit 1e34d53 Sep 16, 2025
34 checks passed

rina23q deleted the improve/3738/tedge-mapper-c8y-can-reload-supported-opetation-by-option branch September 16, 2025 13:24

		\| <signal_type> \| Signal type. Each signal can define its own payload schema to allow signals to have parameters related to the signal's function. \|
		\| <action> \| Action for the signal type. e.g. `check` \|

Type	Example Topic
Command	/cmd/firmware_update/c8y-1234
Signal	/sig/config_type/check

		self.supported_operations
		.load_all(external_id, &self.config.bridge_config)?;

Conversation

rina23q commented Aug 15, 2025 • edited by reubenmiller Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Proposed changes

Future plan

Types of changes

Paste Link to the issue

Checklist

Further comments

Uh oh!

codecov bot commented Aug 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

reubenmiller commented Aug 18, 2025

Uh oh!

Uh oh!

albinsuresh Aug 28, 2025

Choose a reason for hiding this comment

Uh oh!

reubenmiller Aug 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rina23q Sep 2, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

didier-wenzek Aug 28, 2025

Choose a reason for hiding this comment

Uh oh!

reubenmiller Aug 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rina23q Sep 2, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

github-actions bot commented Sep 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Robot Results

Uh oh!

didier-wenzek left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

didier-wenzek Sep 3, 2025

Choose a reason for hiding this comment

Uh oh!

rina23q Sep 3, 2025

Choose a reason for hiding this comment

Uh oh!

didier-wenzek Sep 3, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

didier-wenzek left a comment

Choose a reason for hiding this comment

Uh oh!

reubenmiller commented Sep 9, 2025

Uh oh!

rina23q commented Sep 10, 2025

Uh oh!

rina23q commented Sep 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

reubenmiller left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

rina23q commented Aug 15, 2025 •

edited by reubenmiller

Loading

codecov bot commented Aug 15, 2025 •

edited

Loading

reubenmiller Aug 28, 2025 •

edited

Loading

reubenmiller Aug 29, 2025 •

edited

Loading

github-actions bot commented Sep 3, 2025 •

edited

Loading

rina23q commented Sep 10, 2025 •

edited

Loading