Skip to content

feat: tedge flows statistics are published over MQTT#3843

Merged
didier-wenzek merged 5 commits intothin-edge:mainfrom
didier-wenzek:feat/tedge-flow-stats-over-mqtt
Feb 25, 2026
Merged

feat: tedge flows statistics are published over MQTT#3843
didier-wenzek merged 5 commits intothin-edge:mainfrom
didier-wenzek:feat/tedge-flow-stats-over-mqtt

Conversation

@didier-wenzek
Copy link
Copy Markdown
Contributor

@didier-wenzek didier-wenzek commented Oct 30, 2025

Proposed changes

  • Publish basic processing stats over MQTT
  • Publish memory usage
  • Configure periodicity
  • Configure what is published, what is not

Types of changes

  • Bugfix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Improvement (general improvements like code refactoring that doesn't explicitly fix a bug or add any new functionality)
  • Documentation Update (if none of the other choices apply)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)

Paste Link to the issue


Checklist

  • I have read the CONTRIBUTING doc
  • I have signed the CLA (in all commits with git commit -s. You can activate automatic signing by running just prepare-dev once)
  • I ran just format as mentioned in CODING_GUIDELINES
  • I used just check as mentioned in CODING_GUIDELINES
  • I have added tests that prove my fix is effective or that my feature works
  • I have added necessary documentation (if appropriate)

Further comments

@codecov
Copy link
Copy Markdown

codecov bot commented Oct 30, 2025

Codecov Report

❌ Patch coverage is 70.45455% with 52 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
crates/extensions/tedge_flows/src/stats.rs 75.92% 12 Missing and 1 partial ⚠️
crates/core/tedge_mapper/src/flows/mod.rs 0.00% 11 Missing ⚠️
crates/core/tedge_mapper/src/aws/mapper.rs 0.00% 8 Missing ⚠️
crates/core/tedge_mapper/src/az/mapper.rs 0.00% 8 Missing ⚠️
crates/core/tedge_mapper/src/c8y/mapper.rs 0.00% 8 Missing ⚠️
crates/extensions/tedge_flows/src/actor.rs 91.42% 1 Missing and 2 partials ⚠️
crates/extensions/tedge_flows/src/lib.rs 97.29% 1 Missing ⚠️

📢 Thoughts on this report? Let us know!

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Nov 20, 2025

Robot Results

✅ Passed ❌ Failed ⏭️ Skipped Total Pass % ⏱️ Duration
843 0 3 843 100 2h20m52.092924s

Comment on lines +23 to +26
let service_config = FlowsMapperConfig {
statistics_topic: Topic::new(&format!("{te}/{service_id}/statistics"))?,
status_topic: Topic::new(&format!("{te}/{service_id}/status"))?,
};
Copy link
Copy Markdown
Contributor Author

@didier-wenzek didier-wenzek Nov 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's now time to discuss the appropriate topics for flows status and statistics #3846 (comment)

I think that it would make sense to publish the statistics as measurements under {te}/{service_id}/m/statistics. But then these stats will be pushed by default to the cloud.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah publishing under the standard thin-edge.io mqtt topic structure would be more consistent, but we might need to use a different telemetry endpoint to publish this information to avoid sending too much information to the cloud by default.

We could create a convention where some local/internal information which is meant for local/on-device consumption (rather than intended for the cloud):

{te}/{service_id}/local/statistics

Alternatively, we could use flows as the type since the flows statistics maybe published on serveral services in the future, e.g. tedge-mapper-c8y, tedge-mapper-az, tedge-flows etc.

{te}/{service_id}/flows/statistics

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A new proposal for the topic would be:

{te}/{service_id}/status/metrics

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A new proposal for the topic would be:

{te}/{service_id}/status/metrics

Done: f0f75b7

@didier-wenzek didier-wenzek force-pushed the feat/tedge-flow-stats-over-mqtt branch from 65eab46 to 9bd711c Compare December 19, 2025 09:56
@didier-wenzek didier-wenzek force-pushed the feat/tedge-flow-stats-over-mqtt branch from 9bd711c to 98acd21 Compare December 19, 2025 13:58
@didier-wenzek didier-wenzek force-pushed the feat/tedge-flow-stats-over-mqtt branch from 98acd21 to f7afd1b Compare January 15, 2026 17:15
@didier-wenzek didier-wenzek marked this pull request as ready for review January 19, 2026 08:46
@reubenmiller
Copy link
Copy Markdown
Contributor

reubenmiller commented Feb 20, 2026

@didier-wenzek Is there an example of some of the expected stats? For testing I changed the stats interval to 1s but I seem to be only getting the memory stats and not the processing (though I have one flow installed). I was expecting to see some processing stats like number of messages processed etc. as the tedge-flows log entries seems to indicate that some "processing statistics" would be present.

Below is the only message that I'm seeing (at least at the interval as set in the config)

[te/device/main/service/tedge-flows/status/metrics/memory] {"function_count":0,"malloc_bytes":91095,"memory_used_bytes":82848,"object_count":192}

@reubenmiller
Copy link
Copy Markdown
Contributor

@didier-wenzek Is there an example of some of the expected stats? For testing I changed the stats interval to 1s but I seem to be only getting the memory stats and not the processing (though I have one flow installed). I was expecting to see some processing stats like number of messages processed etc. as the tedge-flows log entries seems to indicate that some "processing statistics" would be present.

Below is the only message that I'm seeing (at least at the interval as set in the config)

[te/device/main/service/tedge-flows/status/metrics/memory] {"function_count":0,"malloc_bytes":91095,"memory_used_bytes":82848,"object_count":192}

I found the stats I was looking for

[te/device/main/service/tedge-mapper-c8y/status/metrics/memory] {"function_count":2,"malloc_bytes":96047,"memory_used_bytes":87241,"object_count":198}
[te/device/main/service/tedge-mapper-c8y/status/metrics/add-timestamp] {"cpu-max":"15.334µs","cpu-min":"1.417µs","error":0,"input":18,"output":18}
[te/device/main/service/tedge-mapper-c8y/status/metrics/alarms.toml] {"cpu-max":"13.71µs","cpu-min":"4.543µs","error":0,"input":6,"output":0}
[te/device/main/service/tedge-mapper-c8y/status/metrics/into-c8y-health-status] {"cpu-max":"411.626µs","cpu-min":"13.835µs","error":0,"input":6,"output":7}
[te/device/main/service/tedge-mapper-c8y/status/metrics/runtime] {"cpu-max":"639.567µs","cpu-min":"5.959µs","error":0,"input":12,"output":7}
[te/device/main/service/tedge-mapper-c8y/status/metrics/events.toml] {"cpu-max":"42.004µs","cpu-min":"5.5µs","error":0,"input":6,"output":0}
[te/device/main/service/tedge-mapper-c8y/status/metrics/cache-early-messages] {"cpu-max":"167.851µs","cpu-min":"1.834µs","error":0,"input":30,"output":6}
[te/device/main/service/tedge-mapper-c8y/status/metrics/collectd.toml] {"error":0,"input":0,"output":0}
[te/device/main/service/tedge-mapper-c8y/status/metrics/measurements.toml] {"cpu-max":"13.376µs","cpu-min":"4.751µs","error":0,"input":6,"output":0}
[te/device/main/service/tedge-mapper-c8y/status/metrics/units.toml] {"error":0,"input":0,"output":0}
[te/device/main/service/tedge-mapper-c8y/status/metrics/health.toml] {"cpu-max":"632.232µs","cpu-min":"2.709µs","error":0,"input":12,"output":7}
[te/device/main/service/tedge-mapper-c8y/status/metrics/into-c8y-alarms.onInterval] {"cpu-max":"18.335µs","cpu-min":"1.917µs","error":0,"input":26,"output":0}

@reubenmiller
Copy link
Copy Markdown
Contributor

It seems that the stats are recorded on both the overall flow and individual steps.

For instance, the alarms.toml file uses the add-timestamp function within the flow definition.

[te/device/main/service/tedge-mapper-c8y/status/metrics/add-timestamp] {"cpu-max":"15.334µs","cpu-min":"1.417µs","error":0,"input":18,"output":18}
[te/device/main/service/tedge-mapper-c8y/status/metrics/alarms.toml] {"cpu-max":"13.71µs","cpu-min":"4.543µs","error":0,"input":6,"output":0}

It would be useful to be able to distinguish between the flow stats and the flow-step stats. Currently you could infer that a flow's stats has the ".toml" file in the name, however it isn't so obvious.

We could just add a property the payload like "type", to determine if it is a "flow" or an onMesssage/onInterval handler, or introduce another nested topic level (e.g. /status/metrics/flows/alarms.toml, and /status/metrics/function/add-timestamp).

@didier-wenzek
Copy link
Copy Markdown
Contributor Author

It would be useful to be able to distinguish between the flow stats and the flow-step stats. Currently you could infer that a flow's stats has the ".toml" file in the name, however it isn't so obvious.

We could just add a property the payload like "type", to determine if it is a "flow" or an onMesssage/onInterval handler, or introduce another nested topic level (e.g. /status/metrics/flows/alarms.toml, and /status/metrics/function/add-timestamp).

I added a type to each stats record as well as a way to enable/disable low level stats (onMessage and onInterval)

See b14d815

@reubenmiller
Copy link
Copy Markdown
Contributor

@didier-wenzek Nice improvements, though I would only propose one small fix to apply a minimum interval to prevent misconfiguration (1 second would be fine), as the user can set a very short interval then the stats publishing spams the mqtt broker, for example:

tedge config set flows.stats.interval 0s
systemctl restart tedge-mapper-c8y
tedge mqtt sub '#'

@didier-wenzek
Copy link
Copy Markdown
Contributor Author

@didier-wenzek Nice improvements, though I would only propose one small fix to apply a minimum interval to prevent misconfiguration (1 second would be fine), as the user can set a very short interval then the stats publishing spams the mqtt broker, for example:

tedge config set flows.stats.interval 0s
systemctl restart tedge-mapper-c8y
tedge mqtt sub '#'

Done 8db18c6

Copy link
Copy Markdown
Contributor

@reubenmiller reubenmiller left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved, great persistence :)

Signed-off-by: Didier Wenzek <didier.wenzek@free.fr>
Signed-off-by: Didier Wenzek <didier.wenzek@free.fr>
Signed-off-by: Didier Wenzek <didier.wenzek@free.fr>
The default is a statistics dump every hours

Signed-off-by: Didier Wenzek <didier.wenzek@free.fr>
Signed-off-by: Didier Wenzek <didier.wenzek@free.fr>
@didier-wenzek didier-wenzek force-pushed the feat/tedge-flow-stats-over-mqtt branch from e397633 to d025e0d Compare February 25, 2026 10:54
@didier-wenzek didier-wenzek added this pull request to the merge queue Feb 25, 2026
Merged via the queue into thin-edge:main with commit fa3e244 Feb 25, 2026
32 checks passed
@didier-wenzek didier-wenzek deleted the feat/tedge-flow-stats-over-mqtt branch February 25, 2026 12:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants