Skip to content

fix: MQTT messages are forwarded twice to subscribers with overlapping subscriptions#3789

Merged
didier-wenzek merged 2 commits intothin-edge:mainfrom
didier-wenzek:fix/duplicated-mqtt-message
Sep 19, 2025
Merged

fix: MQTT messages are forwarded twice to subscribers with overlapping subscriptions#3789
didier-wenzek merged 2 commits intothin-edge:mainfrom
didier-wenzek:fix/duplicated-mqtt-message

Conversation

@didier-wenzek
Copy link
Copy Markdown
Contributor

When an actor subscribes to overlapping topics, e.g. a/+ and +/b, then all messages received on the overlapping topics (in that case a/b) are forwarded twice to the actor, and even four times with some MQTT brokers (rumqttc and nanomq).

  • Little can be done when the broker itself duplicates the messages. But at least, the tedge_mqtt_ext layer should not add its own set of duplicates.
  • Its difficult to avoid such overlapping subscriptions with tedge flows where the subscriptions are independently configured for each flow.

Proposed changes

  • Let tedge_mqtt_ext removes duplicates in the list of clients for a message

Types of changes

  • Bugfix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Improvement (general improvements like code refactoring that doesn't explicitly fix a bug or add any new functionality)
  • Documentation Update (if none of the other choices apply)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)

Paste Link to the issue


Checklist

  • I have read the CONTRIBUTING doc
  • I have signed the CLA (in all commits with git commit -s. You can activate automatic signing by running just prepare-dev once)
  • I ran just format as mentioned in CODING_GUIDELINES
  • I used just check as mentioned in CODING_GUIDELINES
  • I have added tests that prove my fix is effective or that my feature works
  • I have added necessary documentation (if appropriate)

Further comments

Signed-off-by: Didier Wenzek <didier.wenzek@free.fr>
Signed-off-by: Didier Wenzek <didier.wenzek@free.fr>
Comment on lines -630 to +631
for client in matches {
for client in HashSet::<ClientId>::from_iter(matches) {
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This fixes the issue, but might not be the best way. Can this de-duplication be done internally when collecting the subscribers from the trie?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe we could deduplicate in the trie logic, but I don't think we would gain anything over this solution by doing that.

Comment on lines +275 to +278
// a more convincing example would be to subscribe to "+/b"
// but in that case rumqttd sends the message twice
// something we can do little about
subscribe: ["a/b".into()].into(),
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adding a subscription to +/b leads to 4 messages being delivered to the client.

@didier-wenzek didier-wenzek changed the title fix: MQTT messages being forwarded twice to subscribers with overlapping subscriptions fix: MQTT messages are forwarded twice to subscribers with overlapping subscriptions Sep 18, 2025
@didier-wenzek didier-wenzek added bug Something isn't working theme:mqtt Theme: mqtt and mosquitto related topics labels Sep 18, 2025
@github-actions
Copy link
Copy Markdown
Contributor

Robot Results

✅ Passed ❌ Failed ⏭️ Skipped Total Pass % ⏱️ Duration
693 0 3 693 100 2h0m13.632920999s

Copy link
Copy Markdown
Contributor

@albinsuresh albinsuresh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The fix looks fine and AFAIK, this topic overlap issue was highlighted as a known limitation when dynamic subscription was implemented. So, I doubt if anything else can be done. So, I'll let @jarhodes314 approve the PR.

@codecov
Copy link
Copy Markdown

codecov bot commented Sep 19, 2025

Codecov Report

❌ Patch coverage is 87.80488% with 5 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
crates/extensions/tedge_mqtt_ext/src/tests.rs 87.50% 0 Missing and 5 partials ⚠️

📢 Thoughts on this report? Let us know!

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@didier-wenzek didier-wenzek added this pull request to the merge queue Sep 19, 2025
Merged via the queue into thin-edge:main with commit f14465c Sep 19, 2025
51 of 52 checks passed
@didier-wenzek didier-wenzek deleted the fix/duplicated-mqtt-message branch September 20, 2025 08:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working theme:mqtt Theme: mqtt and mosquitto related topics

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants