I'm wondering if this would meet the dynamic subscription requirements of the dynamic mapper, where in most cases when new pipelines are dynamically added, they are most likely going to subscribe to already subscribed topics like te/+/+/+/+/m/temp/meta and would be expecting the retained messages from those topics. But, since dynamic connections are established only when a new subscription is added or an existing one is upgraded, the newly added pipelines won't get the existing retained messages as far as I understand. This won't meet our requirement, right? I'm just wondering if blindly establishing a new connection per peer/client (pipeline) would be better here.

But, since dynamic connections are established only when a new subscription is added or an existing one is upgraded

No, the exact opposite. We establish a dynamic connection when we receive a subscription that does already exist.

I'm just wondering if blindly establishing a new connection would be better here.

If we did this blindly for all topics, all that would achieve is retain messages sent to newly subscribed topics would be received twice.

No, the exact opposite. We establish a dynamic connection when we receive a subscription that does already exist.

Not sure what I was doing wrong then. When I tested the same by tweaking the dynamic_subscribers_receive_retain_messages test with the following code at the end:

client_0
        .send(PublishOrSubscribe::Subscribe(SubscriptionRequest {
            diff: SubscriptionDiff {
                subscribe: ["a/b".into()].into(),
                unsubscribe: [].into(),
            },
            client_id: client_id_1,
        }))
        .await
        .unwrap();
assert_eq!(timeout(client_0.recv()).await.unwrap(), msg);

the assertion validating if client0 gets the same retained message again, timed out.

That is the intention. A new client subscribing shouldn't redeliver the retain messages to an existing subscription?

That's where I see a problem. For all the pipelines, the generic mapper is the only peer of the mqtt actor, right? While it is connected to the mqtt actor while being built, it would have declared a set of subscriptions based on all the pipelines that existed at that time. And when a new pipeline is dynamically registered, but with some topics that the previous pipelines had already subscribed, it should still get the retained messages on those previously registered topics as well, right? Since the new pipeline is not registering as a new peer/client, but issuing the subscription request via the same gen mapper peer which has already received the retained messages for the same topics, it wouldn't get those retained messages even though they are "new" to the new pipeline, right? That's my concern.

As that is not so easy to fix, an option is to let the tedge_gen_mapper maintains a connection to the MQTT actor per pipeline.

With the dynamic mapper more likely to use the newly added dynamic connection requests, I'm just wondering who/when the dynamic subscription requests would be used.

I think the mapper should be using both (if we change the input topic for a mapping rule, for instance, that will cause a change in the subscription, but shouldn't need a new MQTT actor connection).

Also, the "connection" part has nothing to do with MQTT connections and is merely a dynamic connection between two actors.

With the dynamic mapper more likely to use the newly added dynamic connection requests, I'm just wondering who/when the dynamic subscription requests would be used.