Search before asking

• I searched in the issues and found nothing similar.

Read release policy

• I understand that unsupported versions don't get bug fixes. I will attempt to reproduce the issue on a supported version of Pulsar client and Pulsar broker.

Version

any released version

Minimal reproduce step

problem description:
This problem applies to Shared and Key_Shared subscriptions.
When a consumer disconnects and while there are other active consumers, the unacknowledged (pending) message ids get added to the redelivery (replay) queue in the dispatcher. When these messages get dispatched on the next round in the Shared or Key_Shared subscription dispatcher, the reads for the messages don't get deduplicated or limited by managedLedgerMaxReadsInFlightSizeInMB. This is a problem for high fanout scenarios for Shared subscriptions, when a large amount of consumers disconnect and reconnect while there's at least one active consumer so that the dispatcher doesn't discard the replay queue and "rewind" in between.

Additional problem

When an entry gets added to the replay queue, it doesn't get added to the cache. The read for this entry will usually happen short after and this will result in a new read to bookkeeper. This issue will be reported separately so that it's possible to address issues individually.

What did you expect to see?

That redelivery reads get deduplicate and limited by managedLedgerMaxReadsInFlightSizeInMB.

What did you see instead?

PendingReadsManager isn't used, so all reads will be made multiple times to bookkeeper (in fanout with multiple consumers). The reads won't be limited by managedLedgerMaxReadsInFlightSizeInMB.

Anything else?

No response

Are you willing to submit a PR?

• I'm willing to submit a PR!