Skip to content

[Bug] Consumer Stops Receiving Messages with Large Backlogs Post-Processing #22435

@KannarFr

Description

@KannarFr

Search before asking

  • I searched in the issues and found nothing similar.

Read release policy

  • I understand that unsupported versions don't get bug fixes. I will attempt to reproduce the issue on a supported version of Pulsar client and Pulsar broker.

Version

3.2.2

Minimal reproduce step

Open consumer on a subscription using very large backlog by passing "old" MessageId.

What did you expect to see?

Consume all messages and wait for upcoming messages when the backlog is consumed.

What did you see instead?

Note that the following paragraph mentions a NonDurable consumer.

Initially, everything functions correctly—the process begins with a backlog of (let's say) 100,000 messages, which gradually decreases to 0 as we approach the present ("now"). However, once the backlog is fully processed, the consumer unexpectedly stops receiving new messages, leading to an increase in the backlog again. The consumer is still connected. This behavior is consistently reproducible with my topics that have a substantial amount of data.
I've also noticed that when the starting point (since) is relatively close to the current time (now), this problem does not occur.

Anything else?

We do not know if this bug was introduced by v3.2.2. We didn't see it before. We are currently rollbacking brokers to 3.2.1 to confirm this.

Otherwise, it may be related to #22191.

Are you willing to submit a PR?

  • I'm willing to submit a PR!

Metadata

Metadata

Assignees

Labels

type/bugThe PR fixed a bug or issue reported a bug

Type

No type
No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions