fix 8115 Some partitions get stuck after adding additional consumers to the KEY_SHARED subscriptions #10096
Conversation
.../apache/pulsar/broker/service/persistent/PersistentStickyKeyDispatcherMultipleConsumers.java
Show resolved
Hide resolved
.../apache/pulsar/broker/service/persistent/PersistentStickyKeyDispatcherMultipleConsumers.java
Show resolved
Hide resolved
|
@baomingyu Please help fix the ci failure for checkstyle violation. |
|
/pulsarbot run-failure-checks |
1 similar comment
|
/pulsarbot run-failure-checks |
|
I tested these code. but it still stucked when I added a new key_shared consumer. |
|
@gaozhangmin Could you please help create a new issue for your last comment? It's better to provide the topic stats and topic internal stats, so that we can investigate it in detail. I will merge this fix first, properly this one can't fix all cases about the consumer stuck issue, but it has fixed the certain one. |
new issue #10167 |
…to the KEY_SHARED subscriptions (apache#10096) Fixes apache#8115 Master Issue: apache#8115 ### Motivation first point: Sometimes it will not success to call this method and the method readMoreEntries will not be called ` if (future.isSuccess() && keyNumbers.decrementAndGet() == 0) { readMoreEntries(); } ` second point: Sometimes keyNumbers will not be decrement to zero , and broker will not be start next loop to readMoreEntries. some partition topic will be stunk and stop to push message to consumer ,even though there is permits in consumers.
…to the KEY_SHARED subscriptions (#10096) Fixes #8115 Master Issue: #8115 first point: Sometimes it will not success to call this method and the method readMoreEntries will not be called ` if (future.isSuccess() && keyNumbers.decrementAndGet() == 0) { readMoreEntries(); } ` second point: Sometimes keyNumbers will not be decrement to zero , and broker will not be start next loop to readMoreEntries. some partition topic will be stunk and stop to push message to consumer ,even though there is permits in consumers. (cherry picked from commit c4f154e)
| } else if (currentThreadKeyNumber == 0) { | ||
| topic.getBrokerService().executor().schedule(() -> { | ||
| synchronized (PersistentStickyKeyDispatcherMultipleConsumers.this) { | ||
| readMoreEntries(); | ||
| } | ||
| }, 100, TimeUnit.MILLISECONDS); |
There was a problem hiding this comment.
One detail is that before this PR there was a bug that if any connection was backpressured, readMoreEntries wouldn't be called and that would block all progress.
The consequence of this change seems to be that readMoreEntries is called twice. That doesn't seem to make sense to me. It seems that #16812 was added to prevent duplicate calls happening at the same time so I guess that is mitigated.
Fixes #8115
Master Issue: #8115
Motivation
first point:
Sometimes it will not success to call this method and the method readMoreEntries will not be called
if (future.isSuccess() && keyNumbers.decrementAndGet() == 0) { readMoreEntries(); }second point:
Sometimes keyNumbers will not be decrement to zero , and broker will not be start next loop to readMoreEntries.
some partition topic will be stunk and stop to push message to consumer ,even though there is permits in consumers.