[fix][broker] Prevent unexpected recycle failure in dispatcher's read callback #24741
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Contributor
Author
|
While the reason of #24697 is still in investigation, this PR aims at the hard-to-debug recycling issue that did happen in real world by eliminating the recycler in read path |
lhotari
reviewed
Sep 15, 2025
...va/org/apache/pulsar/broker/service/persistent/PersistentDispatcherSingleActiveConsumer.java
Outdated
Show resolved
Hide resolved
Closed
3 tasks
nodece
reviewed
Sep 16, 2025
...va/org/apache/pulsar/broker/service/persistent/PersistentDispatcherSingleActiveConsumer.java
Outdated
Show resolved
Hide resolved
coderzc
approved these changes
Sep 17, 2025
lhotari
reviewed
Sep 18, 2025
Member
lhotari
left a comment
There was a problem hiding this comment.
LGTM, great job @BewareMyPower.
Member
|
Cherry-picking to branch-3.0 causes merge conflicts mainly due to #22285 being missing in branch-3.0, perhaps others. It was fairly easy to backport to branch-3.3. @BewareMyPower can you please handle cherry-picking to branch-3.0? I already resolved other merge conflicts in branch-3.3 commit 65ddd44 so you should be able to cherry-pick that to branch-3.0 once the preconditions in branch-3.0 have been addressed. |
Contributor
Author
|
Okay, I will do that |
ganesh-ctds
pushed a commit
to datastax/pulsar
that referenced
this pull request
Sep 18, 2025
… callback (apache#24741) (cherry picked from commit ccbd245) (cherry picked from commit 215a1fd)
Contributor
Author
srinath-ctds
pushed a commit
to datastax/pulsar
that referenced
this pull request
Sep 18, 2025
… callback (apache#24741) (cherry picked from commit ccbd245) (cherry picked from commit 215a1fd)
nodece
pushed a commit
to ascentstream/pulsar
that referenced
this pull request
Sep 22, 2025
… callback (apache#24741) (cherry picked from commit ccbd245)
KannarFr
pushed a commit
to CleverCloud/pulsar
that referenced
this pull request
Sep 22, 2025
walkinggo
pushed a commit
to walkinggo/pulsar
that referenced
this pull request
Oct 8, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Main Issue: #24697
Motivation
Using callback-based read API with a Netty recyclable object is buggy that once the
ReadEntriesCtxobject has been recycled twice.Modifications
asyncReadEntriesWithSkipOrWaittestConcurrentReadto verify concurrent read will failCompactedTopicUtils#asyncReadCompactedEntriesAPI to return a future as wellReadEntriesCtxIn extreme cases like #24697, it's safe to deliver messages to the consumer after that.
Documentation
docdoc-requireddoc-not-neededdoc-completeMatching PR in forked repository
PR in forked repository: