-
Notifications
You must be signed in to change notification settings - Fork 3.7k
[fix][client] Fix deduplication for getPartitionedTopicMetadata to include method parameters #24965
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[fix][client] Fix deduplication for getPartitionedTopicMetadata to include method parameters #24965
Conversation
…clude method parameters" Key partitionedMetadataInProgress by TopicName, metadataAutoCreationEnabled, and useFallbackForNonPIP344Brokers. This prevents coalescing of requests that should be distinct. Fixes apache#24963 Signed-off-by: Vinkal Chudgar <vinkal.chudgar@gmail.com>
Signed-off-by: Vinkal Chudgar <vinkal.chudgar@gmail.com>
|
Why could take the method params into account ? There will be more request to the broker, but the result is the same. |
lhotari
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
@Technoboy- I think that this is more about correctness than the possibility that the broker response might be the same in most cases. It's strictly not the same in all cases. Therefore I think that such bugs should be fixed. |
This is not a bug |
Technoboy-
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The cache is ok for the topicName, please give more possible issue for this bug
@Technoboy- I think that depends on how you define a bug. The impact of the issue is that |
@Technoboy- The reason we must include these parameters in the key is that they can lead to fundamentally different outcomes for the caller, breaking the API contract if not handled separately. The logic within Here is a practical example of why caching them separately is critical, as two calls with the same
Regarding your point about increasing request volume: this change still correctly deduplicates all concurrent calls that use the exact same parameters. A new request is only sent when the parameters are different, ensuring we respect the caller's intent while still preventing redundant traffic for identical calls. |
Make sense |
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #24965 +/- ##
============================================
- Coverage 74.44% 74.30% -0.14%
+ Complexity 34071 33520 -551
============================================
Files 1920 1920
Lines 150062 150077 +15
Branches 17404 17408 +4
============================================
- Hits 111708 111515 -193
- Misses 29495 29672 +177
- Partials 8859 8890 +31
Flags with carried forward coverage won't be shown. Click here to find out more.
🚀 New features to boost your workflow:
|
…clude method parameters (apache#24965) Signed-off-by: Vinkal Chudgar <vinkal.chudgar@gmail.com> (cherry picked from commit 0cdab92)
…clude method parameters (apache#24965) Signed-off-by: Vinkal Chudgar <vinkal.chudgar@gmail.com> (cherry picked from commit 0cdab92) (cherry picked from commit fac8f15)
…clude method parameters (apache#24965) Signed-off-by: Vinkal Chudgar <vinkal.chudgar@gmail.com> (cherry picked from commit 0cdab92) (cherry picked from commit fac8f15)
…clude method parameters (apache#24965) Signed-off-by: Vinkal Chudgar <vinkal.chudgar@gmail.com> (cherry picked from commit 0cdab92) (cherry picked from commit d0ff77e)
…clude method parameters (apache#24965) Signed-off-by: Vinkal Chudgar <vinkal.chudgar@gmail.com> (cherry picked from commit 0cdab92) (cherry picked from commit d0ff77e)
Fixes #24963
Motivation
The in-flight deduplication for
BinaryProtoLookupService#getPartitionedTopicMetadataused onlyTopicName. Calls that differ inmetadataAutoCreationEnabledoruseFallbackForNonPIP344Brokerswere coalesced, which could return a future that does not match the requested behavior.Concrete example:
getPartitionedTopicMetadata(topic, true, false)and creates a futuregetPartitionedTopicMetadata(topic, false, true)and receives the same future from AmetadataAutoCreationEnabled=true, which is incorrectThis PR fixes deduplication by keying in-flight requests on all method parameters.
Modifications
PartitionedTopicMetadataKey(static inner class) to uniquely identify requests bytopicName,metadataAutoCreationEnabled,useFallbackForNonPIP344BrokerspartitionedMetadataInProgressto usePartitionedTopicMetadataKeyas the map key instead ofTopicNamecomputeIfAbsentwith the new key and removed the entry on completion withremove(key, future)Verifying this change
This change added tests and can be verified as follows:
Does this pull request potentially affect one of the following parts:
If the box was checked, please highlight the changes
Documentation
docdoc-requireddoc-not-neededdoc-completeMatching PR in forked repository
PR in forked repository: vinkal-chudgar#5