-
Notifications
You must be signed in to change notification settings - Fork 1.8k
Metadata refresh should skip updates when metadata response is empty #2664
Description
Description
After my PR, the Sarama client will always pick up a least loaded broker from cached brokers to fetch metadata. Then the metadata response will update our cached brokers (updateBroker).
But if the chosen broker returns a response with empty brokers, updateBroker will clear our cached brokers into empty. The client will fall back to the seed broker to refresh metadata in the next metadata refresh because cached brokers are empty. This is what we want to avoid using the seed broker after bootstrap due to the stale metadata issue (issues).
Technically, the broker should not return an empty broker in the response. But we do come across this situation where the broker returns empty brokers without any error and emptys our cached brokers. It looks like a bug in Kafka. The Java client skips the empty response to update the metadata cache (code).
In Sarama, we should also implement the same logic to skip the update when the response is empty and move to the next candidate broker.
I can make a PR if you think it is reasonable.