Skip to content

Metadata refresh should skip updates when metadata response is empty #2664

@HaoSun1993

Description

@HaoSun1993
Description

After my PR, the Sarama client will always pick up a least loaded broker from cached brokers to fetch metadata. Then the metadata response will update our cached brokers (updateBroker).

But if the chosen broker returns a response with empty brokers, updateBroker will clear our cached brokers into empty. The client will fall back to the seed broker to refresh metadata in the next metadata refresh because cached brokers are empty. This is what we want to avoid using the seed broker after bootstrap due to the stale metadata issue (issues).

Technically, the broker should not return an empty broker in the response. But we do come across this situation where the broker returns empty brokers without any error and emptys our cached brokers. It looks like a bug in Kafka. The Java client skips the empty response to update the metadata cache (code).

In Sarama, we should also implement the same logic to skip the update when the response is empty and move to the next candidate broker.

I can make a PR if you think it is reasonable.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions