Skip to content

[Bug] The leader broker identifier becomes unresolvable if the internal listener is not configured to match the advertised address. #21897

@Meet0861

Description

@Meet0861

Search before asking

  • I searched in the issues and found nothing similar.

Version

2.10.4

Minimal reproduce step

We are hitting a bug where a leader broker identifier becomes unresolvable when a internalListener is not configured to match the advertised address because the leader broker's identifier uses the internal listener's host name and in all other cases the advertisedAddress is used.

warnings in broker logs:
05:19:57.541 [pulsar-2-5] WARN org.apache.pulsar.broker.namespace.NamespaceService - The current leader broker LeaderBroker(serviceUrl=http://xx.yy.zz.xy:8080/) isn't active. Handling load manager decisions in a decentralized way. NamespaceBundle[tenant/ns/0x2fffffff_0x3269f69d]

Reason -
05:19:55.176 [pulsar-2-6] WARN org.apache.pulsar.broker.namespace.NamespaceService - Broker http://xx.yy.zz.xy:8080/ (xx.yy.zz.xy:8080) couldn't be found in available brokers prod-xx-1-pulsar-broker-10:8080,prod-xx-1-pulsar-broker-yy-7:8080,prod-xx-1-pulsar-broker-zz-5:8080,prod-xx-1-pulsar-broker-7:8080,prod-xx-1-pulsar-broker-zz-2:8080,prod-xx-1-pulsar-broker-yy-1:8080,prod-xx-1-pulsar-broker-yy-4:8080,prod-xx-1-pulsar-broker-2:8080,prod-xx-1-pulsar-broker-12:8080,prod-xx-1-pulsar-broker-yy-2:8080,prod-xx-1-pulsar-broker-15:8080

here it seems related to the advertised listerners/address. we are setting the advertisedAddress as a hostname of a stateful set and advertisedListeners as broker pod ips and not setting a internalListenerName

from above logs, we can see that for leader broker's load manager using the (ip:port) because if we dont set the internal Listener Name it picks the first advertised listener as the internalListener Name.

Configs :

advertisedAddress=prod-xx-pulsar-broker-0
advertisedListeners=public_pulsar:pulsar://xx.yy.zz.xy:6650,public_http:http://xx.yy.zz.xy:8080/    

What did you expect to see?

There should be consistent ways for a unique identifier for a broker.

What did you see instead?

Inconsistency in resolving the leader broker identifier, compared to other brokers, results in incorrect information regarding the unavailability of the leader broker. Consequently, client-side disconnections occur due to the unresolvable information about the leader broker.

Anything else?

No response

Are you willing to submit a PR?

  • I'm willing to submit a PR!

Metadata

Metadata

Assignees

No one assigned

    Labels

    type/bugThe PR fixed a bug or issue reported a bug

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions