Search before asking
Version
2.10.4
Minimal reproduce step
We are hitting a bug where a leader broker identifier becomes unresolvable when a internalListener is not configured to match the advertised address because the leader broker's identifier uses the internal listener's host name and in all other cases the advertisedAddress is used.
warnings in broker logs:
05:19:57.541 [pulsar-2-5] WARN org.apache.pulsar.broker.namespace.NamespaceService - The current leader broker LeaderBroker(serviceUrl=http://xx.yy.zz.xy:8080/) isn't active. Handling load manager decisions in a decentralized way. NamespaceBundle[tenant/ns/0x2fffffff_0x3269f69d]
Reason -
05:19:55.176 [pulsar-2-6] WARN org.apache.pulsar.broker.namespace.NamespaceService - Broker http://xx.yy.zz.xy:8080/ (xx.yy.zz.xy:8080) couldn't be found in available brokers prod-xx-1-pulsar-broker-10:8080,prod-xx-1-pulsar-broker-yy-7:8080,prod-xx-1-pulsar-broker-zz-5:8080,prod-xx-1-pulsar-broker-7:8080,prod-xx-1-pulsar-broker-zz-2:8080,prod-xx-1-pulsar-broker-yy-1:8080,prod-xx-1-pulsar-broker-yy-4:8080,prod-xx-1-pulsar-broker-2:8080,prod-xx-1-pulsar-broker-12:8080,prod-xx-1-pulsar-broker-yy-2:8080,prod-xx-1-pulsar-broker-15:8080
here it seems related to the advertised listerners/address. we are setting the advertisedAddress as a hostname of a stateful set and advertisedListeners as broker pod ips and not setting a internalListenerName
from above logs, we can see that for leader broker's load manager using the (ip:port) because if we dont set the internal Listener Name it picks the first advertised listener as the internalListener Name.
Configs :
advertisedAddress=prod-xx-pulsar-broker-0
advertisedListeners=public_pulsar:pulsar://xx.yy.zz.xy:6650,public_http:http://xx.yy.zz.xy:8080/
What did you expect to see?
There should be consistent ways for a unique identifier for a broker.
What did you see instead?
Inconsistency in resolving the leader broker identifier, compared to other brokers, results in incorrect information regarding the unavailability of the leader broker. Consequently, client-side disconnections occur due to the unresolvable information about the leader broker.
Anything else?
No response
Are you willing to submit a PR?
Search before asking
Version
2.10.4
Minimal reproduce step
We are hitting a bug where a leader broker identifier becomes unresolvable when a internalListener is not configured to match the advertised address because the leader broker's identifier uses the internal listener's host name and in all other cases the advertisedAddress is used.
warnings in broker logs:
05:19:57.541 [pulsar-2-5] WARN org.apache.pulsar.broker.namespace.NamespaceService - The current leader broker LeaderBroker(serviceUrl=http://xx.yy.zz.xy:8080/) isn't active. Handling load manager decisions in a decentralized way. NamespaceBundle[tenant/ns/0x2fffffff_0x3269f69d]Reason -
05:19:55.176 [pulsar-2-6] WARN org.apache.pulsar.broker.namespace.NamespaceService - Broker http://xx.yy.zz.xy:8080/ (xx.yy.zz.xy:8080) couldn't be found in available brokers prod-xx-1-pulsar-broker-10:8080,prod-xx-1-pulsar-broker-yy-7:8080,prod-xx-1-pulsar-broker-zz-5:8080,prod-xx-1-pulsar-broker-7:8080,prod-xx-1-pulsar-broker-zz-2:8080,prod-xx-1-pulsar-broker-yy-1:8080,prod-xx-1-pulsar-broker-yy-4:8080,prod-xx-1-pulsar-broker-2:8080,prod-xx-1-pulsar-broker-12:8080,prod-xx-1-pulsar-broker-yy-2:8080,prod-xx-1-pulsar-broker-15:8080here it seems related to the advertised listerners/address. we are setting the advertisedAddress as a hostname of a stateful set and advertisedListeners as broker pod ips and not setting a internalListenerName
from above logs, we can see that for leader broker's load manager using the (ip:port) because if we dont set the internal Listener Name it picks the first advertised listener as the internalListener Name.
Configs :
What did you expect to see?
There should be consistent ways for a unique identifier for a broker.
What did you see instead?
Inconsistency in resolving the leader broker identifier, compared to other brokers, results in incorrect information regarding the unavailability of the leader broker. Consequently, client-side disconnections occur due to the unresolvable information about the leader broker.
Anything else?
No response
Are you willing to submit a PR?