Skip to content

[fix][broker]Fix NonPersistentDispatcherMultipleConsumers ArrayIndexOutOfBoundsException#21856

Merged
Technoboy- merged 1 commit into
apache:masterfrom
Technoboy-:fix-20924
Jan 8, 2024
Merged

[fix][broker]Fix NonPersistentDispatcherMultipleConsumers ArrayIndexOutOfBoundsException#21856
Technoboy- merged 1 commit into
apache:masterfrom
Technoboy-:fix-20924

Conversation

@Technoboy-

@Technoboy- Technoboy- commented Jan 5, 2024 •
edited
Loading

Copy link
Copy Markdown
Contributor

Fixes #20924

Motivation

pulsar/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/nonpersistent/NonPersistentDispatcherMultipleConsumers.java

Line 195 in 67eb3c4

Consumer consumer = TOTAL_AVAILABLE_PERMITS_UPDATER.get(this) > 0 ? getNextConsumer() : null;

The root cause is that : when call getNextConsumer (), we don't have any lock, the consumerList has changed after line-120.

pulsar/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/AbstractDispatcherMultipleConsumers.java

Lines 114 to 127 in 67eb3c4

public Consumer getNextConsumer() {
if (consumerList.isEmpty() || IS_CLOSED_UPDATER.get(this) == TRUE) {
// abort read if no consumers are connected or if disconnect is initiated
return null;
}
if (currentConsumerRoundRobinIndex >= consumerList.size()) {
currentConsumerRoundRobinIndex = 0;
}
int currentRoundRobinConsumerPriority = consumerList.get(currentConsumerRoundRobinIndex).getPriorityLevel();
// first find available-consumer on higher level unless currentIndex is not on highest level which is 0
if (currentRoundRobinConsumerPriority != 0) { 

2023-07-28T09:13:34,703+0800 [pulsar-io-12-13] WARN  org.apache.pulsar.broker.service.ServerCnx - [/10.65.17.39:65308] Got exception java.lang.ArrayIndexOutOfBoundsException: Index 2 out of bounds for length 2
	at java.base/java.util.concurrent.CopyOnWriteArrayList.elementAt(CopyOnWriteArrayList.java:385)
	at java.base/java.util.concurrent.CopyOnWriteArrayList.get(CopyOnWriteArrayList.java:398)
	at org.apache.pulsar.broker.service.AbstractDispatcherMultipleConsumers.getNextConsumerFromSameOrLowerLevel(AbstractDispatcherMultipleConsumers.java:192)
	at org.apache.pulsar.broker.service.AbstractDispatcherMultipleConsumers.getNextConsumer(AbstractDispatcherMultipleConsumers.java:137)
	at org.apache.pulsar.broker.service.nonpersistent.NonPersistentDispatcherMultipleConsumers.sendMessages(NonPersistentDispatcherMultipleConsumers.java:188)
	at org.apache.pulsar.broker.service.nonpersistent.NonPersistentTopic.lambda$publishMessage$2(NonPersistentTopic.java:202)
	at org.apache.pulsar.common.util.collections.ConcurrentOpenHashMap$Section.forEach(ConcurrentOpenHashMap.java:554)
	at org.apache.pulsar.common.util.collections.ConcurrentOpenHashMap.forEach(ConcurrentOpenHashMap.java:277)
	at org.apache.pulsar.broker.service.nonpersistent.NonPersistentTopic.publishMessage(NonPersistentTopic.java:196)
	at org.apache.pulsar.broker.service.Producer.publishMessageToTopic(Producer.java:281)
	at org.apache.pulsar.broker.service.Producer.publishMessage(Producer.java:194)
	at org.apache.pulsar.broker.service.ServerCnx.handleSend(ServerCnx.java:1733)
	at org.apache.pulsar.common.protocol.PulsarDecoder.channelRead(PulsarDecoder.java:222)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
	at io.netty.handler.flow.FlowControlHandler.dequeue(FlowControlHandler.java:202)

Documentation

  • doc
  • doc-required
  • doc-not-needed
  • doc-complete

@Technoboy- Technoboy- self-assigned this Jan 5, 2024
@Technoboy- Technoboy- added this to the 3.3.0 milestone Jan 5, 2024
@Technoboy- Technoboy- added ready-to-test doc-not-needed Your PR changes do not impact docs labels Jan 5, 2024
@codecov-commenter

codecov-commenter commented Jan 5, 2024 •
edited
Loading

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 73.59%. Comparing base (1a4a9d9) to head (f33ab8c).
⚠️ Report is 1373 commits behind head on master.

Additional details and impacted files

Impacted file tree graph

@@             Coverage Diff              @@
##             master   #21856      +/-   ##
============================================
- Coverage     73.66%   73.59%   -0.08%     
+ Complexity    32343    32302      -41     
============================================
  Files          1858     1858              
  Lines        138146   138141       -5     
  Branches      15141    15141              
============================================
- Hits         101763   101660     -103     
- Misses        28544    28608      +64     
- Partials       7839     7873      +34
Flag Coverage Δ
inttests 24.15% <ø> (-0.04%) ⬇️
systests 23.69% <ø> (-0.18%) ⬇️
unittests 72.89% <ø> (-0.06%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
...tent/NonPersistentDispatcherMultipleConsumers.java 68.67% <ø> (ø)

... and 86 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@Technoboy- Technoboy- modified the milestones: 3.3.0, 3.2.0 Jan 5, 2024
@Technoboy- Technoboy- merged commit cea5c93 into apache:master Jan 8, 2024
Technoboy- added a commit that referenced this pull request Jan 8, 2024
@Technoboy-
[fix][broker]Fix NonPersistentDispatcherMultipleConsumers ArrayIndexO…
f7b768d 
…utOfBoundsException (#21856)
Technoboy- added a commit that referenced this pull request Jan 16, 2024
@Technoboy-
[fix][broker]Fix NonPersistentDispatcherMultipleConsumers ArrayIndexO…
434b2c3 
…utOfBoundsException (#21856)
Technoboy- added a commit that referenced this pull request Jan 16, 2024
@Technoboy-
[fix][broker]Fix NonPersistentDispatcherMultipleConsumers ArrayIndexO…
12f9e43 
…utOfBoundsException (#21856)
Technoboy- added a commit that referenced this pull request Jan 22, 2024
@Technoboy-
[fix][broker]Fix NonPersistentDispatcherMultipleConsumers ArrayIndexO…
5af9cdb 
…utOfBoundsException (#21856)
Technoboy- added a commit that referenced this pull request Jan 22, 2024
@Technoboy-
[fix][broker]Fix NonPersistentDispatcherMultipleConsumers ArrayIndexO…
5931c05 
…utOfBoundsException (#21856)
nodece pushed a commit to nodece/pulsar that referenced this pull request Feb 23, 2024
@Technoboy- @nodece
[fix][broker]Fix NonPersistentDispatcherMultipleConsumers ArrayIndexO…
dace72c 
…utOfBoundsException (apache#21856)
mukesh-ctds pushed a commit to datastax/pulsar that referenced this pull request Mar 1, 2024
@Technoboy- @mukesh-ctds
[fix][broker]Fix NonPersistentDispatcherMultipleConsumers ArrayIndexO…
a96b685 
…utOfBoundsException (apache#21856)

(cherry picked from commit 434b2c3)
mukesh-ctds pushed a commit to datastax/pulsar that referenced this pull request Mar 6, 2024
@Technoboy- @mukesh-ctds
[fix][broker]Fix NonPersistentDispatcherMultipleConsumers ArrayIndexO…
1e28b2e 
…utOfBoundsException (apache#21856)

(cherry picked from commit 434b2c3)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@AnonHxy AnonHxy AnonHxy left review comments

@codelipenghui codelipenghui codelipenghui approved these changes

@RobertIndie RobertIndie RobertIndie approved these changes

@poorbarcode poorbarcode poorbarcode approved these changes

Assignees

@Technoboy- Technoboy-

Labels

cherry-picked/branch-2.10 cherry-picked/branch-2.11 cherry-picked/branch-3.0 cherry-picked/branch-3.1 doc-not-needed Your PR changes do not impact docs ready-to-test release/2.10.6 release/2.11.4 release/3.0.3 release/3.1.3

Projects

None yet

Milestone

3.2.0

Development

Successfully merging this pull request may close these issues.

[Bug] when the broker processes non-persistent Consumers in a multi-threaded ，Got exception: ArrayIndexOutOfBoundsException

6 participants

@Technoboy- @codecov-commenter @AnonHxy @codelipenghui @RobertIndie @poorbarcode