[fix][admin] Backlog quota's policy is null which causes a NPE #24192

poorbarcode · 2025-04-21T15:35:34Z

Motivation

pulsar-admin namespaces set-backlog-quota <namespace> does not allow a null attribute policy, but HTTP API POST /admin/v2/namespaces/{namespace}/backlogQuota allows.
pulsar-admin topicPolicies set-backlog-quota <namespace> does not allow a null attribute policy, but HTTP API POST /admin/v2/namespaces/{topic}/backlogQuota allows.

It leads a NPE

2025-04-21T03:04:02,254+0000 [pulsar-backlog-quota-checker-OrderedScheduler-0-0] INFO  org.apache.pulsar.broker.service.BacklogQuotaManager - Backlog quota type destination_storage exceeded for topic [persistent://public/default/internal-partition-1]. Applying [null] policy
2025-04-21T03:04:02,254+0000 [pulsar-backlog-quota-checker-OrderedScheduler-0-0] ERROR org.apache.pulsar.broker.service.BrokerService - Error when checkBacklogQuota(persistent://public/default/internal-partition-1) in monitorBacklogQuota
java.util.concurrent.CompletionException: java.lang.NullPointerException: Cannot invoke "org.apache.pulsar.common.policies.data.BacklogQuota$RetentionPolicy.ordinal()" because the return value of "org.apache.pulsar.common.policies.data.BacklogQuota.getPolicy()" is null
        at java.base/java.util.concurrent.CompletableFuture.encodeThrowable(Unknown Source) ~[?:?]
        at java.base/java.util.concurrent.CompletableFuture.uniAcceptNow(Unknown Source) ~[?:?]
        at java.base/java.util.concurrent.CompletableFuture.uniAcceptStage(Unknown Source) ~[?:?]
        at java.base/java.util.concurrent.CompletableFuture.thenAccept(Unknown Source) ~[?:?]
        at org.apache.pulsar.broker.service.BrokerService.lambda$monitorBacklogQuota$104(BrokerService.java:2221) ~[io.streamnative-pulsar-broker-3.3.5.jar:3.3.5]
        at org.apache.pulsar.broker.service.BrokerService.lambda$forEachPersistentTopic$99(BrokerService.java:2211) ~[io.streamnative-pulsar-broker-3.3.5.jar:3.3.5]
        at java.base/java.util.Optional.ifPresent(Unknown Source) ~[?:?]
        at org.apache.pulsar.broker.service.BrokerService.lambda$forEachPersistentTopic$100(BrokerService.java:2211) ~[io.streamnative-pulsar-broker-3.3.5.jar:3.3.5]
        at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(Unknown Source) ~[?:?]
        at java.base/java.util.stream.ReferencePipeline$3$1.accept(Unknown Source) ~[?:?]
        at java.base/java.util.stream.ReferencePipeline$3$1.accept(Unknown Source) ~[?:?]
        at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(Unknown Source) ~[?:?]
        at java.base/java.util.stream.AbstractPipeline.copyInto(Unknown Source) ~[?:?]
        at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(Unknown Source) ~[?:?]
        at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(Unknown Source) ~[?:?]
        at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(Unknown Source) ~[?:?]
        at java.base/java.util.stream.AbstractPipeline.evaluate(Unknown Source) ~[?:?]
        at java.base/java.util.stream.ReferencePipeline.forEach(Unknown Source) ~[?:?]
        at org.apache.pulsar.broker.service.BrokerService.forEachPersistentTopic(BrokerService.java:2211) ~[io.streamnative-pulsar-broker-3.3.5.jar:3.3.5]
        at org.apache.pulsar.broker.service.BrokerService.monitorBacklogQuota(BrokerService.java:2220) ~[io.streamnative-pulsar-broker-3.3.5.jar:3.3.5]
        at com.google.common.util.concurrent.MoreExecutors$ScheduledListeningDecorator$NeverSuccessfulListenableFutureTask.run(MoreExecutors.java:741) ~[com.google.guava-guava-32.1.2-jre.jar:?]
        at org.apache.bookkeeper.common.util.SingleThreadSafeScheduledExecutorService$SafeRunnable.run(SingleThreadSafeScheduledExecutorService.java:45) ~[io.streamnative-bookkeeper-common-4.17.1.2.jar:4.17.1.2]
        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ~[?:?]
        at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) ~[?:?]
        at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) ~[?:?]
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) ~[?:?]
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ~[?:?]
        at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[io.netty-netty-common-4.1.119.Final.jar:4.1.119.Final]
        at java.base/java.lang.Thread.run(Unknown Source) [?:?]
Caused by: java.lang.NullPointerException: Cannot invoke "org.apache.pulsar.common.policies.data.BacklogQuota$RetentionPolicy.ordinal()" because the return value of "org.apache.pulsar.common.policies.data.BacklogQuota.getPolicy()" is null
        at org.apache.pulsar.broker.service.BacklogQuotaManager.handleExceededBacklogQuota(BacklogQuotaManager.java:93) ~[io.streamnative-pulsar-broker-3.3.5.jar:3.3.5]
        at org.apache.pulsar.broker.service.BrokerService.lambda$monitorBacklogQuota$102(BrokerService.java:2223) ~[io.streamnative-pulsar-broker-3.3.5.jar:3.3.5]
        ... 28 more
2025-04-21T03:04:02,256+0000 [pulsar-backlog-quota-checker-OrderedScheduler-0-0] INFO  org.apache.pulsar.broker.service.BacklogQuotaManager - Backlog quota type destination_storage exceeded for topic [persistent://public/default/output-partition-1]. Applying [null] policy

Modifications

Let HTTP API does not allow the null policy as well

Documentation

doc
doc-required
doc-not-needed
doc-complete

Matching PR in forked repository

PR in forked repository: x

poorbarcode · 2025-04-21T15:54:42Z

/pulsarbot rerun-failure-checks

Technoboy- · 2025-04-22T02:14:21Z

Pulsar CI / Build Pulsar on MacOS (pull_request)

checkstyle failed

lhotari

The test org.apache.pulsar.broker.admin.AdminApiTest.testRetentionAndBacklogQuotaCheck fails.

pulsar-broker/src/test/java/org/apache/pulsar/broker/admin/TopicPoliciesTest.java

lhotari

LGTM

codecov-commenter · 2025-04-25T07:42:31Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 74.23%. Comparing base (bbc6224) to head (d4559a3).
Report is 1047 commits behind head on master.

Additional details and impacted files

@@             Coverage Diff              @@
##             master   #24192      +/-   ##
============================================
+ Coverage     73.57%   74.23%   +0.65%     
+ Complexity    32624    32178     -446     
============================================
  Files          1877     1866      -11     
  Lines        139502   144750    +5248     
  Branches      15299    16534    +1235     
============================================
+ Hits         102638   107448    +4810     
+ Misses        28908    28811      -97     
- Partials       7956     8491     +535

Flag	Coverage Δ
inttests	`26.71% <0.00%> (+2.12%)`	⬆️
systests	`23.28% <0.00%> (-1.04%)`	⬇️
unittests	`73.73% <100.00%> (+0.88%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
.../org/apache/pulsar/broker/admin/AdminResource.java	`78.17% <100.00%> (+0.54%)`	⬆️
...pache/pulsar/broker/admin/impl/NamespacesBase.java	`76.09% <100.00%> (+3.01%)`	⬆️
...pulsar/broker/admin/impl/PersistentTopicsBase.java	`70.14% <100.00%> (+4.69%)`	⬆️
...ache/pulsar/common/policies/data/BacklogQuota.java	`100.00% <100.00%> (ø)`

... and 1078 files with indirect coverage changes

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

(cherry picked from commit cb018d9)

…e#24192) (cherry picked from commit cb018d9) (cherry picked from commit d1bb49c)

…e#24192) (cherry picked from commit cb018d9) (cherry picked from commit dead8cc)

…e#24192) (cherry picked from commit cb018d9)

…e#24192)

poorbarcode added type/bug The PR fixed a bug or issue reported a bug release/3.0.12 release/3.3.7 release/4.0.5 labels Apr 21, 2025

poorbarcode added this to the 4.1.0 milestone Apr 21, 2025

poorbarcode requested a review from zymap April 21, 2025 15:35

poorbarcode self-assigned this Apr 21, 2025

github-actions bot added the doc-not-needed Your PR changes do not impact docs label Apr 21, 2025

gaoran10 approved these changes Apr 21, 2025

View reviewed changes

gaoran10 added the ready-to-test label Apr 21, 2025

Technoboy- closed this Apr 22, 2025

Technoboy- reopened this Apr 22, 2025

Technoboy- approved these changes Apr 22, 2025

View reviewed changes

zymap approved these changes Apr 22, 2025

View reviewed changes

lhotari reviewed Apr 22, 2025

View reviewed changes

pulsar-broker/src/test/java/org/apache/pulsar/broker/admin/TopicPoliciesTest.java Show resolved Hide resolved

dao-jun approved these changes Apr 24, 2025

View reviewed changes

poorbarcode added 5 commits April 24, 2025 15:29

[fix][admin]Backlog quota's policy is null which causes a NPE

18c7b5f

checkstyle

ed81d76

fix tests

30868c2

fix tests

e981000

fix tests and deal comments

0c25c11

poorbarcode force-pushed the fix/admin_npe branch from 043a58c to 0c25c11 Compare April 24, 2025 07:35

poorbarcode requested a review from lhotari April 24, 2025 07:35

lhotari approved these changes Apr 24, 2025

View reviewed changes

lhotari changed the title ~~[fix][admin]Backlog quota's policy is null which causes a NPE~~ [fix][admin] Backlog quota's policy is null which causes a NPE Apr 24, 2025

fix tests

d4559a3

lhotari merged commit cb018d9 into apache:master Apr 25, 2025
52 checks passed

lhotari pushed a commit that referenced this pull request Apr 25, 2025

[fix][admin] Backlog quota's policy is null which causes a NPE (#24192)

dead8cc

(cherry picked from commit cb018d9)

lhotari pushed a commit that referenced this pull request Apr 25, 2025

[fix][admin] Backlog quota's policy is null which causes a NPE (#24192)

d1bb49c

(cherry picked from commit cb018d9)

lhotari pushed a commit that referenced this pull request Apr 25, 2025

[fix][admin] Backlog quota's policy is null which causes a NPE (#24192)

80bce82

(cherry picked from commit cb018d9)

lhotari added cherry-picked/branch-4.0 cherry-picked/branch-3.3 cherry-picked/branch-3.0 labels Apr 25, 2025

manas-ctds pushed a commit to datastax/pulsar that referenced this pull request Apr 30, 2025

[fix][admin] Backlog quota's policy is null which causes a NPE (apach…

04b9903

…e#24192) (cherry picked from commit cb018d9) (cherry picked from commit d1bb49c)

manas-ctds pushed a commit to datastax/pulsar that referenced this pull request Apr 30, 2025

[fix][admin] Backlog quota's policy is null which causes a NPE (apach…

9063896

…e#24192) (cherry picked from commit cb018d9) (cherry picked from commit d1bb49c)

ganesh-ctds pushed a commit to datastax/pulsar that referenced this pull request Apr 30, 2025

[fix][admin] Backlog quota's policy is null which causes a NPE (apach…

d023db9

…e#24192) (cherry picked from commit cb018d9) (cherry picked from commit d1bb49c)

ganesh-ctds pushed a commit to datastax/pulsar that referenced this pull request May 1, 2025

[fix][admin] Backlog quota's policy is null which causes a NPE (apach…

41a331a

…e#24192) (cherry picked from commit cb018d9) (cherry picked from commit d1bb49c)

ganesh-ctds pushed a commit to datastax/pulsar that referenced this pull request May 1, 2025

[fix][admin] Backlog quota's policy is null which causes a NPE (apach…

ba94615

…e#24192) (cherry picked from commit cb018d9) (cherry picked from commit d1bb49c)

ganesh-ctds pushed a commit to datastax/pulsar that referenced this pull request May 1, 2025

[fix][admin] Backlog quota's policy is null which causes a NPE (apach…

07e1d41

…e#24192) (cherry picked from commit cb018d9) (cherry picked from commit d1bb49c)

manas-ctds pushed a commit to datastax/pulsar that referenced this pull request May 2, 2025

[fix][admin] Backlog quota's policy is null which causes a NPE (apach…

52325f6

…e#24192) (cherry picked from commit cb018d9) (cherry picked from commit d1bb49c)

manas-ctds pushed a commit to datastax/pulsar that referenced this pull request May 2, 2025

[fix][admin] Backlog quota's policy is null which causes a NPE (apach…

13c8055

…e#24192) (cherry picked from commit cb018d9) (cherry picked from commit dead8cc)

srinath-ctds pushed a commit to datastax/pulsar that referenced this pull request May 6, 2025

[fix][admin] Backlog quota's policy is null which causes a NPE (apach…

1a6b37c

…e#24192) (cherry picked from commit cb018d9) (cherry picked from commit dead8cc)

nodece pushed a commit to ascentstream/pulsar that referenced this pull request May 28, 2025

[fix][admin] Backlog quota's policy is null which causes a NPE (apach…

80a49d9

…e#24192) (cherry picked from commit cb018d9)

KannarFr pushed a commit to CleverCloud/pulsar that referenced this pull request Sep 22, 2025

[fix][admin] Backlog quota's policy is null which causes a NPE (apach…

e1a0684

…e#24192)

walkinggo pushed a commit to walkinggo/pulsar that referenced this pull request Oct 8, 2025

[fix][admin] Backlog quota's policy is null which causes a NPE (apach…

39e7e0a

…e#24192)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[fix][admin] Backlog quota's policy is null which causes a NPE #24192

[fix][admin] Backlog quota's policy is null which causes a NPE #24192

Uh oh!

poorbarcode commented Apr 21, 2025

Uh oh!

poorbarcode commented Apr 21, 2025

Uh oh!

Technoboy- commented Apr 22, 2025

Uh oh!

lhotari left a comment

Uh oh!

Uh oh!

lhotari left a comment

Uh oh!

codecov-commenter commented Apr 25, 2025 •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

[fix][admin] Backlog quota's policy is null which causes a NPE #24192

[fix][admin] Backlog quota's policy is null which causes a NPE #24192

Uh oh!

Conversation

poorbarcode commented Apr 21, 2025

Motivation

Modifications

Documentation

Matching PR in forked repository

Uh oh!

poorbarcode commented Apr 21, 2025

Uh oh!

Technoboy- commented Apr 22, 2025

Uh oh!

lhotari left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

lhotari left a comment

Choose a reason for hiding this comment

Uh oh!

codecov-commenter commented Apr 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

codecov-commenter commented Apr 25, 2025 •

edited

Loading