Skip to content

[fix][broker] Reject create non-existent persistent partitions#19086

Merged
eolivelli merged 20 commits into
masterfrom
reject_create_non_existent_partitions
Jan 9, 2023
Merged

[fix][broker] Reject create non-existent persistent partitions#19086
eolivelli merged 20 commits into
masterfrom
reject_create_non_existent_partitions

Conversation

@mattisonchao

@mattisonchao mattisonchao commented Dec 28, 2022

Copy link
Copy Markdown
Member

Fixes #19085

Master Issue: #19085

Motivation

See #19085

ML: https://lists.apache.org/thread/3s7b7dh0h9qcnn8x3lclxmw75okkqf37

Modifications

  • Add check to avoid create non-existent persistent partitions.

Verifying this change

  • Make sure that the change passes the CI checks.

Documentation

  • doc
  • doc-required
  • doc-not-needed
  • doc-complete

@github-actions github-actions Bot added the doc-not-needed Your PR changes do not impact docs label Dec 28, 2022
@mattisonchao mattisonchao self-assigned this Dec 28, 2022
@mattisonchao mattisonchao added this to the 2.12.0 milestone Dec 28, 2022
Comment thread pulsar-broker/src/main/java/org/apache/pulsar/broker/service/BrokerService.java Outdated

@liangyepianzhou liangyepianzhou left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

return pulsar.getPulsarResources().getNamespaceResources().getPartitionedTopicResources()
.getPartitionedTopicMetadataAsync(topicName).thenApply(metadata -> {
// if the partitioned topic is not found in metadata, then the topic is not partitioned
return metadata.orElseGet(() -> new PartitionedTopicMetadata());

Maybe we should change this to
return metadata.orElseGet(() -> new PartitionedTopicMetadata(-1));

@congbobo184 congbobo184 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

without this change, the test also can pass @mattisonchao

return CompletableFuture.completedFuture(Optional.empty());

this line has already guaranteed can't create partition-4 topic.

@codecov-commenter

codecov-commenter commented Dec 28, 2022

Copy link
Copy Markdown

Codecov Report

Merging #19086 (58f516f) into master (492a9c3) will decrease coverage by 1.18%.
The diff coverage is 47.78%.

Impacted file tree graph

@@             Coverage Diff              @@
##             master   #19086      +/-   ##
============================================
- Coverage     47.46%   46.27%   -1.19%     
+ Complexity    10727     9318    -1409     
============================================
  Files           711      633      -78     
  Lines         69456    59871    -9585     
  Branches       7452     6238    -1214     
============================================
- Hits          32964    27706    -5258     
+ Misses        32810    29111    -3699     
+ Partials       3682     3054     -628     
Flag Coverage Δ
unittests 46.27% <47.78%> (-1.19%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
...va/org/apache/pulsar/client/impl/ConsumerImpl.java 15.06% <0.00%> (-0.04%) ⬇️
...pulsar/broker/admin/impl/PersistentTopicsBase.java 58.61% <46.15%> (-0.84%) ⬇️
...rg/apache/pulsar/broker/service/BrokerService.java 56.43% <100.00%> (-0.48%) ⬇️
...lsar/client/impl/conf/ClientConfigurationData.java 96.66% <100.00%> (ø)
...sar/broker/stats/metrics/ManagedLedgerMetrics.java 23.88% <0.00%> (-76.12%) ⬇️
...ice/streamingdispatch/PendingReadEntryRequest.java 0.00% <0.00%> (-68.19%) ⬇️
...ervice/streamingdispatch/StreamingEntryReader.java 0.00% <0.00%> (-59.65%) ⬇️
...istentStreamingDispatcherSingleActiveConsumer.java 0.00% <0.00%> (-48.46%) ⬇️
...ersistentStreamingDispatcherMultipleConsumers.java 0.00% <0.00%> (-45.55%) ⬇️
...ar/broker/loadbalance/impl/BundleSplitterTask.java 54.00% <0.00%> (-28.00%) ⬇️
... and 139 more

@mattisonchao

mattisonchao commented Dec 28, 2022

Copy link
Copy Markdown
Member Author

without this change, the test also can pass @mattisonchao

return CompletableFuture.completedFuture(Optional.empty());

this line has already guaranteed can't create partition-4 topic.

@congbobo184 Aha, sorry. I need to change the topic name to persistent. please take another look. :)

@mattisonchao mattisonchao added the release/important-notice The changes which are important should be mentioned in the release note label Dec 30, 2022
@mattisonchao

Copy link
Copy Markdown
Member Author

I use the workaround method to fix the restful endpoint updatePartitionedTopic. It's ugly though. I will refactor it in the next PR.

@eolivelli @codelipenghui @liangyepianzhou Could you review it again?

@eolivelli eolivelli left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@eolivelli eolivelli merged commit 56a7b89 into master Jan 9, 2023
@mattisonchao mattisonchao deleted the reject_create_non_existent_partitions branch January 9, 2023 12:44
@momo-jun momo-jun changed the title [fix][broker] Reject create non existent persistent partitions. [fix][broker] Reject create non-existent persistent partitions Apr 28, 2023
if (topicName.isPartitioned()) {
return fetchPartitionedTopicMetadataAsync(TopicName.get(topicName.getPartitionedTopicName()))
.thenCompose((metadata) -> {
// Allow crate non-partitioned persistent topic that name includes `partition`

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mattisonchao There's a spelling problem here. crate -> create

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

doc-not-needed Your PR changes do not impact docs ready-to-test release/important-notice The changes which are important should be mentioned in the release note

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] Unreasonable partitioned topic creation.

9 participants