Skip to content

[fix][broker] Fix Replicated Topic unload bug when ExtensibleLoadManager is enabled #22496

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
heesung-sohn merged 2 commits into from
Apr 16, 2024
Merged

[fix][broker] Fix Replicated Topic unload bug when ExtensibleLoadManager is enabled #22496

heesung-sohn merged 2 commits into from
Apr 16, 2024

Conversation

@heesung-sohn
Copy link
Contributor

@heesung-sohn heesung-sohn commented Apr 12, 2024 •
edited
Loading

Motivation

Noticed that Replicated Topic is not unloaded upon setNamespaceReplicationClusters when ExtensibleLoadManager is enabled.

Modifications

  • Fix unloading bundles upon namespace policy update. (isNamespaceBundleOwned didn't check the bundle ownership properly when ExtensibleLoadManager is enabled.)
  • Do not replicate ExtensibleLoadManager internal topics (bundle state, broker and bundle load data topics) because they are only used for the local cluster.
  • Copy ReplicatorGlobalNSTest for ExtensibleLoadManager.

Verifying this change

  • Make sure that the change passes the CI checks.

Does this pull request potentially affect one of the following parts:

If the box was checked, please highlight the changes

  • Dependencies (add or upgrade a dependency)
  • The public API
  • The schema
  • The default values of configurations
  • The threading model
  • The binary protocol
  • The REST endpoints
  • The admin CLI options
  • The metrics
  • Anything that affects deployment

Documentation

  • doc
  • doc-required
  • doc-not-needed
  • doc-complete

Matching PR in forked repository

PR in forked repository:

@github-actions
Copy link

github-actions bot commented Apr 12, 2024

@heesung-sn Please add the following content to your PR description and select a checkbox:

- [ ] `doc` <!-- Your PR contains doc changes -->
- [ ] `doc-required` <!-- Your PR changes impact docs and you will update later -->
- [ ] `doc-not-needed` <!-- Your PR changes do not impact docs -->
- [ ] `doc-complete` <!-- Docs have been already added -->

@heesung-sohn heesung-sohn self-assigned this Apr 12, 2024
@github-actions github-actions bot added doc-not-needed Your PR changes do not impact docs and removed doc-label-missing labels Apr 12, 2024
@merlimat merlimat added this to the 3.3.0 milestone Apr 13, 2024
Demogorgon314
Demogorgon314 reviewed Apr 16, 2024
View reviewed changes
Copy link
Member

@Demogorgon314 Demogorgon314 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we also check the other place such as producer.close(), looks like it may also get stuck forever.

@heesung-sohn
Copy link
Contributor Author

heesung-sohn commented Apr 16, 2024

Can we also check the other place such as producer.close(), looks like it may also get stuck forever.

I understand the producer.flush might get stuck if no leader broker is found in the cluster, but actually, I wonder how producer.close() can get stuck.

@Demogorgon314
Copy link
Member

Demogorgon314 commented Apr 16, 2024

I understand the producer.flush might get stuck if no leader broker is found in the cluster, but actually, I wonder how producer.close() can get stuck.

Oh, I see it has operationTimeoutMs set to 30000, so it won't be stuck.

@heesung-sohn heesung-sohn merged commit 203f305 into apache:master Apr 16, 2024
lhotari
lhotari reviewed Apr 25, 2024
View reviewed changes
pulsar-broker/src/test/java/org/apache/pulsar/broker/service/ReplicatorGlobalNSTest.java
public static Object[][] loadManagerClassName() {
return new Object[][]{
{ModularLoadManagerImpl.class.getName()},
{ExtensibleLoadManagerImpl.class.getName()}
Copy link
Member

@lhotari lhotari Apr 25, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@heesung-sn I think that this is triggering a problem in tests.

Copy link
Contributor Author

@heesung-sohn heesung-sohn Apr 25, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure. We can comment out this line if this is problematic.

@lhotari lhotari mentioned this pull request Apr 25, 2024
Closed
3 tasks
heesung-sohn added a commit to heesung-sohn/pulsar that referenced this pull request Jun 26, 2024
@heesung-sohn
[fix][broker] Fix Replicated Topic unload bug when ExtensibleLoadMana…
15db04d 
…ger is enabled (apache#22496)

(cherry picked from commit 203f305)
heesung-sohn added a commit to heesung-sohn/pulsar that referenced this pull request Jun 26, 2024
@heesung-sohn
[fix][broker] Fix Replicated Topic unload bug when ExtensibleLoadMana…
f467f37 
…ger is enabled (apache#22496)

(cherry picked from commit 203f305)
heesung-sohn added a commit to heesung-sohn/pulsar that referenced this pull request Jun 26, 2024
@heesung-sohn
[fix][broker] Fix Replicated Topic unload bug when ExtensibleLoadMana…
42440d8 
…ger is enabled (apache#22496)

(cherry picked from commit 203f305)
(cherry picked from commit f467f37)
heesung-sohn added a commit to heesung-sohn/pulsar that referenced this pull request Jun 26, 2024
@heesung-sohn
[fix][broker] Fix Replicated Topic unload bug when ExtensibleLoadMana…
c357c29 
…ger is enabled (apache#22496)

(cherry picked from commit 203f305)
heesung-sohn added a commit to heesung-sohn/pulsar that referenced this pull request Jun 26, 2024
@heesung-sohn
[fix][broker] Fix Replicated Topic unload bug when ExtensibleLoadMana…
431cbaa 
…ger is enabled (apache#22496)

(cherry picked from commit 203f305)
(cherry picked from commit f467f37)
heesung-sohn added a commit to heesung-sohn/pulsar that referenced this pull request Jun 27, 2024
@heesung-sohn
[fix][broker] Fix Replicated Topic unload bug when ExtensibleLoadMana…
2557db6 
…ger is enabled (apache#22496)

(cherry picked from commit 203f305)
(cherry picked from commit f467f37)
nikhil-ctds pushed a commit to datastax/pulsar that referenced this pull request Jun 27, 2024
@heesung-sohn @nikhil-ctds
[fix][broker] Fix Replicated Topic unload bug when ExtensibleLoadMana…
730ce56 
…ger is enabled (apache#22496)

(cherry picked from commit 203f305)
(cherry picked from commit f467f37)
(cherry picked from commit 2557db6)
srinath-ctds pushed a commit to datastax/pulsar that referenced this pull request Jul 1, 2024
@heesung-sohn @srinath-ctds
[fix][broker] Fix Replicated Topic unload bug when ExtensibleLoadMana…
2049c72 
…ger is enabled (apache#22496)

(cherry picked from commit 203f305)
(cherry picked from commit f467f37)
(cherry picked from commit 2557db6)
hanmz pushed a commit to hanmz/pulsar that referenced this pull request Feb 12, 2025
@heesung-sohn @hanmz
[fix][broker] Fix Replicated Topic unload bug when ExtensibleLoadMana…
843f522 
…ger is enabled (apache#22496)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@lhotari lhotari lhotari left review comments

@Technoboy- Technoboy- Technoboy- approved these changes

@Demogorgon314 Demogorgon314 Demogorgon314 approved these changes

Assignees

@heesung-sohn heesung-sohn

Labels

cherry-picked/branch-3.0 cherry-picked/branch-3.2 doc-not-needed Your PR changes do not impact docs ready-to-test release/3.0.6 release/3.2.4

Projects

None yet

Milestone

3.3.0

Development

Successfully merging this pull request may close these issues.

5 participants

@heesung-sohn @Demogorgon314 @lhotari @Technoboy- @merlimat