Skip to content

Close the previous reader of the health check topic#7724

Merged
codelipenghui merged 1 commit into
apache:masterfrom
codelipenghui:penghui/close_previous_reader_for_healthcheck
Aug 3, 2020
Merged

Close the previous reader of the health check topic#7724
codelipenghui merged 1 commit into
apache:masterfrom
codelipenghui:penghui/close_previous_reader_for_healthcheck

Conversation

@codelipenghui

Copy link
Copy Markdown
Contributor

Modifications

Close the previous reader of the health check topic. Not sure what caused the reader to not close properly, this PR tries to delete the previous reader of the health check to make sure the previous readers can be closed.

If the previous reader can't be closed, the backlog of the health check topic will keep increasing. According to the broker heap dump, there is more than one reader can't be closed.

image

This PR is related to #7589

Verifying this change

The problem is hard to reproduce.

Does this pull request potentially affect one of the following parts:

If yes was chosen, please highlight the changes

  • Dependencies (does it add or upgrade a dependency): (no)
  • The public API: (no)
  • The schema: (no)
  • The default values of configurations: (no)
  • The wire protocol: (no)
  • The rest endpoints: (no)
  • The admin cli options: (no)
  • Anything that affects deployment: (no)

Documentation

  • Does this pull request introduce a new feature? (no)

@codelipenghui codelipenghui self-assigned this Aug 3, 2020
@codelipenghui codelipenghui added the type/bug The PR fixed a bug or issue reported a bug label Aug 3, 2020
@codelipenghui codelipenghui added this to the 2.7.0 milestone Aug 3, 2020
@codelipenghui codelipenghui merged commit ca98a89 into apache:master Aug 3, 2020
@codelipenghui codelipenghui deleted the penghui/close_previous_reader_for_healthcheck branch August 3, 2020 11:02
@jiazhai

jiazhai commented Aug 4, 2020

Copy link
Copy Markdown
Member

@wolfstudy to help cherry-pick to both branch-2.5 and branch-2.6

wolfstudy pushed a commit that referenced this pull request Aug 4, 2020
@wolfstudy

Copy link
Copy Markdown
Member

@wolfstudy to help cherry-pick to both branch-2.5 and branch-2.6

Sure, already cherry-pick the pull request to branch-2.5 and branch-2.6

wolfstudy pushed a commit that referenced this pull request Aug 4, 2020
huangdx0726 pushed a commit to huangdx0726/pulsar that referenced this pull request Aug 24, 2020
lbenc135 pushed a commit to lbenc135/pulsar that referenced this pull request Sep 5, 2020
lbenc135 pushed a commit to lbenc135/pulsar that referenced this pull request Sep 5, 2020
lbenc135 pushed a commit to lbenc135/pulsar that referenced this pull request Sep 5, 2020
codelipenghui pushed a commit that referenced this pull request Mar 14, 2022
…#14658)

Original PR #14367

Because the PR #14367 is based on PR #14091, so I want to cherry-pick these two PRs to branch-2.9, the PR #13525 is also needed.

---

Fix Issue: #14362

### Motivation

According to relative PR #7724, we will force delete all subscriptions when calling ``healthCheck`` REST API. but it has a race condition when two threads call this API.
Please consider this case:

> Thread A: Clean up all subscriptions, then create a reader.
> Thread B: Prepare to force delete all subscriptions.

So, in this case, the reader of thread A is deleted and then an NPE or other exception occurs.

### Modifications

- Use ``Completable#handle`` to fix problem 1, the reader needs to be closed regardless of whether there is an exception.
- Recheck the subscription after closing reading, and force deletion if it still exists after closing reading.
- Added multi-threaded tests for health checks.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

release/2.5.3 release/2.6.1 type/bug The PR fixed a bug or issue reported a bug

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants