Avoid prefetch too much data when offloading data to HDFS #6717
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Contributor
Author
|
@congbobo184 is it possible for you to review this PR? |
congbobo184
reviewed
Apr 11, 2020
.../org/apache/bookkeeper/mledger/offload/filesystem/impl/FileSystemManagedLedgerOffloader.java
Outdated
Show resolved
Hide resolved
pheecian
commented
Apr 11, 2020
.../org/apache/bookkeeper/mledger/offload/filesystem/impl/FileSystemManagedLedgerOffloader.java
Show resolved
Hide resolved
added 2 commits
April 12, 2020 15:42
Contributor
Author
|
@congbobo184 Is it possible for you to review the PR again? |
congbobo184
reviewed
Apr 12, 2020
.../org/apache/bookkeeper/mledger/offload/filesystem/impl/FileSystemManagedLedgerOffloader.java
Outdated
Show resolved
Hide resolved
.../org/apache/bookkeeper/mledger/offload/filesystem/impl/FileSystemManagedLedgerOffloader.java
Outdated
Show resolved
Hide resolved
added 2 commits
April 12, 2020 22:45
Contributor
Author
|
@congbobo184 your comment makes sense. I have update the code. Thanks. Any more modification for me? |
congbobo184
approved these changes
Apr 13, 2020
Contributor
|
/pulsarbot run-failure-checks |
Contributor
Author
|
@congbobo184 need I handle the CI-Integration test fail? I think it is not related to this PR |
Contributor
|
/pulsarbot run-failure-checks |
Contributor
|
you don't need to handle the CI-Integration. |
codelipenghui
approved these changes
Apr 13, 2020
Contributor
|
/pulsarbot run-failure-checks |
Contributor
Contributor
|
/pulsarbot run-failure-checks |
sijie
reviewed
Apr 14, 2020
.../org/apache/bookkeeper/mledger/offload/filesystem/impl/FileSystemManagedLedgerOffloader.java
Outdated
Show resolved
Hide resolved
Contributor
Author
sijie
approved these changes
Apr 21, 2020
Contributor
|
Nice work! 👍 @pheecian |
addisonj
pushed a commit
to instructure/pulsar
that referenced
this pull request
May 7, 2020
Fixes apache#6692 avoid prefetch too much data when offloading, which may lead to OOM; fix object not close issue, which is also mentioned by congbobo184 apache#6697 *Explain here the context, and why you're making that change. What is the problem you're trying to solve.* *If `yes` was chosen, please highlight the changes* - Dependencies (does it add or upgrade a dependency): (no) - The public API: (no) - The schema: (no) - The default values of configurations: (no) - The wire protocol: (no) - The rest endpoints: (no) - The admin cli options: (no) - Anything that affects deployment: (no) - Does this pull request introduce a new feature? (no)
This was referenced May 8, 2020
jiazhai
pushed a commit
that referenced
this pull request
May 8, 2020
Fixes #6692 ### Motivation avoid prefetch too much data when offloading, which may lead to OOM; fix object not close issue, which is also mentioned by congbobo184 #6697 *Explain here the context, and why you're making that change. What is the problem you're trying to solve.* ### Does this pull request potentially affect one of the following parts: *If `yes` was chosen, please highlight the changes* - Dependencies (does it add or upgrade a dependency): (no) - The public API: (no) - The schema: (no) - The default values of configurations: (no) - The wire protocol: (no) - The rest endpoints: (no) - The admin cli options: (no) - Anything that affects deployment: (no) ### Documentation - Does this pull request introduce a new feature? (no) (cherry picked from commit 514b6af)
cdbartholomew
added a commit
to kafkaesque-io/pulsar
that referenced
this pull request
May 25, 2020
…ache#6717)" This reverts commit 74c668a.
huangdx0726
pushed a commit
to huangdx0726/pulsar
that referenced
this pull request
Aug 24, 2020
Fixes apache#6692 ### Motivation avoid prefetch too much data when offloading, which may lead to OOM; fix object not close issue, which is also mentioned by congbobo184 apache#6697 *Explain here the context, and why you're making that change. What is the problem you're trying to solve.* ### Does this pull request potentially affect one of the following parts: *If `yes` was chosen, please highlight the changes* - Dependencies (does it add or upgrade a dependency): (no) - The public API: (no) - The schema: (no) - The default values of configurations: (no) - The wire protocol: (no) - The rest endpoints: (no) - The admin cli options: (no) - Anything that affects deployment: (no) ### Documentation - Does this pull request introduce a new feature? (no)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes #6692
Motivation
avoid prefetch too much data when offloading, which may lead to OOM;
fix object not close issue, which is also mentioned by congbobo184 #6697
Explain here the context, and why you're making that change. What is the problem you're trying to solve.
Does this pull request potentially affect one of the following parts:
If
yeswas chosen, please highlight the changesDocumentation