Skip to content

[fix][offload] Don't cleanup data when offload met MetaStore exception#21686

Merged
merlimat merged 2 commits into
apache:masterfrom
zymap:fix-regression
Dec 7, 2023
Merged

[fix][offload] Don't cleanup data when offload met MetaStore exception#21686
merlimat merged 2 commits into
apache:masterfrom
zymap:fix-regression

Conversation

@zymap

@zymap zymap commented Dec 7, 2023

Copy link
Copy Markdown
Member

Motivation

#17915 changes the fix #17512 which lead the offload data is deleted when metadata store exception happened. Then the ledger can not be read.

The logs shows

Failed to update offloaded metadata for the ledgerId 6197907, the offloaded data will not be cleaned up

But the ledger deleted. Then managed ledger failed to open it

Error opening ledger for reading at position 6197907:0

Modifications

Verifying this change

  • Make sure that the change passes the CI checks.

(Please pick either of the following options)

This change is a trivial rework / code cleanup without any test coverage.

(or)

This change is already covered by existing tests, such as (please describe tests).

(or)

This change added tests and can be verified as follows:

(example:)

  • Added integration tests for end-to-end deployment with large payloads (10MB)
  • Extended integration test for recovery after broker failure

Does this pull request potentially affect one of the following parts:

If the box was checked, please highlight the changes

  • Dependencies (add or upgrade a dependency)
  • The public API
  • The schema
  • The default values of configurations
  • The threading model
  • The binary protocol
  • The REST endpoints
  • The admin CLI options
  • The metrics
  • Anything that affects deployment

Documentation

  • doc
  • doc-required
  • doc-not-needed
  • doc-complete

Matching PR in forked repository

PR in forked repository:

…xception

---

### Motivation

apache#17915 changes the fix apache#17512 which lead the offload data
is deleted when metadata store exception happened. Then the
ledger can not be read.

The logs shows
```
Failed to update offloaded metadata for the ledgerId 6197907, the offloaded data will not be cleaned up
```
But the ledger deleted. Then managed ledger failed to open it
```
Error opening ledger for reading at position 6197907:0
```
@zymap zymap added this to the 3.2.0 milestone Dec 7, 2023
@zymap zymap self-assigned this Dec 7, 2023
@github-actions github-actions Bot added the doc-not-needed Your PR changes do not impact docs label Dec 7, 2023
@zymap zymap changed the title [fix][tiered-storage] Don't cleanup data when offload met MetaStore exception [fix][offload] Don't cleanup data when offload met MetaStore exception Dec 7, 2023
@zymap

zymap commented Dec 7, 2023

Copy link
Copy Markdown
Member Author

/pulsarbot rerun-failure-checks

@codecov-commenter

codecov-commenter commented Dec 7, 2023

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 73.37%. Comparing base (ab77ca2) to head (769c30e).
⚠️ Report is 1416 commits behind head on master.

Additional details and impacted files

Impacted file tree graph

@@             Coverage Diff              @@
##             master   #21686      +/-   ##
============================================
- Coverage     73.39%   73.37%   -0.03%     
+ Complexity    32763    32744      -19     
============================================
  Files          1893     1893              
  Lines        140753   140754       +1     
  Branches      15503    15503              
============================================
- Hits         103306   103272      -34     
- Misses        29346    29373      +27     
- Partials       8101     8109       +8     
Flag Coverage Δ
inttests 24.15% <0.00%> (+0.03%) ⬆️
systests 24.77% <100.00%> (-0.06%) ⬇️
unittests 72.65% <100.00%> (-0.02%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
...che/bookkeeper/mledger/impl/ManagedLedgerImpl.java 80.65% <100.00%> (+<0.01%) ⬆️

... and 66 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants