Skip to content

os/bluestore: enable async manual compactions#57107

Merged
yuriw merged 2 commits intoceph:mainfrom
ifed01:wip-ifed-async-compact
Jun 27, 2024
Merged

os/bluestore: enable async manual compactions#57107
yuriw merged 2 commits intoceph:mainfrom
ifed01:wip-ifed-async-compact

Conversation

@ifed01
Copy link
Contributor

@ifed01 ifed01 commented Apr 26, 2024

Let's perform manual and OSD starting DB compaction asynchronously.

Fixes: https://tracker.ceph.com/issues/67040

Contribution Guidelines

  • To sign and title your commits, please refer to Submitting Patches to Ceph.

  • If you are submitting a fix for a stable branch (e.g. "quincy"), please refer to Submitting Patches to Ceph - Backports for the proper workflow.

  • When filling out the below checklist, you may click boxes directly in the GitHub web UI. When entering or editing the entire PR message in the GitHub web UI editor, you may also select a checklist item by adding an x between the brackets: [x]. Spaces and capitalization matter when checking off items this way.

Checklist

  • Tracker (select at least one)
    • References tracker ticket
    • Very recent bug; references commit where it was introduced
    • New feature (ticket optional)
    • Doc update (no ticket needed)
    • Code cleanup (no ticket needed)
  • Component impact
    • Affects Dashboard, opened tracker ticket
    • Affects Orchestrator, opened tracker ticket
    • No impact that needs to be tracked
  • Documentation (select at least one)
    • Updates relevant documentation
    • No doc update is appropriate
  • Tests (select at least one)
Show available Jenkins commands
  • jenkins retest this please
  • jenkins test classic perf
  • jenkins test crimson perf
  • jenkins test signed
  • jenkins test make check
  • jenkins test make check arm64
  • jenkins test submodules
  • jenkins test dashboard
  • jenkins test dashboard cephadm
  • jenkins test api
  • jenkins test docs
  • jenkins render docs
  • jenkins test ceph-volume all
  • jenkins test ceph-volume tox
  • jenkins test windows
  • jenkins test rook e2e

@pereman2
Copy link
Contributor

jenkins test api

f->open_object_section("compact_result");
f->dump_float("elapsed_time", duration);
f->close_section();
int r = store->compact();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wouldn't it be better if we had here control between compact and compact async ?


if (cct->_conf.get_val<bool>("osd_compact_on_start")) {
dout(2) << "compacting object store's omap" << dendl;
dout(2) << "compacting object store's DB" << dendl;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1, makes more sense.

ifed01 added 2 commits May 8, 2024 18:37
Signed-off-by: Igor Fedotov <igor.fedotov@croit.io>
Signed-off-by: Igor Fedotov <igor.fedotov@croit.io>
@ifed01 ifed01 force-pushed the wip-ifed-async-compact branch from b1c9996 to 2e06b3d Compare May 8, 2024 17:07
@ifed01
Copy link
Contributor Author

ifed01 commented May 8, 2024

@aclamk - just added more perf counters to track pending/running compactions:
"compact": 4,
"compact_running": 1, #whether compaction is running
"compact_completed": 3,
"compact_lasted": 0.000054410, #last completed compaction duration
"compact_queue_len": 0, #pending compactions

@pereman2
Copy link
Contributor

pereman2 commented May 9, 2024

jenkins test api

1 similar comment
@ifed01
Copy link
Contributor Author

ifed01 commented May 10, 2024

jenkins test api

@ifed01
Copy link
Contributor Author

ifed01 commented May 13, 2024

@aclamk - OK to go?

@amathuria
Copy link
Contributor

@yuriw yuriw merged commit 70580ac into ceph:main Jun 27, 2024
@ifed01 ifed01 deleted the wip-ifed-async-compact branch July 10, 2024 14:13
ifed01 added a commit to ifed01/ceph that referenced this pull request Aug 30, 2024
Merging ceph#57107 updated BlueFS data
layout for some of spillover tests. This has negative impact (broken
test cases) in Quincy release only though.
See https://tracker.ceph.com/issues/67838.

But let's rollback to the original "sync compaction" behavior for other
releases as well. Just in case...

Signed-off-by: Igor Fedotov <igor.fedotov@croit.io>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants