Skip to content

quincy: os/bluesore: cumulative backport for Onode stuff and more#50048

Merged
yuriw merged 17 commits intoceph:quincyfrom
ifed01:wip-ifed-no-statfs-qui
Mar 6, 2023
Merged

quincy: os/bluesore: cumulative backport for Onode stuff and more#50048
yuriw merged 17 commits intoceph:quincyfrom
ifed01:wip-ifed-no-statfs-qui

Conversation

@ifed01
Copy link
Contributor

@ifed01 ifed01 commented Feb 9, 2023 •
edited
Loading

This includes the following PRs:

Signed-off-by: Igor Fedotov igor.fedotov@croit.io

Contribution Guidelines

Checklist

  • Tracker (select at least one)
    • References tracker ticket
    • Very recent bug; references commit where it was introduced
    • New feature (ticket optional)
    • Doc update (no ticket needed)
    • Code cleanup (no ticket needed)
  • Component impact
    • Affects Dashboard, opened tracker ticket
    • Affects Orchestrator, opened tracker ticket
    • No impact that needs to be tracked
  • Documentation (select at least one)
    • Updates relevant documentation
    • No doc update is appropriate
  • Tests (select at least one)
Show available Jenkins commands
  • jenkins retest this please
  • jenkins test classic perf
  • jenkins test crimson perf
  • jenkins test signed
  • jenkins test make check
  • jenkins test make check arm64
  • jenkins test submodules
  • jenkins test dashboard
  • jenkins test dashboard cephadm
  • jenkins test api
  • jenkins test docs
  • jenkins render docs
  • jenkins test ceph-volume all
  • jenkins test ceph-volume tox
  • jenkins test windows

ifed01 and others added 17 commits February 9, 2023 11:42
@ifed01
os/bluestore: cleanup around null freelist manager
0c8554e 
Signed-off-by: Igor Fedotov <igor.fedotov@croit.io>
(cherry picked from commit 8840110)
@ifed01
test/store_test: enable sporadic full NCB recovery
4678757 
Signed-off-by: Igor Fedotov <igor.fedotov@croit.io>
(cherry picked from commit 9544de8)
@ifed01
os/bluestore: get rid off redundant is_null_fm() func
f405d94 
Signed-off-by: Igor Fedotov <igor.fedotov@croit.io>
(cherry picked from commit 59d973e)
@ifed01
os/bluestore: mark has_null_manager() const
7b11fb8 
Signed-off-by: Igor Fedotov <igor.fedotov@croit.io>
(cherry picked from commit 7c28956)
@ifed01
os/bluestore: refactor NCB recovery procedure.
bac9a5f 
This implements a basis for statfs recovery from persistent Onode metadata.
Plus some redesign to make this procedure more lightweight and performant
 - via avoiding full Onode rebuild.

Signed-off-by: Igor Fedotov <igor.fedotov@croit.io>
(cherry picked from commit bda1a20)
@ifed01
os/bluestore: do not persist statfs on every txc
2140070 
Signed-off-by: Igor Fedotov <igor.fedotov@croit.io>
(cherry picked from commit b343580)
@ifed01
test/objectstore/store_test: update tests due to new statfs mechanics
fe2047b 
Signed-off-by: Igor Fedotov <igor.fedotov@croit.io>
(cherry picked from commit 62c6d8e)
@ifed01
os/bluestore: get rid off redundant close_db_leave_bluefs() method
576eec0 
Signed-off-by: Igor Fedotov <igor.fedotov@croit.io>
(cherry picked from commit 56e8bb2)
@aclamk @ifed01
os/bluestore, kv/rocksdb: Cleanup on close_db_environment
9450711 
Refine actions taken in close_db_environment. Its role is to close
db handle and environment when db was used in special modes - repair/reshard,
and is not actually open to typical r/w.

Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
(cherry picked from commit e78504a)
@ifed01
os/bluestore: invalidate statfs completely when destaging allocations
44293c6 
Signed-off-by: Igor Fedotov <ifedotov@croit.io>
(cherry picked from commit 18cc766)
@ifed01
os/bluestore: rename Collection::onode_map to onode_space
d1c439e 
to ease code reading.

Signed-off-by: Igor Fedotov <igor.fedotov@croit.io>
(cherry picked from commit 0c02f3e)
@ifed01
os/bluestore: minor refactoring around OnodeSpace struct to improve code
1a59ea4 
readability.

Signed-off-by: Igor Fedotov <igor.fedotov@croit.io>
(cherry picked from commit 3df4a8d)
@ifed01
os/bluestore: avoid duplicate lookup when adding entry to OnodeSpace
50d5609 
Signed-off-by: Igor Fedotov <igor.fedotov@croit.io>
(cherry picked from commit 7eed358)
@ifed01
os/bluestore: fix onode ref counting
6b5a5b4 
This should eliminate duplicate onode releases that could happen before.
Additionally onode pinning is performed during cache trimming not onode
ref count increment.

[Hopefully] fixes: https://tracker.ceph.com/issues/53002

Signed-off-by: Igor Fedotov <igor.fedotov@croit.io>
(cherry picked from commit a3057f4)
@ifed01
os/bluestore: unify OnodeRef usage
ebfb518 
Signed-off-by: Igor Fedotov <igor.fedotov@croit.io>
(cherry picked from commit 3b40c5c)

 Conflicts:
	src/os/bluestore/BlueStore.cc
	src/os/bluestore/BlueStore.h
 <unrelated modification for the same code>
@ifed01
os/bluestore: simplify onode cache trimming
564cfef 
Signed-off-by: Igor Fedotov <igor.fedotov@croit.io>
(cherry picked from commit 5aaa2e0)
@ifed01
os/bluestore: touch onode in cache LRU explicitly.
2ec0acb 
Signed-off-by: Igor Fedotov <igor.fedotov@croit.io>
(cherry picked from commit 1d4abf7)
@ifed01 ifed01 requested a review from a team as a code owner February 9, 2023 09:20
@ifed01 ifed01 requested a review from aclamk February 9, 2023 09:20
@github-actions github-actions bot added this to the quincy milestone Feb 9, 2023
@ifed01 ifed01 requested a review from benhanokh February 9, 2023 09:21
@ifed01
Copy link
Contributor Author

ifed01 commented Feb 9, 2023

jenkins test api

aclamk
aclamk approved these changes Mar 1, 2023
View reviewed changes
Copy link
Contributor

@aclamk aclamk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is OK, as far as I can manually inspect.

@ifed01 ifed01 added the needs-qa label Mar 1, 2023
@ljflores
Copy link
Member

ljflores commented Mar 6, 2023

Rados suite review: https://pulpito.ceph.com/?branch=wip-yuri3-testing-2023-03-01-0812-quincy

Failures, unrelated:
1. https://tracker.ceph.com/issues/49287
2. https://tracker.ceph.com/issues/58146
3. https://tracker.ceph.com/issues/49961

Details:
1. failed to write to cgroup.procs - Ceph - Orchestrator
2. test_cephadm.sh: Error: Error initializing source docker://quay.ceph.io/ceph-ci/ceph:master - Ceph - Orchestrator
3. scrub/osd-recovery-scrub.sh: TEST_recovery_scrub_1 failed - Ceph - RADOS

@yuriw yuriw merged commit 5f902ab into ceph:quincy Mar 6, 2023
@ifed01 ifed01 deleted the wip-ifed-no-statfs-qui branch March 7, 2023 08:24
@ljflores ljflores mentioned this pull request Apr 11, 2023
Merged
14 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@aclamk aclamk aclamk approved these changes

@benhanokh benhanokh Awaiting requested review from benhanokh

Assignees

No one assigned

Labels

backport bluestore bug-fix common core needs-qa tests wip-yuri3-testing

Projects

None yet

Milestone

quincy

Development

Successfully merging this pull request may close these issues.

4 participants

@ifed01 @ljflores @aclamk @yuriw