Skip to content

luminous: os/bluestore: Added rescue procedure for bluefs log replay#35776

Merged
neha-ojha merged 3 commits intoceph:luminousfrom
aclamk:wip-bluefs-log-replay-rescue-lum
Jul 15, 2020
Merged

luminous: os/bluestore: Added rescue procedure for bluefs log replay#35776
neha-ojha merged 3 commits intoceph:luminousfrom
aclamk:wip-bluefs-log-replay-rescue-lum

Conversation

@aclamk
Copy link
Contributor

@aclamk aclamk commented Jun 25, 2020

This is a recovery procedure for cases where bluefs log grew so much that it cannot be read.
This fixes https://tracker.ceph.com/issues/46195 .

Recommended procedure:

  1. Check if recovery procedure works in your case:
    ceph-bluestore-tool -l /proc/self/fd/1 --log-level 5 --path dev/osd1 fsck --debug_bluefs=5/5 --bluefs_replay_recovery=true --bluefs_replay_recovery_disable_compact=true
    Fsck should complete with minimal errors.
  2. Apply recovery
    ceph-bluestore-tool -l /proc/self/fd/1 --log-level 5 --path dev/osd1 fsck --debug_bluefs=5/5 --bluefs_replay_recovery=true

@aclamk aclamk force-pushed the wip-bluefs-log-replay-rescue-lum branch from f722ef8 to c6674ed Compare June 25, 2020 13:24
@aclamk aclamk requested review from ifed01, jdurgin and neha-ojha June 25, 2020 13:42
@ifed01
Copy link
Contributor

ifed01 commented Jun 25, 2020

@aclamk - we do want the same in master, don't we?

@aclamk
Copy link
Contributor Author

aclamk commented Jun 25, 2020

@ifed01 EDIT: I think it should go to all versions, but luminous is urgent.
Master may be exempt, since we merged #35473 .

@ifed01
Copy link
Contributor

ifed01 commented Jun 25, 2020

@aclamk - this might look like a grumbling but IMO that's a bad procedure that is bringing a mess into commit history... May be try pushing hard the patch forward through the master first?

@aclamk
Copy link
Contributor Author

aclamk commented Jun 25, 2020

@ifed01 Problem is, that master does not need it. I would be unable to recreate state that is begin fixed here on master.

@neha-ojha
Copy link
Member

@ifed01 Problem is, that master does not need it. I would be unable to recreate state that is begin fixed here on master.

you could add an explanation in your commit message to explain why this commit is not cherry-picked from master

@ifed01 ifed01 changed the title os/bluestore: Added rescue procedure for bluefs log replay luminous:os/bluestore: Added rescue procedure for bluefs log replay Jun 26, 2020
@tchaikov tchaikov added this to the luminous milestone Jun 27, 2020
@aclamk aclamk force-pushed the wip-bluefs-log-replay-rescue-lum branch from 67cae05 to 564c246 Compare June 28, 2020 09:18
@yuriw
Copy link
Contributor

yuriw commented Jul 8, 2020

@yuriw
Copy link
Contributor

yuriw commented Jul 13, 2020

aclamk added 3 commits July 15, 2020 21:58
This is a procedure tries to find on disk unreachable extents and pretend they were
already a part of bluefs log. If this gives proper crc, accept it.

Fixes: https://tracker.ceph.com/issues/46195
Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
This ability only makes sense as a step that allows to perform fsck before commiting recovered bluefs log.

Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
Adds additional paragraph to ceph-bluestore-tool documentation,
describing how to use *special* options --bluefs_replay_recovery
and --bluefs_replay_recovery_disable_compact to recover large
bluefs log.

Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
@aclamk aclamk force-pushed the wip-bluefs-log-replay-rescue-lum branch from a7b7f3e to cc4c069 Compare July 15, 2020 20:07
@neha-ojha neha-ojha merged commit 5a69e6b into ceph:luminous Jul 15, 2020
@smithfarm smithfarm changed the title luminous:os/bluestore: Added rescue procedure for bluefs log replay luminous: os/bluestore: Added rescue procedure for bluefs log replay Aug 13, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants