tools/bluestore_tool: add support for zap-device command#59632
Closed
tools/bluestore_tool: add support for zap-device command#59632
Conversation
Fixes: https://tracker.ceph.com/issues/67926 Signed-off-by: Igor Fedotov <igor.fedotov@croit.io>
Signed-off-by: Igor Fedotov <igor.fedotov@croit.io>
Contributor
|
thanks for your prompt action @ifed01 ! |
Contributor
Author
|
jenkins test make check |
1 similar comment
Contributor
|
jenkins test make check |
aclamk
reviewed
Sep 9, 2024
| int r = _bdev->open(dev); | ||
| if (r < 0) | ||
| goto fail; | ||
| if (!gap_size) { |
Contributor
There was a problem hiding this comment.
I think any freedom around selecting zap brush size (gap_size) is redundant.
We want to brush just and exactly as much as needed, which is:
max(BDEV_LABEL_BLOCK_SIZE, bdev->get_block_size())
My proposal is to simplify it out.
Contributor
There was a problem hiding this comment.
For what it's worth, this is only my opinion, but what I expect from the zap-device command is simply to remove the Bluestore signature and nothing else.
I wouldn't really care about the block size being written, as long as the Bluestore signatures are removed altogether. Your call.
guits
approved these changes
Sep 11, 2024
Contributor
guits
left a comment
There was a problem hiding this comment.
tested with ceph-volume:
[root@cbt-zap-1 /]# ceph-volume raw prepare --data /dev/vdb
Running command: /usr/bin/ceph-authtool --gen-print-key
Running command: /usr/bin/ceph-authtool --gen-print-key
Running command: /usr/bin/ceph --cluster ceph --name client.bootstrap-osd --keyring /var/lib/ceph/bootstrap-osd/ceph.keyring -i - osd new db21db26-c916-4bba-beab-ae185d4c86eb
Running command: /usr/bin/mount -t tmpfs tmpfs /var/lib/ceph/osd/ceph-0
Running command: /usr/bin/chown -R ceph:ceph /dev/vdb
Running command: /usr/bin/ln -s /dev/vdb /var/lib/ceph/osd/ceph-0/block
Running command: /usr/bin/ceph --cluster ceph --name client.bootstrap-osd --keyring /var/lib/ceph/bootstrap-osd/ceph.keyring mon getmap -o /var/lib/ceph/osd/ceph-0/activate.monmap
stderr: got monmap epoch 1
--> Creating keyring file for osd.0
Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0/keyring
Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0/
Running command: /usr/bin/ceph-osd --cluster ceph --osd-objectstore bluestore --mkfs -i 0 --monmap /var/lib/ceph/osd/ceph-0/activate.monmap --keyfile - --osd-data /var/lib/ceph/osd/ceph-0/ --osd-uuid db21db26-c916-4bba-beab-ae185d4c86eb --setuser ceph --setgroup ceph
stderr: 2024-09-11T13:15:04.550+0000 7f2015d54640 -1 bluestore(/var/lib/ceph/osd/ceph-0//block) _read_bdev_label unable to decode label /var/lib/ceph/osd/ceph-0//block at offset 102: void bluestore_bdev_label_t::decode(ceph::buffer::v15_2_0::list::const_iterator&) decode past end of struct encoding: Malformed input [buffer:3]
stderr: 2024-09-11T13:15:04.550+0000 7f2015d54640 -1 bluestore(/var/lib/ceph/osd/ceph-0//block) _read_bdev_label unable to decode label /var/lib/ceph/osd/ceph-0//block at offset 102: void bluestore_bdev_label_t::decode(ceph::buffer::v15_2_0::list::const_iterator&) decode past end of struct encoding: Malformed input [buffer:3]
stderr: 2024-09-11T13:15:04.550+0000 7f2015d54640 -1 bluestore(/var/lib/ceph/osd/ceph-0//block) _read_bdev_label unable to decode label /var/lib/ceph/osd/ceph-0//block at offset 102: void bluestore_bdev_label_t::decode(ceph::buffer::v15_2_0::list::const_iterator&) decode past end of struct encoding: Malformed input [buffer:3]
stderr: 2024-09-11T13:15:04.550+0000 7f2015d54640 -1 bluestore(/var/lib/ceph/osd/ceph-0//block) _read_bdev_label unable to decode label /var/lib/ceph/osd/ceph-0//block at offset 102: void bluestore_bdev_label_t::decode(ceph::buffer::v15_2_0::list::const_iterator&) decode past end of struct encoding: Malformed input [buffer:3]
stderr: 2024-09-11T13:15:04.550+0000 7f2015d54640 -1 bluestore(/var/lib/ceph/osd/ceph-0//block) No valid bdev label found
stderr: 2024-09-11T13:15:04.813+0000 7f2015d54640 -1 bluestore(/var/lib/ceph/osd/ceph-0/) _read_fsid unparsable uuid
--> ceph-volume raw prepare successful for: /dev/vdb
[root@cbt-zap-1 /]# hexdump -C -s 0 -n 22 /dev/vdb
00000000 62 6c 75 65 73 74 6f 72 65 20 62 6c 6f 63 6b 20 |bluestore block |
00000010 64 65 76 69 63 65 |device|
00000016
[root@cbt-zap-1 /]# hexdump -C -s 1073741824 -n 22 /dev/vdb
40000000 62 6c 75 65 73 74 6f 72 65 20 62 6c 6f 63 6b 20 |bluestore block |
40000010 64 65 76 69 63 65 |device|
40000016
[root@cbt-zap-1 /]# hexdump -C -s 10737418240 -n 22 /dev/vdb
280000000 62 6c 75 65 73 74 6f 72 65 20 62 6c 6f 63 6b 20 |bluestore block |
280000010 64 65 76 69 63 65 |device|
280000016
[root@cbt-zap-1 /]# ceph-bluestore-tool zap-device --dev /dev/vdb --yes-i-really-really-mean-it
[root@cbt-zap-1 /]# hexdump -C -s 0 -n 22 /dev/vdb
00000000 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
00000010 00 00 00 00 00 00 |......|
00000016
[root@cbt-zap-1 /]# hexdump -C -s 1073741824 -n 22 /dev/vdb
40000000 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
40000010 00 00 00 00 00 00 |......|
40000016
[root@cbt-zap-1 /]# hexdump -C -s 10737418240 -n 22 /dev/vdb
280000000 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
280000010 00 00 00 00 00 00 |......|
280000016
[root@cbt-zap-1 /]#
guits
added a commit
to guits/ceph
that referenced
this pull request
Sep 11, 2024
BlueStore now writes its metadata at multiple offset on devices [1]. It means `ceph-volume lvm zap` doesn't remove BlueStore signature altogether. This can confuse ceph-volume when redeploying an OSD on a previously zapped device because there is still old BlueStore metadata on it. ceph-volume should call `ceph-bluestore-tool zap-device` [2] in addition to the existing calls when wiping a device. [1] ceph#55374 [2] ceph#59632 Fixes: https://tracker.ceph.com/issues/68035 Signed-off-by: Guillaume Abrioux <gabrioux@ibm.com>
guits
added a commit
to guits/ceph
that referenced
this pull request
Sep 11, 2024
BlueStore now writes its metadata at multiple offset on devices [1]. It means `ceph-volume lvm zap` doesn't remove BlueStore signature altogether. This can confuse ceph-volume when redeploying an OSD on a previously zapped device because there is still old BlueStore metadata on it. ceph-volume should call `ceph-bluestore-tool zap-device` [2] in addition to the existing calls when wiping a device. [1] ceph#55374 [2] ceph#59632 Fixes: https://tracker.ceph.com/issues/68035 Signed-off-by: Guillaume Abrioux <gabrioux@ibm.com>
guits
added a commit
to guits/ceph
that referenced
this pull request
Sep 11, 2024
BlueStore now writes its metadata at multiple offset on devices [1]. It means `ceph-volume lvm zap` doesn't remove BlueStore signature altogether. This can confuse ceph-volume when redeploying an OSD on a previously zapped device because there is still old BlueStore metadata on it. ceph-volume should call `ceph-bluestore-tool zap-device` [2] in addition to the existing calls when wiping a device. [1] ceph#55374 [2] ceph#59632 Fixes: https://tracker.ceph.com/issues/68035 Signed-off-by: Guillaume Abrioux <gabrioux@ibm.com>
14 tasks
guits
added a commit
to guits/ceph
that referenced
this pull request
Sep 13, 2024
BlueStore now writes its metadata at multiple offset on devices [1]. It means `ceph-volume lvm zap` doesn't remove BlueStore signature altogether. This can confuse ceph-volume when redeploying an OSD on a previously zapped device because there is still old BlueStore metadata on it. ceph-volume should call `ceph-bluestore-tool zap-device` [2] in addition to the existing calls when wiping a device. [1] ceph#55374 [2] ceph#59632 Fixes: https://tracker.ceph.com/issues/68035 Signed-off-by: Guillaume Abrioux <gabrioux@ibm.com>
guits
added a commit
to guits/ceph
that referenced
this pull request
Sep 13, 2024
BlueStore now writes its metadata at multiple offset on devices [1]. It means `ceph-volume lvm zap` doesn't remove BlueStore signature altogether. This can confuse ceph-volume when redeploying an OSD on a previously zapped device because there is still old BlueStore metadata on it. ceph-volume should call `ceph-bluestore-tool zap-device` [2] in addition to the existing calls when wiping a device. [1] ceph#55374 [2] ceph#59632 Fixes: https://tracker.ceph.com/issues/68035 Signed-off-by: Guillaume Abrioux <gabrioux@ibm.com>
guits
added a commit
to guits/ceph
that referenced
this pull request
Sep 13, 2024
BlueStore now writes its metadata at multiple offset on devices [1]. It means `ceph-volume lvm zap` doesn't remove BlueStore signature altogether. This can confuse ceph-volume when redeploying an OSD on a previously zapped device because there is still old BlueStore metadata on it. ceph-volume should call `ceph-bluestore-tool zap-device` [2] in addition to the existing calls when wiping a device. [1] ceph#55374 [2] ceph#59632 Fixes: https://tracker.ceph.com/issues/68035 Signed-off-by: Guillaume Abrioux <gabrioux@ibm.com>
Contributor
Author
|
Closed in favor of #59762 |
guits
added a commit
to guits/ceph
that referenced
this pull request
Sep 25, 2024
BlueStore now writes its metadata at multiple offset on devices [1]. It means `ceph-volume lvm zap` doesn't remove BlueStore signature altogether. This can confuse ceph-volume when redeploying an OSD on a previously zapped device because there is still old BlueStore metadata on it. ceph-volume should call `ceph-bluestore-tool zap-device` [2] in addition to the existing calls when wiping a device. [1] ceph#55374 [2] ceph#59632 Fixes: https://tracker.ceph.com/issues/68035 Signed-off-by: Guillaume Abrioux <gabrioux@ibm.com> (cherry picked from commit dcf7439)
Naveenaidu
pushed a commit
to Naveenaidu/ceph
that referenced
this pull request
Oct 3, 2024
BlueStore now writes its metadata at multiple offset on devices [1]. It means `ceph-volume lvm zap` doesn't remove BlueStore signature altogether. This can confuse ceph-volume when redeploying an OSD on a previously zapped device because there is still old BlueStore metadata on it. ceph-volume should call `ceph-bluestore-tool zap-device` [2] in addition to the existing calls when wiping a device. [1] ceph#55374 [2] ceph#59632 Fixes: https://tracker.ceph.com/issues/68035 Signed-off-by: Guillaume Abrioux <gabrioux@ibm.com>
oshrey16
pushed a commit
to oshrey16/ceph
that referenced
this pull request
Oct 13, 2024
BlueStore now writes its metadata at multiple offset on devices [1]. It means `ceph-volume lvm zap` doesn't remove BlueStore signature altogether. This can confuse ceph-volume when redeploying an OSD on a previously zapped device because there is still old BlueStore metadata on it. ceph-volume should call `ceph-bluestore-tool zap-device` [2] in addition to the existing calls when wiping a device. [1] ceph#55374 [2] ceph#59632 Fixes: https://tracker.ceph.com/issues/68035 Signed-off-by: Guillaume Abrioux <gabrioux@ibm.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
As BlueStore might have multiple bdev labels on a single device it might be trickier to zap OSD device(s) using dd tool. So let's implement this in ceph-bluestore-tool and update ceph-volume accordingly. The latter is out of scope of this PR though.
Please note that this PR relies on
#59627
to fix --yes-i-really-really-mean-it option specification .
Contribution Guidelines
To sign and title your commits, please refer to Submitting Patches to Ceph.
If you are submitting a fix for a stable branch (e.g. "quincy"), please refer to Submitting Patches to Ceph - Backports for the proper workflow.
When filling out the below checklist, you may click boxes directly in the GitHub web UI. When entering or editing the entire PR message in the GitHub web UI editor, you may also select a checklist item by adding an
xbetween the brackets:[x]. Spaces and capitalization matter when checking off items this way.Checklist
Show available Jenkins commands
jenkins retest this pleasejenkins test classic perfjenkins test crimson perfjenkins test signedjenkins test make checkjenkins test make check arm64jenkins test submodulesjenkins test dashboardjenkins test dashboard cephadmjenkins test apijenkins test docsjenkins render docsjenkins test ceph-volume alljenkins test ceph-volume toxjenkins test windowsjenkins test rook e2e