crimson: Add objectstore tool with partial functionality#63604
crimson: Add objectstore tool with partial functionality#63604
Conversation
4b86652 to
4621266
Compare
|
jenkins test make check |
|
jenkins test make check arm64 |
There was a problem hiding this comment.
Sorry about the late review. I'm not very familiar with the background of objectstore tool yet.
It will be helpful to understand the specific scenarios this objectstore tool is designed for as well as the related Ceph infrastructure.
See existing obejctstore tool used in classic: https://docs.ceph.com/en/latest/man/8/ceph-objectstore-tool/
This offline tool allows exposing and manipulating the underlying rados objects. This tool would be essential to mitigate issue with deployed clusters or for debugging/reproducing issues for development (e.g listing the objects to spot leaks, setting specific metadata to reproduce scenarios or exposing dedicated behavior to mitigate issues like "reset-last-complete"). This should largely help with identifying issues at the objectstore level in the future (and possibly now).
@guojidan, Thank you for working on this one! After @cyx1231st comments are addressed, would you like to consider switching some of the intorduced code to coroutines? This would make the tool easier to maintain in future iterations of it.
|
WIP |
79bf069 to
975533c
Compare
|
f7474cf to
c99aff5
Compare
cyx1231st
left a comment
There was a problem hiding this comment.
Generally looks good from SeaStore perspective!
Matan-B
left a comment
There was a problem hiding this comment.
Reviewing qa/.. side first, left few comments.
| - install: | ||
| ceph: | ||
| flavor: crimson | ||
| - ceph: | ||
| flavor: crimson |
There was a problem hiding this comment.
The structure of the new added test suite should look something like this:
Matan-B@4cccea5
Feel free to cherry-pick or edit according to the structure there.
qa/tasks/crimson_objectstore_tool.py
Outdated
| db = {} | ||
|
|
||
| # Setup local and remote data | ||
| LOCALDIR = tempfile.mkdtemp("cos") |
There was a problem hiding this comment.
What does "cos" stand for?
There was a problem hiding this comment.
😆 I don't know. I'm just imitating ceph_objectstore_tool.py to write, so I'll keep consistent with ceph_objectstore_tool.py and use cod.
qa/tasks/crimson_objectstore_tool.py
Outdated
| @@ -0,0 +1,536 @@ | |||
| """ | |||
| crimson_objectstore_tool - Test of crimson-objectstore-tool utility | |||
There was a problem hiding this comment.
Can we reuse the existing qa/tasks/ceph_objectstore_tool.py? IIUC, the main difference is the tool binary name which could be defined based on a new supported parameter? Something like:
- objectstore_tool:
crimson_objectstore_tool: true #use crimson-objectstore-tool
objects: 5
pgnum: 8
and a corresponding:
CRIMSON = config.get('crimson_objectstore_tool', false)
Any unsupported features could be also skipped by introducing a SKIP_IF_CRIMSON method. We use similar approach with all /src/test/.. unittests.
|
rebasing on top of #62530 which changed omap behavior and require changes |
|
The tool is finally available in the rpm package: |
fixes: https://tracker.ceph.com/issues/71504 Signed-off-by: junxiang Mu <1948535941@qq.com> Signed-off-by: Matan Breizman <mbreizma@redhat.com>
Include the tool when packaging for deb/rpm. Signed-off-by: Matan Breizman <mbreizma@redhat.com>
|
jenkins retest this please |
|
jenkins test make check arm64 |
2 similar comments
|
jenkins test make check arm64 |
|
jenkins test make check arm64 |
|
jenkins test make check |
fixes: https://tracker.ceph.com/issues/71504 Signed-off-by: junxiang Mu <1948535941@qq.com> Signed-off-by: Matan Breizman <mbreizma@redhat.com>
Signed-off-by: Matan Breizman <mbreizma@redhat.com>
|
rados (non-crimson) sanity run: |
Contribution Guidelines
To sign and title your commits, please refer to Submitting Patches to Ceph.
If you are submitting a fix for a stable branch (e.g. "quincy"), please refer to Submitting Patches to Ceph - Backports for the proper workflow.
When filling out the below checklist, you may click boxes directly in the GitHub web UI. When entering or editing the entire PR message in the GitHub web UI editor, you may also select a checklist item by adding an
xbetween the brackets:[x]. Spaces and capitalization matter when checking off items this way.Checklist
Show available Jenkins commands
jenkins test classic perfJenkins Job | Jenkins Job Definitionjenkins test crimson perfJenkins Job | Jenkins Job Definitionjenkins test signedJenkins Job | Jenkins Job Definitionjenkins test make checkJenkins Job | Jenkins Job Definitionjenkins test make check arm64Jenkins Job | Jenkins Job Definitionjenkins test submodulesJenkins Job | Jenkins Job Definitionjenkins test dashboardJenkins Job | Jenkins Job Definitionjenkins test dashboard cephadmJenkins Job | Jenkins Job Definitionjenkins test apiJenkins Job | Jenkins Job Definitionjenkins test docsReadTheDocs | Github Workflow Definitionjenkins test ceph-volume allJenkins Jobs | Jenkins Jobs Definitionjenkins test windowsJenkins Job | Jenkins Job Definitionjenkins test rook e2eJenkins Job | Jenkins Job Definitionclosed: https://tracker.ceph.com/issues/71504
PR Description:
This PR introduces a new
crimson-objectstore-toolthat provides essential debugging and inspection capabilities for Crimson object stores.Features:
Key Components:
crimson_objectstore_tool.cc- Main tool implementation with CLI interfaceobjectstore_tool.{cc,h}- Core functionality wrapper around FuturizedStoreUsage Examples:
it's fully compatible with ceph-objectstore-tool