rgw/dedup: full object dedup continuous work#63560
Conversation
|
jenkins test api |
|
jenkins test windows |
|
can you please give this pr and commit a real title? you just copied the exact same text from #62179 |
|
jenkins test windows |
|
jenkins test windows |
|
jenkins test make check |
| int ret = get_epoch(store, dpp, &old_epoch, __func__); | ||
| if (ret != 0) { | ||
| return ret; | ||
| // generate an empty epoch with zero counters |
There was a problem hiding this comment.
this is a change in behavior.
what could be the reason for the failure? do we always create an empty epoch regardless of why we failed?
There was a problem hiding this comment.
We need to have an EPOCH object to be able to run scans.
The EPOCH object is created on the first scan (we used to create one when loading the code)
The only objects created at startup are the watch objects
| return false; | ||
| ldpp_dout(dpp, 1) << __func__ << "::failed shard_progress_t decode!" << dendl; | ||
| completed_arr[shard] = TOKEN_STATE_CORRUPTED; | ||
| continue; |
There was a problem hiding this comment.
this is a behavior change - not bailing out on a malformed token?
There was a problem hiding this comment.
The first pass collects state from all token, but will still fail the call.
After a wait time of 120 seconds the caller will pass "FORCE" option allowing it to skip passed failing tokens.
Active in-process tokens will still wait for completion.
|
are the 2 following counters the same or could be different:
|
answered above |
|
jenkins test api |
|
This pull request can no longer be automatically merged: a rebase is needed and changes have to be manually resolved |
Moved all control objects (EPOCH, WATCH, Tokens) to default.rgw.control pool. Add dedup_pool to RGWZoneParams to make the name unique across zones rgw.dedup pool is created on dedup start and removed when the scan is over report duplicated space after dedup because of the head-object report potential dedup for smaller objects (64KB-4MB) added tests for the new reporting facilities Signed-off-by: Gabriel BenHanokh <gbenhano@redhat.com>
|
restarted failing tests (mostly No module named 'tasks.ceph') |
|
rebased and repeated the failing tests: 1 failure in rgw multisite test caused by a missing SSH private key: 1 failure from kafka_failover with rgw stdout reporting 1 more failure I don't understand from rgw/singleton/ I saw nothing of interest on the system except for a report on ceph.log complaining that osd.0 was reported immediately failed by osd.1, osd.2 and osd.3 |
|
Moved all control objects (EPOCH, WATCH, Tokens) to default.rgw.control pool.
rgw.dedup pool is created on dedup start and removed when the scan is over
report duplicated space after dedup because of the head-object
report potential dedup for smaller objects (64KB-4MB)
code cleanup
Contribution Guidelines
To sign and title your commits, please refer to Submitting Patches to Ceph.
If you are submitting a fix for a stable branch (e.g. "quincy"), please refer to Submitting Patches to Ceph - Backports for the proper workflow.
When filling out the below checklist, you may click boxes directly in the GitHub web UI. When entering or editing the entire PR message in the GitHub web UI editor, you may also select a checklist item by adding an
xbetween the brackets:[x]. Spaces and capitalization matter when checking off items this way.Checklist
Show available Jenkins commands
jenkins test classic perfJenkins Job | Jenkins Job Definitionjenkins test crimson perfJenkins Job | Jenkins Job Definitionjenkins test signedJenkins Job | Jenkins Job Definitionjenkins test make checkJenkins Job | Jenkins Job Definitionjenkins test make check arm64Jenkins Job | Jenkins Job Definitionjenkins test submodulesJenkins Job | Jenkins Job Definitionjenkins test dashboardJenkins Job | Jenkins Job Definitionjenkins test dashboard cephadmJenkins Job | Jenkins Job Definitionjenkins test apiJenkins Job | Jenkins Job Definitionjenkins test docsReadTheDocs | Github Workflow Definitionjenkins test ceph-volume allJenkins Jobs | Jenkins Jobs Definitionjenkins test windowsJenkins Job | Jenkins Job Definitionjenkins test rook e2eJenkins Job | Jenkins Job Definition