Skip to content

crimson: Support Partial Object Recovery#49594

Merged
athanatos merged 7 commits intoceph:mainfrom
Matan-B:wip-matanb-crimson-calc-subsets
Jan 4, 2024
Merged

crimson: Support Partial Object Recovery#49594
athanatos merged 7 commits intoceph:mainfrom
Matan-B:wip-matanb-crimson-calc-subsets

Conversation

@Matan-B
Copy link
Contributor

@Matan-B Matan-B commented Dec 29, 2022 •
edited
Loading

TODO:

do_recovery() Comparisons:

Classic:

Crimson:

Contribution Guidelines

Checklist

  • Tracker (select at least one)
    • References tracker ticket
    • Very recent bug; references commit where it was introduced
    • New feature (ticket optional)
    • Doc update (no ticket needed)
    • Code cleanup (no ticket needed)
  • Component impact
    • Affects Dashboard, opened tracker ticket
    • Affects Orchestrator, opened tracker ticket
    • No impact that needs to be tracked
  • Documentation (select at least one)
    • Updates relevant documentation
    • No doc update is appropriate
  • Tests (select at least one)
Show available Jenkins commands
  • jenkins retest this please
  • jenkins test classic perf
  • jenkins test crimson perf
  • jenkins test signed
  • jenkins test make check
  • jenkins test make check arm64
  • jenkins test submodules
  • jenkins test dashboard
  • jenkins test dashboard cephadm
  • jenkins test api
  • jenkins test docs
  • jenkins render docs
  • jenkins test ceph-volume all
  • jenkins test ceph-volume tox
  • jenkins test windows

@Matan-B Matan-B added the DNM label Dec 29, 2022
@Matan-B Matan-B force-pushed the wip-matanb-crimson-calc-subsets branch from 23098d2 to f80fc76 Compare January 4, 2023 17:09
athanatos
athanatos requested changes Jan 4, 2023
View reviewed changes
Copy link
Contributor

@athanatos athanatos left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Close!

@github-actions
Copy link

github-actions bot commented Jan 11, 2023

This pull request can no longer be automatically merged: a rebase is needed and changes have to be manually resolved

@Matan-B Matan-B force-pushed the wip-matanb-crimson-calc-subsets branch from f80fc76 to c9874d7 Compare January 15, 2023 09:43
@Matan-B
Copy link
Contributor Author

Matan-B commented Jan 15, 2023

Rebased and updated TODO. (Still wip)

@github-actions
Copy link

github-actions bot commented Apr 27, 2023

This pull request can no longer be automatically merged: a rebase is needed and changes have to be manually resolved

@github-actions
Copy link

github-actions bot commented Jul 30, 2023

This pull request has been automatically marked as stale because it has not had any activity for 60 days. It will be closed if no further activity occurs for another 30 days.
If you are a maintainer or core committer, please follow-up on this pull request to identify what steps should be taken by the author to move this proposed change forward.
If you are the author of this pull request, thank you for your proposed contribution. If you believe this change is still appropriate, please ensure that any feedback has been addressed and ask for a code review.

@github-actions github-actions bot added the stale label Jul 30, 2023
@github-actions
Copy link

github-actions bot commented Aug 29, 2023

This pull request has been automatically closed because there has been no activity for 90 days. Please feel free to reopen this pull request (or open a new one) if the proposed change is still appropriate. Thank you for your contribution!

@github-actions github-actions bot closed this Aug 29, 2023
@Matan-B Matan-B reopened this Aug 30, 2023
@github-actions github-actions bot removed the stale label Aug 30, 2023
@github-actions
Copy link

github-actions bot commented Oct 29, 2023

This pull request has been automatically marked as stale because it has not had any activity for 60 days. It will be closed if no further activity occurs for another 30 days.
If you are a maintainer or core committer, please follow-up on this pull request to identify what steps should be taken by the author to move this proposed change forward.
If you are the author of this pull request, thank you for your proposed contribution. If you believe this change is still appropriate, please ensure that any feedback has been addressed and ask for a code review.

@github-actions github-actions bot added the stale label Oct 29, 2023
@athanatos athanatos self-requested a review October 30, 2023 05:55
@athanatos
Copy link
Contributor

athanatos commented Oct 30, 2023

@Matan-B Is this waiting on a review from me?

@github-actions github-actions bot removed the stale label Oct 30, 2023
@Matan-B
Copy link
Contributor Author

Matan-B commented Oct 30, 2023

@Matan-B Is this waiting on a review from me?

Missing unit tests here, I'll add tests and ask for a review.

@github-actions
Copy link

github-actions bot commented Dec 29, 2023

This pull request has been automatically marked as stale because it has not had any activity for 60 days. It will be closed if no further activity occurs for another 30 days.
If you are a maintainer or core committer, please follow-up on this pull request to identify what steps should be taken by the author to move this proposed change forward.
If you are the author of this pull request, thank you for your proposed contribution. If you believe this change is still appropriate, please ensure that any feedback has been addressed and ask for a code review.

@github-actions github-actions bot added the stale label Dec 29, 2023
@Matan-B Matan-B force-pushed the wip-matanb-crimson-calc-subsets branch from c9874d7 to 26a180f Compare December 31, 2023 15:58
@Matan-B Matan-B force-pushed the wip-matanb-crimson-calc-subsets branch 2 times, most recently from 444df4d to b242f92 Compare January 2, 2024 10:44
@Matan-B Matan-B marked this pull request as ready for review January 2, 2024 10:46
@Matan-B Matan-B requested a review from a team as a code owner January 2, 2024 10:46
@Matan-B Matan-B changed the title [WIP] crimson/osd/replicated_recovery_backend: Introduce calc_*_subsets crimson/osd/replicated_recovery_backend: Introduce calc_*_subsets Jan 2, 2024
@Matan-B
Copy link
Contributor Author

Matan-B commented Jan 2, 2024

@athanatos, ready for review now, thanks!

@Matan-B Matan-B changed the title crimson/osd/replicated_recovery_backend: Introduce calc_*_subsets crimson: Support Partial Object Recovery Jan 2, 2024
@Matan-B Matan-B force-pushed the wip-matanb-crimson-calc-subsets branch 2 times, most recently from 1f0e06e to e60b3df Compare January 2, 2024 15:44
@neha-ojha
Copy link
Member

neha-ojha commented Jan 3, 2024

@jianwei1216 FYI!

@athanatos athanatos added the wip-sjust-crimson-testing Test run pending by Sam label Jan 3, 2024
@github-actions
Copy link

github-actions bot commented Jan 4, 2024

This pull request can no longer be automatically merged: a rebase is needed and changes have to be manually resolved

Matan-B added 6 commits January 4, 2024 15:18
@Matan-B
crimson/osd/object_metadata_helper: Introduce calc_*_subsets
883f99d 
Signed-off-by: Matan Breizman <mbreizma@redhat.com>
@Matan-B
crimson/osd/replicated_recovery_backend: prepare_pull to support calc…
35e14f8 
…_clone_subsets

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
@Matan-B
rename prep_push to prep_push_to_replica
09b638c 
dismiss head_ctx

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
@Matan-B
crimson/osd/replicated_recovery_backend: seperate prep_push_to_replic…
869fc82 
…a and prep_push

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
@Matan-B
mson/osd/replicated_recovery_backend: Introduce prep_push_to_repli…
0cbe982 
…ca()

Intelligently push an object to a replica.  make use of existing
clones/heads and dup data ranges where possible.

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
@Matan-B
crimson/osd/replicated_recovery_backend: handle_pull_response() - rec…
1367490 
…alc_subsets()

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
@Matan-B Matan-B force-pushed the wip-matanb-crimson-calc-subsets branch from e60b3df to 5bd65a0 Compare January 4, 2024 15:37
@Matan-B
crimson/osd: Keep track of modified_ranges
b5df21f 
* `modifies_ranges` interval_set is added to osd_op_params_t
* keep track of modified_ranges while executing relevant ops
* Add `osd_op_params` parameter to `PGBackend::remove()`.

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
@Matan-B Matan-B force-pushed the wip-matanb-crimson-calc-subsets branch from 5bd65a0 to b5df21f Compare January 4, 2024 15:44
@athanatos
Copy link
Contributor

athanatos commented Jan 4, 2024

Test run looks good: https://pulpito.ceph.com/sjust-2024-01-04_00:41:39-crimson-rados-wip-sjust-crimson-testing-2024-01-03-distro-default-smithi/

@athanatos athanatos merged commit d451898 into ceph:main Jan 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@athanatos athanatos athanatos approved these changes

Assignees

No one assigned

Labels

build/ops crimson crimson-perf needs-crimson-qa tests wip-sjust-crimson-testing Test run pending by Sam

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@Matan-B @athanatos @neha-ojha