Skip to content

Multisite fixes to be merged into main#48936

Merged
soumyakoduri merged 4 commits intoceph:wip-rgw-multisite-reshard-reeffrom
soumyakoduri:wip-skoduri-multisite-reef
Nov 21, 2022
Merged

Multisite fixes to be merged into main#48936
soumyakoduri merged 4 commits intoceph:wip-rgw-multisite-reshard-reeffrom
soumyakoduri:wip-skoduri-multisite-reef

Conversation

@soumyakoduri
Copy link
Contributor

@soumyakoduri soumyakoduri commented Nov 17, 2022

Checklist

  • Tracker (select at least one)
    • References tracker ticket
    • Very recent bug; references commit where it was introduced
    • New feature (ticket optional)
    • Doc update (no ticket needed)
    • Code cleanup (no ticket needed)
  • Component impact
    • Affects Dashboard, opened tracker ticket
    • Affects Orchestrator, opened tracker ticket
    • No impact that needs to be tracked
  • Documentation (select at least one)
    • Updates relevant documentation
    • No doc update is appropriate
  • Tests (select at least one)
Show available Jenkins commands
  • jenkins retest this please
  • jenkins test classic perf
  • jenkins test crimson perf
  • jenkins test signed
  • jenkins test make check
  • jenkins test make check arm64
  • jenkins test submodules
  • jenkins test dashboard
  • jenkins test dashboard cephadm
  • jenkins test api
  • jenkins test docs
  • jenkins render docs
  • jenkins test ceph-volume all
  • jenkins test ceph-volume tox
  • jenkins test windows

@soumyakoduri
rgw/multisite: if remote datalog is trimmed prematurely, sync status …
4243004 
…will report that it's behind the remote's max-marker even if there are no more entries to sync for each behind shard. if we get an empty listing, remove that shard from behind_shards.

Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>
@github-actions github-actions bot added the rgw label Nov 17, 2022
@soumyakoduri soumyakoduri mentioned this pull request Nov 17, 2022
Merged
cbodley
cbodley approved these changes Nov 17, 2022
View reviewed changes
Copy link
Contributor

@cbodley cbodley left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good!

the debug logging commit for radosgw-admin seems a bit excessive though. do you think that's still necessary?

@soumyakoduri
Copy link
Contributor Author

soumyakoduri commented Nov 18, 2022

looks good!

the debug logging commit for radosgw-admin seems a bit excessive though. do you think that's still necessary?

For "behind shards" issue, reading local & remote markers helped me to grep through the logs and analyse few races (wrt marker updates) among the sync servers. The logs are set to level '30' so that it doesn't flood regular dev environment. That said I am okay to keep it only in the downstream builds.

@cbodley
Copy link
Contributor

cbodley commented Nov 18, 2022

For "behind shards" issue, reading local & remote markers helped me to grep through the logs and analyse few races (wrt marker updates) among the sync servers

all of that information should already be available from other commands like 'datalog status' on the source zone and 'data sync status' on the destination

Shilpa Jagannath and others added 3 commits November 21, 2022 14:08
@soumyakoduri
rgw/multisite: marker_tracker->finish() returns error
f64e85c 
Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>
@soumyakoduri
rgw/bucket-sync: flush marker_tracker when the lease is lost
95ef7f4 
Also clear objv before reading the bucket sync status.

Signed-off-by: Soumya Koduri <skoduri@redhat.com>
@soumyakoduri
rgw/multisite: clear objv before reading bucket sync status
2c8f011 
Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>
@soumyakoduri soumyakoduri force-pushed the wip-skoduri-multisite-reef branch from fd733ea to 2c8f011 Compare November 21, 2022 08:40
@soumyakoduri
Copy link
Contributor Author

soumyakoduri commented Nov 21, 2022

For "behind shards" issue, reading local & remote markers helped me to grep through the logs and analyse few races (wrt marker updates) among the sync servers

all of that information should already be available from other commands like 'datalog status' on the source zone and 'data sync status' on the destination

ok.. thanks!

@soumyakoduri
Copy link
Contributor Author

soumyakoduri commented Nov 21, 2022

Haven't seen any issues with simple sync tests. Merging into multisite branch.

@soumyakoduri soumyakoduri merged commit 263e930 into ceph:wip-rgw-multisite-reshard-reef Nov 21, 2022
@soumyakoduri soumyakoduri deleted the wip-skoduri-multisite-reef branch March 6, 2026 09:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@cbodley cbodley cbodley approved these changes

Assignees

No one assigned

Labels

rgw

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@soumyakoduri @cbodley