Skip to content

pacific: pybind/mgr/mirroring: directory snapshot mirror support#39050

Merged
vshankar merged 6 commits intoceph:pacificfrom
vshankar:wip-mgr-mirroring-interface-pacific
Feb 5, 2021
Merged

pacific: pybind/mgr/mirroring: directory snapshot mirror support#39050
vshankar merged 6 commits intoceph:pacificfrom
vshankar:wip-mgr-mirroring-interface-pacific

Conversation

@vshankar
Copy link
Contributor

Checklist

  • References tracker ticket
  • Updates documentation if necessary
  • Includes tests for new functionality or reproducer for bug

Show available Jenkins commands
  • jenkins retest this please
  • jenkins test classic perf
  • jenkins test crimson perf
  • jenkins test signed
  • jenkins test make check
  • jenkins test make check arm64
  • jenkins test submodules
  • jenkins test dashboard
  • jenkins test api
  • jenkins test docs
  • jenkins render docs
  • jenkins test ceph-volume all
  • jenkins test ceph-volume tox

@vshankar vshankar added the cephfs Ceph File System label Jan 25, 2021
@vshankar vshankar added this to the pacific milestone Jan 25, 2021
@vshankar
Copy link
Contributor Author

jenkins test make check

leseb added a commit to leseb/rook that referenced this pull request Jan 25, 2021
With Ceph Pacific, Rook can now deploy the cephfs-mirror daemon.
This initial commit covers the deployment of a single daemon only.
Multiple mirror daemons is currently untested.
Only a single mirror daemon is recommended.

The configuration of peers will come in a later PR since the mgr module
is still pending upstream: ceph/ceph#39050

The same goes for integration tests, they will get added later once we
start testing on Pacific.

Closes: rook#7002
Signed-off-by: Sébastien Han <seb@redhat.com>
leseb added a commit to leseb/rook that referenced this pull request Jan 25, 2021
With Ceph Pacific, Rook can now deploy the cephfs-mirror daemon.
This initial commit covers the deployment of a single daemon only.
Multiple mirror daemons is currently untested.
Only a single mirror daemon is recommended.

The configuration of peers will come in a later PR since the mgr module
is still pending upstream: ceph/ceph#39050

The same goes for integration tests, they will get added later once we
start testing on Pacific.

Closes: rook#7002
Signed-off-by: Sébastien Han <seb@redhat.com>
leseb added a commit to leseb/rook that referenced this pull request Jan 25, 2021
With Ceph Pacific, Rook can now deploy the cephfs-mirror daemon.
This initial commit covers the deployment of a single daemon only.
Multiple mirror daemons is currently untested.
Only a single mirror daemon is recommended.

The configuration of peers will come in a later PR since the mgr module
is still pending upstream: ceph/ceph#39050

The same goes for integration tests, they will get added later once we
start testing on Pacific.

Closes: rook#7002
Signed-off-by: Sébastien Han <seb@redhat.com>
leseb added a commit to leseb/rook that referenced this pull request Jan 25, 2021
With Ceph Pacific, Rook can now deploy the cephfs-mirror daemon.
This initial commit covers the deployment of a single daemon only.
Multiple mirror daemons is currently untested.
Only a single mirror daemon is recommended.

The configuration of peers will come in a later PR since the mgr module
is still pending upstream: ceph/ceph#39050

The same goes for integration tests, they will get added later once we
start testing on Pacific.

Closes: rook#7002
Signed-off-by: Sébastien Han <seb@redhat.com>
leseb added a commit to leseb/rook that referenced this pull request Jan 25, 2021
With Ceph Pacific, Rook can now deploy the cephfs-mirror daemon.
This initial commit covers the deployment of a single daemon only.
Multiple mirror daemons is currently untested.
Only a single mirror daemon is recommended.

The configuration of peers will come in a later PR since the mgr module
is still pending upstream: ceph/ceph#39050

The same goes for integration tests, they will get added later once we
start testing on Pacific.

Closes: rook#7002
Signed-off-by: Sébastien Han <seb@redhat.com>
leseb added a commit to leseb/rook that referenced this pull request Jan 25, 2021
With Ceph Pacific, Rook can now deploy the cephfs-mirror daemon.
This initial commit covers the deployment of a single daemon only.
Multiple mirror daemons is currently untested.
Only a single mirror daemon is recommended.

The configuration of peers will come in a later PR since the mgr module
is still pending upstream: ceph/ceph#39050

The same goes for integration tests, they will get added later once we
start testing on Pacific.

Closes: rook#7002
Signed-off-by: Sébastien Han <seb@redhat.com>
leseb added a commit to leseb/rook that referenced this pull request Jan 26, 2021
With Ceph Pacific, Rook can now deploy the cephfs-mirror daemon.
This initial commit covers the deployment of a single daemon only.
Multiple mirror daemons is currently untested.
Only a single mirror daemon is recommended.

The configuration of peers will come in a later PR since the mgr module
is still pending upstream: ceph/ceph#39050

The same goes for integration tests, they will get added later once we
start testing on Pacific.

Closes: rook#7002
Signed-off-by: Sébastien Han <seb@redhat.com>
leseb added a commit to leseb/rook that referenced this pull request Jan 26, 2021
With Ceph Pacific, Rook can now deploy the cephfs-mirror daemon.
This initial commit covers the deployment of a single daemon only.
Multiple mirror daemons is currently untested.
Only a single mirror daemon is recommended.

The configuration of peers will come in a later PR since the mgr module
is still pending upstream: ceph/ceph#39050

The same goes for integration tests, they will get added later once we
start testing on Pacific.

Closes: rook#7002
Signed-off-by: Sébastien Han <seb@redhat.com>
leseb added a commit to leseb/rook that referenced this pull request Jan 26, 2021
With Ceph Pacific, Rook can now deploy the cephfs-mirror daemon.
This initial commit covers the deployment of a single daemon only.
Multiple mirror daemons is currently untested.
Only a single mirror daemon is recommended.

The configuration of peers will come in a later PR since the mgr module
is still pending upstream: ceph/ceph#39050

The same goes for integration tests, they will get added later once we
start testing on Pacific.

Closes: rook#7002
Signed-off-by: Sébastien Han <seb@redhat.com>
leseb added a commit to leseb/rook that referenced this pull request Jan 26, 2021
With Ceph Pacific, Rook can now deploy the cephfs-mirror daemon.
This initial commit covers the deployment of a single daemon only.
Multiple mirror daemons is currently untested.
Only a single mirror daemon is recommended.

The configuration of peers will come in a later PR since the mgr module
is still pending upstream: ceph/ceph#39050

The same goes for integration tests, they will get added later once we
start testing on Pacific.

Closes: rook#7002
Signed-off-by: Sébastien Han <seb@redhat.com>
leseb added a commit to leseb/rook that referenced this pull request Jan 26, 2021
With Ceph Pacific, Rook can now deploy the cephfs-mirror daemon.
This initial commit covers the deployment of a single daemon only.
Multiple mirror daemons is currently untested.
Only a single mirror daemon is recommended.

The configuration of peers will come in a later PR since the mgr module
is still pending upstream: ceph/ceph#39050

The same goes for integration tests, they will get added later once we
start testing on Pacific.

Closes: rook#7002
Signed-off-by: Sébastien Han <seb@redhat.com>
leseb added a commit to leseb/rook that referenced this pull request Jan 27, 2021
With Ceph Pacific, Rook can now deploy the cephfs-mirror daemon.
This initial commit covers the deployment of a single daemon only.
Multiple mirror daemons is currently untested.
Only a single mirror daemon is recommended.

The configuration of peers will come in a later PR since the mgr module
is still pending upstream: ceph/ceph#39050

The same goes for integration tests, they will get added later once we
start testing on Pacific.

Closes: rook#7002
Signed-off-by: Sébastien Han <seb@redhat.com>
leseb added a commit to leseb/rook that referenced this pull request Jan 27, 2021
With Ceph Pacific, Rook can now deploy the cephfs-mirror daemon.
This initial commit covers the deployment of a single daemon only.
Multiple mirror daemons is currently untested.
Only a single mirror daemon is recommended.

The configuration of peers will come in a later PR since the mgr module
is still pending upstream: ceph/ceph#39050

The same goes for integration tests, they will get added later once we
start testing on Pacific.

Closes: rook#7002
Signed-off-by: Sébastien Han <seb@redhat.com>
@vshankar vshankar requested a review from mchangir January 27, 2021 13:33
@vshankar
Copy link
Contributor Author

jenkins test make check

leseb added a commit to leseb/rook that referenced this pull request Jan 28, 2021
With Ceph Pacific, Rook can now deploy the cephfs-mirror daemon.
This initial commit covers the deployment of a single daemon only.
Multiple mirror daemons is currently untested.
Only a single mirror daemon is recommended.

The configuration of peers will come in a later PR since the mgr module
is still pending upstream: ceph/ceph#39050

The same goes for integration tests, they will get added later once we
start testing on Pacific.

Closes: rook#7002
Signed-off-by: Sébastien Han <seb@redhat.com>
@github-actions
Copy link

This pull request can no longer be automatically merged: a rebase is needed and changes have to be manually resolved

@vshankar
Copy link
Contributor Author

vshankar commented Feb 1, 2021

@fullerdj suggested that I test this PR in teuthology and merge after getting an ACK. I'll be doing this today (w/ rebase).

@mchangir please take a look at the changes and ack if ok.

Copy link
Contributor

@mchangir mchangir left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good to me

@vshankar vshankar force-pushed the wip-mgr-mirroring-interface-pacific branch from 506d705 to 20d0d83 Compare February 2, 2021 11:45
@vshankar
Copy link
Contributor Author

vshankar commented Feb 2, 2021

rebased and updated.

fs suite run with mirroring,snap_schedule,volumes filter: https://pulpito.ceph.com/vshankar-2021-02-02_05:28:00-fs-wip-mgr-mirroring-pacific-20210201-225248-distro-basic-smithi/

Failures are in volumes related tests and do not seem to be related to this PR. @kotreshhr Appreciate if you could take a look at those.

@kotreshhr
Copy link
Contributor

rebased and updated.

fs suite run with mirroring,snap_schedule,volumes filter: https://pulpito.ceph.com/vshankar-2021-02-02_05:28:00-fs-wip-mgr-mirroring-pacific-20210201-225248-distro-basic-smithi/

Failures are in volumes related tests and do not seem to be related to this PR. @kotreshhr Appreciate if you could take a look at those.

  1. test_idempotency: This is known failure and fix [1] is pushed.
  2. test_async_subvolume_rm: This looks to me system issue. Lot of errors like Pipetimeout, ssh connection lost, socket timeout tracebacks are seen.
  3. test_subvolume_subvolumegroup_snapshot_name_conflict: This also looks to me system issue. 'No route to host', ssh connection lost traceback are seen.

[1] #39038

@vshankar
Copy link
Contributor Author

vshankar commented Feb 3, 2021

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 296e879)
Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit b7acf7f)
Also filter out client-id's starting with "mirror" when
cleaning leftover auth-ids since teuthology would be
configured to create client.mirror and client.mirror_remote
clients before executing mirroring tests.

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 8334bea)
Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit f81e8f1)
Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit 3478b2a)
Remove references to in-progress feature, remove internal interfaces
section and minor cleanups.

Signed-off-by: Venky Shankar <vshankar@redhat.com>
(cherry picked from commit bf19651)
@vshankar vshankar force-pushed the wip-mgr-mirroring-interface-pacific branch from 20d0d83 to d601bd4 Compare February 4, 2021 11:29
@vshankar
Copy link
Contributor Author

vshankar commented Feb 4, 2021

test run: https://pulpito.ceph.com/vshankar-2021-02-04_09:16:45-fs-wip-mgr-mirroring-pacific-20210203-234930-distro-basic-gibba/

Known failures in test_volume_client.

@mchangir -- Included the doc commit. Please take a look.

Copy link
Contributor

@mchangir mchangir left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good to me

@vshankar vshankar merged commit 643156d into ceph:pacific Feb 5, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants