mgr/cephadm: streamline rgw deployment by Daniel-Pivonka · Pull Request #36162 · ceph/ceph

Daniel-Pivonka · 2020-07-17T16:37:28Z

Streamline the rgw deployment process for cephadm:

Currently to deploy rgw you first:

need a healthy cluster (ceph -s -> HEALTH_OK)
create a realm (radosgw-admin realm create --rgw-realm=*<realm-name>* --default)
create a zonegroup (radosgw-admin zonegroup create --rgw-zonegroup=default --master --default)
create a zone (radosgw-admin zone create --rgw-zonegroup=default --rgw-zone=*<zone-name>* --master --default)

then finally deploy the rgw service (ceph orch apply rgw *<realm-name>* *<zone-name>* --placement=*<placement>*)

To streamline this process cephadm handles those steps:

ceph orch apply rgw *<realm-name>* *<zone-name>* --placement=*<placement>* (this one command handles everything)

when a user creates a rgw service they supply the realm and zone they want
and then before creating the rgw service cephadm can:

check the cluster health
creates the realm if needed
create a zonegroup if needed (this is always 'default' for now)
create a zone if needed

then deploy the rgw service like normal

Fixes: https://tracker.ceph.com/issues/43681
Signed-off-by: Daniel-Pivonka dpivonka@redhat.com

mgfritch · 2020-07-17T20:28:53Z

move this logic down into the bin/cephadm deploy .. command?

and then use the rgw keyring for these commands?

something like this inside the rgw container:

$ cat /etc/ceph/keyring               
[client.rgw.realm.zone.subcluster.host1.lxwzqu]
        key = AQBcyRBfcYdDDRAAtSFGuepz+xbxtMunaMxYSQ==


$ radosgw-admin --user rgw.realm.zone.subcluster.host1.lxwzqu realm create --rgw-realm=realm --default
{   
    "id": "5e755f13-9820-4c67-93fb-552772fe7686",
    "name": "realm",
    "current_period": "8f94490f-8632-4336-bd4c-adf219e70c99",
    "epoch": 1
}

mgfritch

add a rgw generate_config function for the config-json...?

similar to this:

ceph/src/pybind/mgr/cephadm/services/cephadmservice.py

Lines 84 to 97 in e51d0e0

    
           def generate_config(self, daemon_spec: CephadmDaemonSpec) -> Tuple[Dict[str, Any], List[str]]: 
        
               # Ceph.daemons (mon, mgr, mds, osd, etc) 
        
               cephadm_config = self.mgr._get_config_and_keyring( 
        
                   daemon_spec.daemon_type, 
        
                   daemon_spec.daemon_id, 
        
                   host=daemon_spec.host, 
        
                   keyring=daemon_spec.keyring, 
        
                   extra_ceph_config=daemon_spec.extra_config.pop('config', '')) 
        
               if daemon_spec.extra_config: 
        
                   cephadm_config.update({'files': daemon_spec.extra_config}) 
        
               return cephadm_config, []

src/pybind/mgr/cephadm/services/cephadmservice.py

sebastian-philipp · 2020-07-20T08:48:15Z

@cbodley we need to call radosgw-admin from the MGR to setup the realm etc. Do you have any other ideas other than extending the MGR keyring accordingly?

cbodley · 2020-07-20T16:08:50Z

Do you have any other ideas other than extending the MGR keyring accordingly?

i think it's reasonable to put it there, yeah

sebastian-philipp

looks great! (even though a bit awkward to call radosgw-admin in the mgr container)

src/pybind/mgr/cephadm/services/cephadmservice.py

sebastian-philipp · 2020-07-22T20:24:31Z

adding dashboard cause this is the base for a future dashboard integration

sebastian-philipp · 2020-07-28T14:38:51Z

______________________________________________________________________________________________________________________________ summary _______________________________________________________________________________________________________________________________
  py3: commands succeeded
  mypy: commands succeeded
  congratulations :)

locally on my system.

sebastian-philipp · 2020-07-28T14:39:01Z

jenkins test make check

smithfarm · 2020-07-28T15:23:53Z

Are you planning to add documentation to this PR, or will that be done via a separate PR?

Daniel-Pivonka · 2020-07-28T20:55:34Z

tests passed finally!
Added doc changes
made this a real pr and removed [WIP]

src/pybind/mgr/cephadm/services/cephadmservice.py

doc/cephadm/install.rst

src/pybind/mgr/cephadm/tests/test_cephadm.py

cephadm will create realm, zonegroup, and zone if needed before creating rgw service fixes: https://tracker.ceph.com/issues/43681 Signed-off-by: Daniel-Pivonka <dpivonka@redhat.com>

mgfritch

lgtm!

sebastian-philipp · 2020-07-30T09:21:27Z

https://pulpito.ceph.com/swagner-2020-07-30_07:40:52-rados:cephadm-wip-swagner-testing-2020-07-29-1507-distro-basic-smithi/

mgfritch reviewed Jul 17, 2020

View reviewed changes

src/pybind/mgr/cephadm/services/cephadmservice.py Outdated Show resolved Hide resolved

batrick added cephadm needs-review labels Jul 18, 2020

sebastian-philipp requested a review from cbodley July 20, 2020 08:46

sebastian-philipp added the rgw label Jul 20, 2020

sebastian-philipp added the needs-rebase label Jul 22, 2020

Daniel-Pivonka force-pushed the cephadm-43681 branch 2 times, most recently from 10a1e4d to 540fcb9 Compare July 22, 2020 20:16

Daniel-Pivonka removed the needs-rebase label Jul 22, 2020

sebastian-philipp reviewed Jul 22, 2020

View reviewed changes

src/pybind/mgr/cephadm/services/cephadmservice.py Outdated Show resolved Hide resolved

src/pybind/mgr/cephadm/services/cephadmservice.py Show resolved Hide resolved

src/pybind/mgr/cephadm/services/cephadmservice.py Outdated Show resolved Hide resolved

sebastian-philipp requested a review from a team July 22, 2020 20:24

Daniel-Pivonka force-pushed the cephadm-43681 branch 2 times, most recently from 8f76b20 to 35a5762 Compare July 27, 2020 22:25

sebastian-philipp added the needs-rebase label Jul 28, 2020

Daniel-Pivonka force-pushed the cephadm-43681 branch from 35a5762 to ce65cf7 Compare July 28, 2020 19:27

Daniel-Pivonka removed the needs-rebase label Jul 28, 2020

Daniel-Pivonka force-pushed the cephadm-43681 branch from ce65cf7 to a81b5e9 Compare July 28, 2020 20:53

Daniel-Pivonka marked this pull request as ready for review July 28, 2020 20:53

Daniel-Pivonka requested a review from a team as a code owner July 28, 2020 20:53

Daniel-Pivonka changed the title ~~[WIP] mgr/cephadm: streamline rgw deployment~~ mgr/cephadm: streamline rgw deployment Jul 28, 2020

mgfritch reviewed Jul 28, 2020

View reviewed changes

src/pybind/mgr/cephadm/services/cephadmservice.py Outdated Show resolved Hide resolved

src/pybind/mgr/cephadm/services/cephadmservice.py Outdated Show resolved Hide resolved

sebastian-philipp suggested changes Jul 29, 2020

View reviewed changes

doc/cephadm/install.rst Outdated Show resolved Hide resolved

src/pybind/mgr/cephadm/tests/test_cephadm.py Show resolved Hide resolved

sebastian-philipp added the wip-swagner-testing My Teuthology tests label Jul 29, 2020

mgr/cephadm: streamline rgw deployment

131001c

cephadm will create realm, zonegroup, and zone if needed before creating rgw service fixes: https://tracker.ceph.com/issues/43681 Signed-off-by: Daniel-Pivonka <dpivonka@redhat.com>

Daniel-Pivonka force-pushed the cephadm-43681 branch from a81b5e9 to 131001c Compare July 29, 2020 18:24

mgfritch approved these changes Jul 30, 2020

View reviewed changes

sebastian-philipp approved these changes Jul 30, 2020

View reviewed changes

sebastian-philipp merged commit 0f7c912 into ceph:master Jul 30, 2020

sebastian-philipp mentioned this pull request Aug 4, 2020

octopus: cephadm batch backport August (1) #36450

Merged

Daniel-Pivonka mentioned this pull request Sep 16, 2020

mgr/cephadm: more rgw streamlining #37203

Closed

	def generate_config(self, daemon_spec: CephadmDaemonSpec) -> Tuple[Dict[str, Any], List[str]]:
	# Ceph.daemons (mon, mgr, mds, osd, etc)
	cephadm_config = self.mgr._get_config_and_keyring(
	daemon_spec.daemon_type,
	daemon_spec.daemon_id,
	host=daemon_spec.host,
	keyring=daemon_spec.keyring,
	extra_ceph_config=daemon_spec.extra_config.pop('config', ''))


	if daemon_spec.extra_config:
	cephadm_config.update({'files': daemon_spec.extra_config})

	return cephadm_config, []

Conversation

Daniel-Pivonka commented Jul 17, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mgfritch commented Jul 17, 2020

Uh oh!

mgfritch left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

sebastian-philipp commented Jul 20, 2020

Uh oh!

cbodley commented Jul 20, 2020

Uh oh!

sebastian-philipp left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

sebastian-philipp commented Jul 22, 2020

Uh oh!

sebastian-philipp commented Jul 28, 2020

Uh oh!

sebastian-philipp commented Jul 28, 2020

Uh oh!

smithfarm commented Jul 28, 2020

Uh oh!

Daniel-Pivonka commented Jul 28, 2020

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

mgfritch left a comment

Choose a reason for hiding this comment

Uh oh!

sebastian-philipp commented Jul 30, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

Daniel-Pivonka commented Jul 17, 2020 •

edited

Loading

mgfritch left a comment •

edited

Loading