Skip to content

mgr/mgr_module.py: CLICommand: Fix parsing of kwargs arguments#36877

Merged
sebastian-philipp merged 1 commit intoceph:masterfrom
sebastian-philipp:mgr-module-fix-kwargs-cli-command
Aug 31, 2020
Merged

mgr/mgr_module.py: CLICommand: Fix parsing of kwargs arguments#36877
sebastian-philipp merged 1 commit intoceph:masterfrom
sebastian-philipp:mgr-module-fix-kwargs-cli-command

Conversation

@sebastian-philipp
Copy link
Contributor

Make parsing of keyword arguments a bit safer by
makeing sure the key is actually a valid parameter name

Fixes: https://tracker.ceph.com/issues/47185
Signed-off-by: Sebastian Wagner sebastian.wagner@suse.com

Checklist

  • References tracker ticket
  • Updates documentation if necessary
  • Includes tests for new functionality or reproducer for bug

Show available Jenkins commands
  • jenkins retest this please
  • jenkins test classic perf
  • jenkins test crimson perf
  • jenkins test signed
  • jenkins test make check
  • jenkins test make check arm64
  • jenkins test submodules
  • jenkins test dashboard
  • jenkins test api
  • jenkins test docs
  • jenkins render docs
  • jenkins test ceph-volume all
  • jenkins test ceph-volume tox

Make parsing of keyword arguments a bit safer by
makeing sure the key is actually a valid parameter name

Fixes: https://tracker.ceph.com/issues/47185
Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>
Copy link
Contributor

@jan--f jan--f left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One behaviour that this patch removes and needs to be re-added is that once a kwarg has been seen, all arguments after the inital must me considered kwargs too. Otherwise this messes with the order of positional arguments.

@sebastian-philipp
Copy link
Contributor Author

sebastian-philipp commented Aug 28, 2020

One behaviour that this patch removes and needs to be re-added is that once a kwarg has been seen, all arguments after the inital must me considered kwargs too. Otherwise this messes with the order of positional arguments.

I think this should work!

as soon as kwargs_switch is True, kwargs_switch is never set to False again. Would have been good to have test that verifies this.

@sebastian-philipp
Copy link
Contributor Author

On there other hand, there are a few things that I don't yet understand:

  1. You could do something like
    cmd = {
        'prefix': 'orch daemon add',
        'daemon_type': 'placement=smithi044:[v2:172.21.15.44:3301,v1:172.21.15.44:6790]=c'',
        'placement': 'daemon_type=mon`,
    }

which is a bit questionable.

  1. the arguments are always a dict already. Parsing the values as k-v args is in general a bit questionable.
  2. you're somewhat duplicating ceph_argparse.py here

@jan--f
Copy link
Contributor

jan--f commented Aug 28, 2020

On there other hand, there are a few things that I don't yet understand:

1. You could do something like
    cmd = {
        'prefix': 'orch daemon add',
        'daemon_type': 'placement=smithi044:[v2:172.21.15.44:3301,v1:172.21.15.44:6790]=c'',
        'placement': 'daemon_type=mon`,
    }

which is a bit questionable.

Somewhat contrived but yes. My solution would have been to require a flag when the argument is defined in the module. Consider

'name=fs,type=CephString,req=false '

if we'd require a flag like 'name=fs,type=CephString,req=false,kw=true ' for arguments that can be passed a kwargs, we'd skirt a lot of the issues.

1. the arguments are **always** a dict already. Parsing the values as k-v args is in general a bit questionable.

Motivation for this is for modules that have a lot of arguments that have default values. If a user wants to specify only one of those arguments, without the kwargs extension they would have to pass the preceding arguments in the right order. So this should add quite a bit of convenience for users.

2. you're somewhat duplicating ceph_argparse.py here

@sebastian-philipp
Copy link
Contributor Author

sebastian-philipp commented Aug 28, 2020

Motivation for this is for modules that have a lot of arguments that have default values. If a user wants to specify only one of those arguments, without the kwargs extension they would have to pass the preceding arguments in the right order. So this should add quite a bit of convenience for users.

Interesting. I'm facing the same issue with the orch commands. Especially monsters like

@_cli_write_command(
'orch apply rgw',
'name=realm_name,type=CephString '
'name=zone_name,type=CephString '
'name=subcluster,type=CephString,req=false '
'name=port,type=CephInt,req=false '
'name=ssl,type=CephBool,req=false '
'name=placement,type=CephString,req=false '
'name=dry_run,type=CephBool,req=false '
'name=format,type=CephChoices,strings=plain|json|json-pretty|yaml,req=false '
'name=unmanaged,type=CephBool,req=false',
'Update the number of RGW instances for the given zone')
def _apply_rgw(self,

with users passing

ceph orch apply rgw myrealm myzone count:3,label:rgws 

really does wrong things.

Anyway.

you want to make a new PR?

@jan--f
Copy link
Contributor

jan--f commented Aug 28, 2020

you want to make a new PR?

Sure thing.

@sebastian-philipp
Copy link
Contributor Author

@neha-ojha might want to add this PR to your batches, as rados/cephadm will be broken otherwise

@sebastian-philipp
Copy link
Contributor Author

@sebastian-philipp
Copy link
Contributor Author

Hm. An idea would be to get this merged now to fix the rados suite and then follow-up with a proper refactorization.

@jan--f would that work?

@tchaikov
Copy link
Contributor

@neha-ojha might want to add this PR to your batches, as rados/cephadm will be broken otherwise

@sebastian-philipp could you please do me a favor and ping me as well when we have such a failure in future?

@jan--f
Copy link
Contributor

jan--f commented Aug 31, 2020

Hm. An idea would be to get this merged now to fix the rados suite and then follow-up with a proper refactorization.

@jan--f would that work?

Sure or a revert would work too. I should have a proper fix later today.

@sebastian-philipp
Copy link
Contributor Author

sebastian-philipp commented Aug 31, 2020

Hm. ok then let's merge this and revert (while keeping the pytest) this commit in your PR?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants