Skip to content

Add boolean in ParameterDescriptor to enforce parameters static typing#115

Closed
ivanpauno wants to merge 2 commits intomasterfrom
ivanpauno/enforce-parameter-type
Closed

Add boolean in ParameterDescriptor to enforce parameters static typing#115
ivanpauno wants to merge 2 commits intomasterfrom
ivanpauno/enforce-parameter-type

Conversation

@ivanpauno
Copy link
Copy Markdown
Member

Currently parameters can change of type any time, which can be avoided by using a custom parameter callback (e.g. here).
When a parameter is changed to an unexpected type, it will usually generate an exception in the code using the parameter (see for example ros2/rclcpp#802), which is undesired.

What users commonly expect is that a parameter not change its type, so it would be good to provide an easy way to ensure that.
The parameter descriptor already contains a type field, but that is being used to reflect the last type that was set.

The proposal of this PR is to add a field called static_typing, when true a parameter cannot be changed of type.
In that case, after the parameter is set to a type different than PARAMETER_NOT_SET, that type is not allowed to change.
Open questions:

  • Should it be possible to declare a parameter with static_typing=true and type=PARAMETER_NOT_SET?
    I would say no, an int parameter should have an int default value provided when declared, and the user override (--ros-args -p ...) should match the type of the default value. i.e.:
    // should raise, initialized with type PARAMETER_NOT_SET and requested static typing
    node->declare_parameter("an_int", rclcpp::ParameterValue(), descriptor_static_typing_true);
    // Okey, and it raises an exception if the user provides an override which is not an integer, e.g.: `-p an_int:=foo`
    node->declare_parameter("an_int", 5, descriptor_static_typing_true);
  • Should it be possible to undeclare a parameter when static_typing=true? IMO, no.
    Allowing that would actually make a parameter of type int behave as an optional<int>, and it would still require special care when getting the parameter to avoid errors.

Alternatives:

  • Add a uint8 allowed_type field in the parameter description, where PARAMETER_NOT_SET would allow dynamic typing. There's no big difference with the proposal above.
  • Use the old type field as the allowed_type field described above.
  • Add a uint8[<=10] allowed_types field (we could use a boolean mask instead of an array). This approach is more flexible, but 99% of the time the user just wants to allow a single type, so not supporting this case doesn't sound important (and can still be achieved with dynamic typing and an user provided parameter callback).

@ivanpauno ivanpauno added the enhancement New feature or request label Nov 5, 2020
@ivanpauno ivanpauno requested a review from jacobperron November 5, 2020 16:20
@ivanpauno ivanpauno self-assigned this Nov 5, 2020
@ivanpauno
Copy link
Copy Markdown
Member Author

ros2/rclcpp#1436 implements the above proposal except this point:

Should it be possible to declare a parameter with static_typing=true and type=PARAMETER_NOT_SET?
I would say no,

Though adding that is easy.

@ivanpauno ivanpauno requested a review from clalancette November 5, 2020 16:22
Copy link
Copy Markdown
Contributor

@clalancette clalancette left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Along with the other code to implement this, I think this is fine.

My one hesitation here is that making this change means that Galactic will not be able to communicate with Foxy and earlier (at least, for ParameterDescriptor). We don't typically guarantee that, but it does happen to mostly work between Dashing, Eloquent, and Foxy. That shouldn't prevent this from going in, but it is something we should note in the release notes.

bool read_only false

# If 'true', the parameter is not allowed to change its type.
bool static_typing true
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe:

Suggested change
bool static_typing true
bool statically_typed true

@ivanpauno ivanpauno requested a review from wjwwood November 10, 2020 16:59
@ivanpauno
Copy link
Copy Markdown
Member Author

My one hesitation here is that making this change means that Galactic will not be able to communicate with Foxy and earlier (at least, for ParameterDescriptor)

Yeah, wire compatibility is hard in this case.
I think that the only way to achieve that here would be to reuse the type field which is already part of the parameter description.

@gbiggs
Copy link
Copy Markdown
Member

gbiggs commented Nov 20, 2020

Should it be possible to declare a parameter with static_typing=true and type=PARAMETER_NOT_SET?
I would say no

I agree.

Should it be possible to undeclare a parameter when static_typing=true? IMO, no.

I agree with this, too.

@clalancette
Copy link
Copy Markdown
Contributor

Should it be possible to declare a parameter with static_typing=true and type=PARAMETER_NOT_SET?
I would say no

While I agree with this in principle, I think this will cause a lot of downstream headaches. For instance, I know that navigation2 does things like this: https://github.com/ros-planning/navigation2/blob/11b25a51981eb53db6a6a3601729ae85238d9241/nav2_dwb_controller/dwb_core/src/dwb_local_planner.cpp#L79 . That declares a parameter as NOT_SET by default, but expects the user to explicitly set this. I think that would run afoul of this limitation. So I think this is one we'll have to allow for pragmatic reasons.

Should it be possible to undeclare a parameter when static_typing=true? IMO, no.

I agree with this one.

Copy link
Copy Markdown
Contributor

@clalancette clalancette left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall, I agree with the idea and changing this descriptor. The details to sort out in my mind:

  1. The name of the parameter (I like my suggested name slightly better)
  2. What to do about allowing NOT_SET -> conversion
  3. The actual implementation.
  4. Adding some documentation to the Galactic release notes about this semantic change.

@ivanpauno
Copy link
Copy Markdown
Member Author

While I agree with this in principle, I think this will cause a lot of downstream headaches. For instance, I know that navigation2 does things like this: https://github.com/ros-planning/navigation2/blob/11b25a51981eb53db6a6a3601729ae85238d9241/nav2_dwb_controller/dwb_core/src/dwb_local_planner.cpp#L79 .

That doesn't seem like a problem, as they are already "failing" when a default value is not provided.
So it seems to me that they are never declaring a parameter without a default value (i.e. type = NOT_SET).

@ivanpauno
Copy link
Copy Markdown
Member Author

The name of the parameter (I like my suggested name slightly better)

That sounds good to me.

Adding some documentation to the Galactic release notes about this semantic change.

Probably, it's worth creating a thread in ros discourse, though I would like first to have an implementation working with all the details sorted out.

@ivanpauno
Copy link
Copy Markdown
Member Author

@jacobperron friendly ping, what are your thoughts about this?

@jacobperron
Copy link
Copy Markdown
Member

That doesn't seem like a problem, as they are already "failing" when a default value is not provided.
So it seems to me that they are never declaring a parameter without a default value (i.e. type = NOT_SET).

I'm a bit confused by this point. I think the concern is valid. The nav stack defaults several parameters to NOT_SET and then is checking that the use provides a value. So, for this type of use-case, users will not be able to use statically typed parameters (unless we allow having an initial NOT_SET state). If users were able to detect if a default value was overridden, then they could use that signal as a workaround.

@jacobperron
Copy link
Copy Markdown
Member

Perhaps it's okay to disallow statically typed parameters that default to NOT_SET, but I think we want to be sure in that choice, since it looks like changing our minds later will require another change to ParameterDescriptor.msg.

@ivanpauno
Copy link
Copy Markdown
Member Author

The nav stack defaults several parameters to NOT_SET and then is checking that the use provides a value

I see, yes they seem to be sometimes not passing a default value.

IMHO, the use case we're missing is when you want a "statically typed" parameter with no default value.
In that case, I would expect to declare the parameter passing the parameter type, and if the user didn't provide an override the first "get" throws an exception (or maybe "declare" should throw (?)).

Something like:

// throws if an override is not provided, or maybe the first `get()` does and `declare` doesn't throw.
node->declare_parameter(param_name, descriptor, rclcpp::ParameterType::Integer);

or

// throws if an override is not provided, or maybe the first `get()` does and `declare` doesn't throw.
descriptor.type = ParameterType::Integer;
node->declare_parameter(param_name, descriptor);

We would need an allowed_types field in the descriptor, or reuse the existing type field differently as mentioned here.

@jacobperron
Copy link
Copy Markdown
Member

I like the allowed_types field. We could use a bitmask. This acts as a replacement for the statically_typed field, right?

@ivanpauno
Copy link
Copy Markdown
Member Author

I like the allowed_types field. We could use a bitmask.

I was actually thinking of a single type (i.e. allowed_type and not allowed_types, sorry for the misspelling), but an allowed_types bitmask is also an option.

This acts as a replacement for the statically_typed field, right?

Yes, we wouldn't need both.

@jacobperron
Copy link
Copy Markdown
Member

Maybe I'm wrong, but it seems like we would get a lot more flexibility for little effort in allowing multiple types. I think the API you described would look the same in the single allowed type case. E.g., only allow integers

node->declare_parameter(param_name, descriptor, rclcpp::ParameterType::Integer);

or allow integers and strings

node->declare_parameter(param_name, descriptor, rclcpp::ParameterType::Integer | rclcpp::ParameterType::String);

I'm okay if we don't support this, but seems like a nice way to further avoid user-defined parameter callbacks.

@ros-discourse
Copy link
Copy Markdown

This pull request has been mentioned on ROS Discourse. There might be relevant details there:

https://discourse.ros.org/t/feedback-required-should-parameters-be-able-to-change-of-type/18319/1

@ivanpauno
Copy link
Copy Markdown
Member Author

ivanpauno commented Jan 7, 2021

I'm okay if we don't support this, but seems like a nice way to further avoid user-defined parameter callbacks.

Yes, I don't mind strongly.
IMO, it looks like a bit of an overkill because I don't think much people is going to use the feature, but it's also not hard to support it.

@jacobperron
Copy link
Copy Markdown
Member

jacobperron commented Jan 7, 2021

It also might help us avoid a future change to the message definition if we decided to support it later.

@StefanFabian
Copy link
Copy Markdown

or allow integers and strings

What's the use case for a parameter that can have such vastly different types?
Can't think of one that wouldn't be bad code design but maybe I'm just not thinking hard enough. It's Friday after all :)

@jacobperron
Copy link
Copy Markdown
Member

jacobperron commented Jan 8, 2021

What's the use case for a parameter that can have such vastly different types?

Good point. I can't think of any specific cases. And I agree it would be bad design. I think I was under the wrong impression that people were currently using parameters with dynamic types (since that's the current default behavior).

I change my mind and think a single "allowed type" is a good option.

…nfigure if duck typing is desired or not

Signed-off-by: Ivan Santiago Paunovic <ivanpauno@ekumenlabs.com>
Signed-off-by: Ivan Santiago Paunovic <ivanpauno@ekumenlabs.com>
@ivanpauno ivanpauno force-pushed the ivanpauno/enforce-parameter-type branch from 40804f9 to f381146 Compare January 11, 2021 19:49
@ivanpauno
Copy link
Copy Markdown
Member Author

I change my mind and think a single "allowed type" is a good option.

@jacobperron do you have any preference between the currently proposed "static_typing" boolean and "allowed_type" field?
(in both cases you can know the actual type by checking the "type" field)

I would like:

node->declare_parameter("an_int", 5);  // parameter_descriptor argument takes default value

to declare an integer parameter that cannot change of type.
In the case of an "static_typing" boolean, the default can be true (like the proposed PR here), and you don't have to modify the default parameter descriptor.
In the case of an "allowed_type" field if this behavior is desired, "declare_parameter" would need to overwrite the "allowed_type" field of the provided descriptor and to avoid that behavior we would need an extra flag "dynamic_typing" argument in declare_parameter.

Moreover in this case:

rcl_interfaces::msg::ParameterDescriptor descriptor;
descriptor.integer_range = /*set integer range to something here*/;
node->declare_parameter("an_int", 5, descriptor);

do you need to write descriptor.allowed_type = rclcpp::PARAMETER_TYPE_INT or is that implicit?
In the case of static_typing, no issue here as it defaults to true.

The only problem is when no default value is provided, that as commented above that seems to be used currently.
I think the options are:

  • In case we go ahead with the "static_typing" field, if the user didn't provide a default value we can either:
    • throw an exception
    • set the parameter to "PARAMETER_NOT_SET". "static_typing" starts being enforced when the first actual value is given (taking the type of the first value as the type to enforce).
  • In the case of "allowed_type":
    • throw an exception
    • set the parameter to "PARAMETER_NOT_SET". The first value being set must satisfy the allowed type (and after that it cannot be undeclared).

The details are trivial to implement, but I'm not sure what is the preferred way forward ...

@jacobperron
Copy link
Copy Markdown
Member

A static_typing field (defaulting to true) sounds okay to me.

In the case where no parameter default value is provided, I'd rather not throw an exception since it seems like a valid use-case to me i.e., enforce that the user make a decision on a value instead of letting them use a default. I could be persuaded otherwise though.

@ivanpauno
Copy link
Copy Markdown
Member Author

i.e., enforce that the user make a decision on a value instead of letting them use a default

I'm not sure what that exactly means.
The only possibility, if not throwing an exception, is to declare the parameter with type "PARAMETER_NOT_SET" and wait until it's set the first time to decide it's type (which is what ros2/rclcpp#1512 does).
But that might be confusing, because it will take the type of the provided override.

@ivanpauno
Copy link
Copy Markdown
Member Author

I'm working on an alternative of the currently opened PRs, in order to experiment what's the best way to modify the behavior without breaking current use cases. I think we can discuss the final details in the review process

@jacobperron
Copy link
Copy Markdown
Member

i.e., enforce that the user make a decision on a value instead of letting them use a default

I'm not sure what that exactly means.

I mean, I think it's a reasonable use-case to have a "required" parameter. In this case the parameter is not set, requiring the user to set it before it is used. Whether the type is set by the node author or inferred by the first override is another design decision that needs to be made. It would be nice for the node author to be able to specify the allowed type for an unset parameter, but I'm also okay with just inferring the type from the override to keep things simple.

@ivanpauno
Copy link
Copy Markdown
Member Author

Closing in favor of #118.

@ivanpauno ivanpauno closed this Feb 17, 2021
@ivanpauno ivanpauno deleted the ivanpauno/enforce-parameter-type branch February 22, 2021 19:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants