I see a few paths forward with passing in more parameters for loss:

1. If we want to be fully generic, then I think loss_param could become a dictionary, which will be passed down into the loss directly. This way, other parameterized losses can be generically supported by this dictionary. We kind of already do this with metric_kwargs.

2. We can copy GradientBoostingRegressor's API and use a single parameter (alpha) that is just for quantile loss. In HistGradientBoosting* case, I would call it quantile instead of alpha.

3. (Likely out of reach in the near-term) Loss become public and users can pass create a PinballLoss(quantile=0.3) and pass it directly into HistGradient*.

I'd like to add the option:

4. loss_param is single parameter with different meaning depending on the loss, i.e. a quantile level for loss="quantile", and tweedie power for loss="tweedie" (in a later PR).

Comments:

1. loss_param dictionary
   This makes the clear distinction which loss is used with which parameter. As the loss parameter should not be used for hyperparameter tuning, this is ok.
2. Special parameter per loss.
   If we add further losses, this would clutter the API.
3. Pass a loss object instance
   To not do this and instead use simple parameter types like string and float was an original design decision for scikit-learn. It would, however, make live sometimes easier (meaning, I'm not opposed to this direction).

General: I really like consistency and would favor to use/adapt the choice we make here to other estimators, e.g. GradientBoostingRegressor, maybe one day DecisionTreeRegressor as well.

loss_param is single parameter with different meaning depending on the loss, i.e. a quantile level for loss="quantile", and tweedie power for loss="tweedie" (in a later PR).

I was thinking of losses with more than one parameter. Although, I think supporting multiple parameters can be a considered a non-issue. For advanced users, they can create their own loss object and pass it in. Kind of like how we unofficially support passing in a Criterion object into the trees:

As a starting point, I am okay with a single loss_param. I think the API discussion is worth bringing up during the next dev meeting.

I'm not sure we already have a design similar to the proposed loss_param in scikit-learn, do we?

Do we already have a clear idea of what future losses we want to support and what parameters they would need, apart from Tweedie? If we don't, using loss_param instead of the more consistent and simpler quantile parameter might be solving a problem we don't really have.

one reason to go towards loss_param be a dictionary is to support losses with more than two parameters. Although traditional losses e.g. epsilon insensitive loss, huber loss have also only one param maybe this can be a restriction in the future?

I'm not sure we already have a design similar to the proposed loss_param in scikit-learn, do we?

Similar to this is param in GenericUnivariateSelect but it's always seemed quite ugly to me!

From the dev meeting, I think the most agreeable and quickest path forward is to change loss_param toquantile and only accept a single float.