[ENH] Add variance_factor to DummyProbaRegressor#860
Conversation
fkiraly
left a comment
There was a problem hiding this comment.
Thanks. Can you give an example in which this would be useful?
The main use case is creating tunable, conservative probabilistic baselines. In real-world applications (finance, supply chain, forecasting), future data is often more volatile than training data. The standard DummyProbaRegressor can be unrealistically confident. By setting variance_factor > 1, users can easily construct a baseline with wider predictoin levels. This provides a much stricter benchmark to beat for uncertainty metrics like CRPS or Log-Likelihood than a naive, unscaled dummy model. |
|
How would you set the |
In practice, the value of the variance_factor could be determined in these ways:
|
|
@fkiraly just checking in to see if there is any update on this? |
# Please enter a commit message to explain why this merge is necessary, # especially if it merges an updated upstream into a topic branch. # # Lines starting with '#' will be ignored, and an empty message aborts # the commit.
This PR adds a variance_factor option to the
DummyProbaRegressorwhen the underlying strategy is normal.Previously, the normal strategy correctly predicted a Normal distribution with the empirical standard deviation. However, probabilistic modeling often requires structural baselines where the prediction uncertainty must be explicitly hedged or aggressively reduced. This enhancement allows practitioners to natively scale the standard deviation to predict structural baselines where the variance is equal to$\sigma_{training}^2 \times \text{factor}$ . I ran validations locallyt and it both passed.
Resolves #7 , implementing a still unchecked baseline titled "always predict a Gaussian with mean = training mean, var = training var".