Skip to content

Feature/information criteria get_fitted_params#942

Merged
fkiraly merged 6 commits intosktime:mainfrom
ltsaprounis:feature/information-criteria-get_fitted_params
Jun 21, 2021
Merged

Feature/information criteria get_fitted_params#942
fkiraly merged 6 commits intosktime:mainfrom
ltsaprounis:feature/information-criteria-get_fitted_params

Conversation

@ltsaprounis
Copy link
Copy Markdown
Contributor

@ltsaprounis ltsaprounis commented Jun 10, 2021

Reference Issues/PRs

Fixes #931

What does this implement/fix? Explain your changes.

Adding the available information criteria (AIC, AICc, BIC, HQIC) for various forecasting algorithms.
TBATS and BATS did not have a get_fitted_params method so I added it in the adapter there. Maybe we'll want to add additional parameters returned there.

Does your contribution introduce a new dependency? If yes, which one?

No

What should a reviewer concentrate their feedback on?

Is get_fitted_params the right place to return those? maybe a similar function e.g. get_fit_results is more appropriate.

PR checklist

For all contributions
  • I've added myself to the list of contributors.
  • Optionally, I've updated sktime's CODEOWNERS to receive notifications about future changes to these files.
  • I've added unit tests and made sure they pass locally.

@fkiraly fkiraly self-requested a review June 11, 2021 08:23
Copy link
Copy Markdown
Collaborator

@fkiraly fkiraly left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice! Looks good from my side. Must have been fiddly to find where the parameters are...

@fkiraly
Copy link
Copy Markdown
Collaborator

fkiraly commented Jun 11, 2021

Regarding this question:

Is get_fitted_params the right place to return those? maybe a similar function e.g. get_fit_results is more appropriate.

A very valid point - do we want to separate fitted parameters from things like summaries or inspection?

The MLJ package, where I've co-designed the interfaces, does separate this, and I think it's a good idea - see
https://alan-turing-institute.github.io/MLJ.jl/stable/machines/#Inspecting-machines

During the dev days later this month, we'll probably refactor the interfaces, and this may be one thing we add - feel free to join.
I've also opened an issue #944 so we don't forget.

@fkiraly
Copy link
Copy Markdown
Collaborator

fkiraly commented Jun 12, 2021

@mloning, have you had a look at this?
This affects the relatively new get_fitted_params interface, would like to have a 2nd opinion.

@mloning
Copy link
Copy Markdown
Contributor

mloning commented Jun 12, 2021

I agree @fkiraly, fitted parameters are different from summary statistics of the model, so suggest to create a separate interface for them.

@ltsaprounis
Copy link
Copy Markdown
Contributor Author

Thanks @mloning and @fkiraly. I think this is the right call, it felt wrong to put AIC etc. in the same place as the fitted model parameters. Do you think I should keep going with this PR and implement the function to get summary statistics here? or is it more appropriate to do it in a separate PR (maybe linked to #944 ).

@mloning
Copy link
Copy Markdown
Contributor

mloning commented Jun 17, 2021

I suggest to work on the interface first in a separate PR, before we start adding it to concrete implementations.

@fkiraly
Copy link
Copy Markdown
Collaborator

fkiraly commented Jun 17, 2021

hm, @mloning, I would suggest we merge this and then perhaps later move it to the other interface - or this nice contribution may be buried under refactors and commits on top?

Rationale being, moving it from get_fitted_params to get_report_params or similar is almost trivial once we have the interface point, while finding the PR again, updating it, and remembering to do this later may not be easy.

@fkiraly
Copy link
Copy Markdown
Collaborator

fkiraly commented Jun 19, 2021

2 approvals, so can be merged now.

@fkiraly
Copy link
Copy Markdown
Collaborator

fkiraly commented Jun 20, 2021

@ltsaprounis, I think @mloning and I have aligned that we'll merge this and then later re-factor along #971.

Would you like to merge?
Minor suggestion, please add the names of the classes you've changed in the squashed commit message.

@fkiraly fkiraly merged commit 811a9fe into sktime:main Jun 21, 2021
@fkiraly
Copy link
Copy Markdown
Collaborator

fkiraly commented Jun 21, 2021

merged due to dependencies and 2 approvals

@ltsaprounis
Copy link
Copy Markdown
Contributor Author

Sorry @fkiraly I was on holidays for the past few days and nowhere near my laptop. Just realised Github has an "out of office" option...

Looks good to me, thanks for merging!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Exposing AIC, BIC and HQIC from sktime models via get_fitted_params

3 participants