Skip to content

[MRG] ENH Allows private methods to be imported#15419

Closed
thomasjpfan wants to merge 11 commits intoscikit-learn:masterfrom
thomasjpfan:adds_private_methods_to_all
Closed

[MRG] ENH Allows private methods to be imported#15419
thomasjpfan wants to merge 11 commits intoscikit-learn:masterfrom
thomasjpfan:adds_private_methods_to_all

Conversation

@thomasjpfan
Copy link
Copy Markdown
Member

Reference Issues/PRs

Resolves #15415

What does this implement/fix? Explain your changes.

Adds methods and classes to __all__ dynamically and then import *

Any other comments?

@NicolasHug
Copy link
Copy Markdown
Member

ping when ready please!

@thomasjpfan
Copy link
Copy Markdown
Member Author

thomasjpfan commented Oct 31, 2019

@NicolasHug Ready now. Note when __all__ is not defined in a module, import * already imports everything.

Edit not true

@jnothman
Copy link
Copy Markdown
Member

jnothman commented Oct 31, 2019 via email

@thomasjpfan thomasjpfan changed the title [MRG] ENH Allows private methods to be imported [WIP] ENH Allows private methods to be imported Oct 31, 2019
@thomasjpfan thomasjpfan changed the title [WIP] ENH Allows private methods to be imported [MRG] ENH Allows private methods to be imported Oct 31, 2019
@thomasjpfan
Copy link
Copy Markdown
Member Author

Tested this PR with from imblearn.ensemble import BalancedBaggingClassifier and was able to import it with some FutureWarnings. (on imblearn 0.5.0)

@NicolasHug
Copy link
Copy Markdown
Member

Most checks are failing @thomasjpfan

Copy link
Copy Markdown
Member

@NicolasHug NicolasHug left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few comments.

I don't like that you are updating the new module's __all__.
Can't we just set the attributes of the generated module?

I think we should also consider the backport proposed by @rth and compare.

# THIS FILE WAS AUTOMATICALLY GENERATED BY deprecated_modules.py
from . import {new_module_name}

if {new_module_name}.__file__.endswith(".py"):
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what about cython files

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It turns out that only _ball_tree.pyx and _kd_tree.pyx defines __all__. Files such as _tree.pyx that does not define __all__ the from from sklearn.tree.tree import * actually imports everything, such as Tree.

In short, even without this PR, importing from the compiled modules works, such as from sklearn.tree.tree import Tree.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what about _ball_tree.pyx and _kd_tree.pyx then?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All the other functions inside those files where cdef so they were not able to be imported in the first place.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, cython files could. But I guess we don't care.

Comment on lines +276 to +277
if not hasattr({new_module_name}, '__all__'):
{new_module_name}.__all__ = []
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe avoid the condition with something like new_module.__all__ = getattr(new_module, '__all__', [])?

@thomasjpfan
Copy link
Copy Markdown
Member Author

I think we should also consider the backport proposed by @rth and compare.

We can't use pep 562 because its >=3.7, so the only option is adding to __all__ everywhere manually.

@NicolasHug
Copy link
Copy Markdown
Member

#15415 (comment)

@thomasjpfan
Copy link
Copy Markdown
Member Author

Ah okay, we can backport it.

@thomasjpfan
Copy link
Copy Markdown
Member Author

Backport alternative at #15431

@rth
Copy link
Copy Markdown
Member

rth commented Nov 4, 2019

#15431 was merged, so we can probably close this? Thanks @thomasjpfan !

@NicolasHug NicolasHug closed this Nov 4, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[RFC] Feedback after scikit-learn module privatization

4 participants