Use type hint for method declaration by himkt · Pull Request #8248 · chainer/chainer

himkt · 2019-10-04T17:34:35Z

[edited: 2019/10/08] I revised module list by running following command.
grep -v "=.\+# type: " **/*.py| grep "# type: "

This intended to exclude variable annotations. (#8248 (comment))

grep result

Details

chainer/backend.py:    # type: (types.DeviceSpec) -> Device
chainer/backends/_chainerx.py:        # type: (chainerx.Device) -> None
chainer/backends/cuda.py:    from cupy import ndarray  # type: ignore # NOQA
chainer/backends/cuda.py:    from cupy.cuda import Device  # type: ignore # NOQA
chainer/backends/cuda.py:    from cupy.cuda import Event  # type: ignore # NOQA
chainer/backends/cuda.py:    from cupy.cuda import Stream  # type: ignore # NOQA
chainer/backends/cuda.py:    class ndarray(object):  # type: ignore # for type testing
chainer/backends/cuda.py:            # type: () -> types.Shape
chainer/backends/cuda.py:            # type: () -> 'Device'
chainer/backends/cuda.py:            # type: (tp.Optional['Stream']) -> numpy.ndarray
chainer/backends/cuda.py:            # type: (numpy.ndarray, tp.Optional['Stream']) -> None
chainer/backends/cuda.py:    class Device(object):  # type: ignore # for type testing
chainer/backends/cuda.py:            # type: (tp.Optional[int]) -> None
chainer/backends/cuda.py:            # type: () -> 'Device'
chainer/backends/cuda.py:            # type: (*tp.Any) -> None
chainer/backends/cuda.py:    class Event(object):  # type: ignore # for type testing
chainer/backends/cuda.py:    class Stream(object):  # type: ignore # for type testing
chainer/backends/cuda.py:    # type: (tp.Optional[int]) -> Device
chainer/backends/cuda.py:    # type: (*ndarray) -> Device
chainer/backends/cuda.py:    # type: (tp.Optional[types.CudaDeviceSpec]) -> Device
chainer/backends/intel64.py:    from ideep4py import mdarray  # type: ignore # NOQA
chainer/backends/intel64.py:    class mdarray(object):  # type: ignore
chainer/device_resident.py:        # type: () -> types.Xp
chainer/device_resident.py:        # type: () -> 'DeviceResident'
chainer/device_resident.py:        # type: (...) -> 'DeviceResident'
chainer/device_resident.py:        # type: () -> 'DeviceResident'
chainer/device_resident.py:            device  # type: types.DeviceSpec
chainer/device_resident.py:        # type: (...) -> 'DeviceResident'
chainer/function.py:        # type: (Function) -> None
chainer/initializer.py:        # type: (tp.Optional[types.DTypeSpec]) -> None
chainer/initializer.py:        # type: (types.NdArray) -> None
chainer/initializers/__init__.py:    # type: (types.AbstractInitializer, types.ShapeSpec, types.Xp, types.DTypeSpec, types.DeviceSpec) -> types.NdArray  # NOQA
chainer/initializers/__init__.py:    # type: (tp.Optional[types.InitializerSpec]) -> types.AbstractInitializer # NOQA
chainer/iterators/multiprocess_iterator.py:from multiprocessing import sharedctypes  # type: ignore
chainer/link.py:    # type: (tp.Optional[tp.Any]) -> bool
chainer/link.py:    # type: (tp.Optional[tp.Any]) -> tp.Tuple[tp.Optional[types.ShapeSpec], types.DTypeSpec] # NOQA
chainer/link.py:        return value  # type: ignore
chainer/link.py:        # type: (**tp.Any) -> None
chainer/link.py:        # type: () -> collections.OrderedDict[str, chainer.LinkHook]
chainer/link.py:        # type: () -> int
chainer/link.py:        # type: () -> bool
chainer/link.py:        # type: () -> tp.Iterator[None]
chainer/link.py:        # type: (*tp.Any, **tp.Any) -> tp.Any # NOQA
chainer/link.py:        # type: (str, tp.Any) -> None
chainer/link.py:        # type: (str) -> None
chainer/link.py:        # type: (str, tp.Optional[types.ShapeSpec], types.DTypeSpec, tp.Optional[types.InitializerSpec]) -> None # NOQA
chainer/link.py:        # type: (str, tp.Any) -> None
chainer/link.py:        # type: (str) -> None
chainer/link.py:        # type: (str) -> 'Link'
chainer/link.py:        # type: (bool) -> tp.Iterator[chainer.Parameter]
chainer/link.py:        # type: (bool) -> tp.Iterator[tp.Tuple[str, chainer.Parameter]]
chainer/link.py:        # type: (bool) -> tp.Iterator['Link']
chainer/link.py:        # type: (bool) -> tp.Iterator[tp.Tuple[str, 'Link']]
chainer/link.py:        # type: () -> tp.Iterator['Link']
chainer/link.py:        # type: ('Link', bool) -> None
chainer/link.py:        # type: () -> None
chainer/link.py:        # type: () -> None
chainer/link.py:        # type: ('Link') -> None
chainer/link.py:        # type: () -> None
chainer/link.py:        # type: () -> None
chainer/link.py:        # type: () -> bool
chainer/link.py:        # type: (chainer.AbstractSerializer) -> None
chainer/link.py:        # type: (int, str) -> chainer.Sequential
chainer/link.py:        # type: () -> int
chainer/link.py:        # type: (chainer.LinkHook, tp.Optional[str]) -> 'Link'
chainer/link.py:        # type: (str) -> None
chainer/link.py:        # type: (**Link) -> None
chainer/link.py:        # type: (str) -> tp.Any
chainer/link.py:        # type: (str, tp.Any) -> None
chainer/link.py:        # type: (str) -> None
chainer/link.py:        # type: (str, Link) -> None
chainer/link.py:        # type: (str) -> 'Chain'
chainer/link.py:        for name in ret._children:  # type: ignore
chainer/link.py:        return ret  # type: ignore
chainer/link.py:        # type: (bool) -> tp.Iterator[chainer.Parameter]
chainer/link.py:        # type: (bool) -> tp.Iterator[tp.Tuple[str, chainer.Parameter]]
chainer/link.py:        # type: (bool) -> tp.Iterator[Link]
chainer/link.py:        # type: (bool) -> tp.Iterator[tp.Tuple[str, Link]]
chainer/link.py:        # type: () -> tp.Iterator[Link]
chainer/link.py:        # type: (Link, bool) -> None
chainer/link.py:        # type: (Link) -> None
chainer/link.py:        # type: (chainer.AbstractSerializer) -> None
chainer/link.py:        # type: (*Link) -> None
chainer/link.py:        # type: (str, tp.Any) -> None
chainer/link.py:        # type: (tp.Union[int, slice], tp.Union[Link, tp.Iterable[Link]]) -> None # NOQA
chainer/link.py:            for i, c in enumerate(self._children):  # type: ignore
chainer/link.py:        # type: (tp.Union[int, slice]) -> None
chainer/link.py:        # type: (int, Link) -> None
chainer/link.py:        # type: () -> tp.Iterator[Link]
chainer/link.py:        # type: () -> int
chainer/link.py:        # type: (Link) -> None
chainer/link.py:        # type: (str) -> 'ChainList'
chainer/link.py:        return ret  # type: ignore
chainer/link.py:        # type: (bool) -> tp.Iterator[chainer.Parameter]
chainer/link.py:        # type: (bool) -> tp.Iterator[tp.Tuple[str, chainer.Parameter]]
chainer/link.py:        # type: (bool) -> tp.Iterator[Link]
chainer/link.py:        # type: (bool) -> tp.Iterator[tp.Tuple[str, Link]]
chainer/link.py:        # type: () -> tp.Iterator[Link]
chainer/link.py:        # type: (Link, bool) -> None # link is actually a ChainList
chainer/link.py:            child.copyparams(link[idx], copy_persistent)  # type: ignore
chainer/link.py:        # type: (Link) -> None # link is actually a ChainList
chainer/link.py:            child.addgrads(link[idx])  # type: ignore
chainer/link.py:        # type: (chainer.AbstractSerializer) -> None
chainer/link_hook.py:        # type: ('chainer.link.Link', str, tp.Tuple[tp.Any, ...], tp.Dict[str, tp.Any]) -> None # NOQA
chainer/link_hook.py:        # type: ('chainer.link.Link', str, tp.Tuple[tp.Any, ...], tp.Dict[str, tp.Any], tp.Any) -> None # NOQA
chainer/link_hook.py:        # type: () -> LinkHook
chainer/link_hook.py:        # type: (tp.Optional['chainer.link.Link']) -> None
chainer/link_hook.py:        # type: (tp.Optional['chainer.link.Link']) -> None
chainer/link_hook.py:        # type: (_ForwardPreprocessCallbackArgs) -> None
chainer/link_hook.py:        # type: (_ForwardPostprocessCallbackArgs) -> None
chainer/links/connection/linear.py:        # type: (tp.Optional[int], tp.Optional[int], bool, tp.Optional[types.InitializerSpec], tp.Optional[types.InitializerSpec]) -> None # NOQA
chainer/links/connection/linear.py:        # type: (int) -> None
chainer/links/connection/linear.py:        self.W.initialize((self.out_size, in_size))  # type: ignore
chainer/links/connection/linear.py:        # type: (variable.Variable, int) -> variable.Variable
chainer/testing/parameterized.py:        *parameters  # type: tp.Iterable[tp.Dict[str, tp.Any]]
chainer/testing/parameterized.py:    # type: (...) -> tp.Iterator[tp.Dict[str, tp.Any]]
chainer/testing/parameterized.py:    # type: (tp.Sequence[int]) -> tp.Iterator[tp.Tuple[int, ...]]
chainer/types.py:        # type: (NdArray) -> None
chainer/variable.py:        # type: (Variable, tp.Optional[str], **tp.Any) -> None
chainer/variable.py:        # type: (tp.Optional[types.NdArray], **tp.Any) -> None
chainer/variable.py:            self._set_chainerx_array(data, grad)  # type: ignore
chainer/variable.py:        # type: (tp.Optional[chainerx.ndarray], tp.Optional[chainerx.ndarray]) -> None # NOQA
chainer/variable.py:        # type: () -> tp.Optional[types.Xp]
chainer/variable.py:        # type: () -> tp.Optional[types.NdArray]
chainer/variable.py:                    self._data[0].as_grad_stopped())  # type: ignore
chainer/variable.py:        # type: (tp.Optional[types.NdArray]) -> None
chainer/variable.py:                    and (d_old.is_backprop_required()  # type: ignore
chainer/variable.py:                         or d.is_backprop_required())):  # type: ignore
chainer/variable.py:            self._set_chainerx_array(d, None)  # type: ignore
chainer/variable.py:            self._node._update_data_info(d)  # type: ignore # _node doesn't have value when xp is chainerx # NOQA
chainer/variable.py:        # type: () -> tp.Optional[types.NdArray]
chainer/variable.py:        # type: (types.NdArray) -> None
chainer/variable.py:        # type: () -> tp.Optional[types.NdArray]
chainer/variable.py:        # type: (tp.Optional[types.NdArray]) -> None
chainer/variable.py:        # type: () -> tp.Optional["Variable"]
chainer/variable.py:        # type: (tp.Optional["Variable"]) -> None
chainer/variable.py:        # type: (tp.Optional[types.InitializerSpec], tp.Optional[types.ShapeSpec], tp.Optional[str]) -> None # NOQA
chainermn/communicators/communicator_base.py:        # type: () -> bool
docs/source/_napoleon_patch.py:        # type: (unicode) -> List[unicode]

ref. https://www.python.org/dev/peps/pep-0484/

himkt · 2019-10-06T12:40:32Z

[note]
I've noticed type hint for variables is available from Python3.6.
Therefore, I revert my commits and force pushed.

ref. https://www.python.org/dev/peps/pep-0526/

himkt · 2019-10-08T13:55:10Z

docs/source/_napoleon_patch.py


-    def _parse_attributes_section(self, section):
-        # type: (unicode) -> List[unicode]
+    def _parse_attributes_section(self, section: str) -> List[str]:


unicode -> str

I'm not sure if this is correct...:cry::bow:

I looks OK (not so sure though).

himkt · 2019-10-09T04:36:12Z

I added type hints for method declarations.
@niboshi Could you please take a look? 🙇

himkt · 2019-10-09T04:37:41Z

Note that CI jobs for Python2.7 (pfn-public-ci/chainer.py27and35.[cg]pu) may fail.

niboshi

Thanks!

chainer/backends/cuda.py

chainer/initializers/__init__.py

niboshi · 2019-10-09T13:38:23Z

Thank you for the fixes.
I will trigger CIs after chainer/chainer-test#523 will be merged.

niboshi · 2019-10-10T07:03:46Z

Jenkins, test this please

chainer-ci · 2019-10-10T07:57:37Z

Jenkins CI test (for commit a673492, target branch master) failed with status FAILURE.

niboshi · 2019-10-10T08:06:10Z

Could you check failures?

himkt · 2019-10-10T08:18:13Z

Sure, I'll check it.

himkt · 2019-10-10T13:16:54Z

Jenkins, test this please

chainer-ci · 2019-10-10T14:00:40Z

Jenkins CI test (for commit f79ea01, target branch master) succeeded!

himkt · 2019-10-10T14:55:22Z

flexCI, test this please

chainer-ci · 2019-10-10T15:35:54Z

Jenkins CI test (for commit f79ea01, target branch master) succeeded!

himkt · 2019-10-10T15:51:46Z

I couldn't launch flexCI...
@niboshi Could you please help me? 🙇

himkt · 2019-10-10T15:57:47Z

chainer/link_hook.py

        link_hooks[self.name].deleted(None)
        del link_hooks[self.name]

-    def added(self, link: tp.Optional['chainer.link.Link']) -> None:


Errors occurred a string ('chainer.link.Link') was evaluated as an ordinal class in Python3.5. 🤔
I avoided this error by surrounding a whole annotation with whitespace.

niboshi · 2019-10-11T01:15:45Z

flexCI, test this please

chainer-ci · 2019-10-11T02:45:41Z

Jenkins CI test (for commit f79ea01, target branch master) failed with status FAILURE.

niboshi · 2019-10-11T03:03:56Z

Jenkins, test this please

chainer-ci · 2019-10-11T04:24:48Z

Jenkins CI test (for commit f79ea01, target branch master) succeeded!

niboshi · 2019-10-11T04:55:36Z

Thank you!

himkt changed the title ~~Use type hint in link.py~~ Use type hint Oct 4, 2019

himkt force-pushed the use-type-hint branch from 9df3e44 to d59e7ed Compare October 6, 2019 12:38

asi1024 assigned niboshi Oct 7, 2019

himkt added 5 commits October 8, 2019 15:42

Use type hint in communicator_base.py

2e878cc

Use type hint in device_resident.py

b641e52

Use type hint in function.py

07e5d09

PEP8

17383e7

Use type hint in backend.py

fe603d7

himkt force-pushed the use-type-hint branch from bd44728 to fe603d7 Compare October 8, 2019 06:46

himkt added 11 commits October 8, 2019 15:49

Use type hint in _chainerx.py

69a9ba2

Use type hint in initializer.py

dd380e4

Use type hint in link_hook.py

b807bf4

Use type hint in parameterized.py

accdf30

Use type hint in linear.py

24165b9

Use type hint in types.py

dbf0181

Use type hint in variable.py

514f132

Use type hint in __init__.py (chainer/initializers)

95c4b92

Use type hint in cuda.py

05c145d

Use type hint in link.py

2f4ff46

Use type hint in _napoleon_patch.py

b350056

himkt commented Oct 8, 2019

View reviewed changes

himkt changed the title ~~Use type hint~~ Use type hint for method declaration Oct 9, 2019

himkt marked this pull request as ready for review October 9, 2019 04:34

niboshi reviewed Oct 9, 2019

View reviewed changes

chainer/backends/cuda.py Show resolved Hide resolved

chainer/initializers/__init__.py Outdated Show resolved Hide resolved

chainer/initializers/__init__.py Outdated Show resolved Hide resolved

Make some arguments Optional

a673492

niboshi added the cat:code-fix Code refactoring that does not change the behavior. label Oct 9, 2019

niboshi added the st:blocked-by-another-pr State indicating that another ticket is preventing this ticket from being closed/merged. label Oct 9, 2019

niboshi added st:test-and-merge State indicating that pull request is approved by a reviewer and can be merged after CI passes. and removed st:blocked-by-another-pr State indicating that another ticket is preventing this ticket from being closed/merged. labels Oct 10, 2019

niboshi added this to the v7.0.0rc1 milestone Oct 10, 2019

niboshi added st:awaiting-author State indicating that response is needed from contributors, often authors of pull request. and removed st:test-and-merge State indicating that pull request is approved by a reviewer and can be merged after CI passes. labels Oct 10, 2019

Suppress ImportError

f79ea01

himkt commented Oct 10, 2019

View reviewed changes

niboshi removed the st:awaiting-author State indicating that response is needed from contributors, often authors of pull request. label Oct 11, 2019

niboshi mentioned this pull request Oct 11, 2019

flaky test: chainer_tests/functions_tests/loss_tests/test_negative_sampling.py #8289

Closed

niboshi merged commit 744c33e into chainer:master Oct 11, 2019

knorth55 mentioned this pull request Dec 16, 2019

Do not install chainer>=7.0.0 in python2 #8517

Merged

Conversation

himkt commented Oct 4, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

grep result

Uh oh!

himkt commented Oct 6, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

himkt Oct 8, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

niboshi Oct 9, 2019

Choose a reason for hiding this comment

Uh oh!

himkt commented Oct 9, 2019

Uh oh!

himkt commented Oct 9, 2019

Uh oh!

niboshi left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

niboshi commented Oct 9, 2019

Uh oh!

niboshi commented Oct 10, 2019

Uh oh!

chainer-ci commented Oct 10, 2019

Uh oh!

niboshi commented Oct 10, 2019

Uh oh!

himkt commented Oct 10, 2019

Uh oh!

himkt commented Oct 10, 2019

Uh oh!

chainer-ci commented Oct 10, 2019

Uh oh!

himkt commented Oct 10, 2019

Uh oh!

chainer-ci commented Oct 10, 2019

Uh oh!

himkt commented Oct 10, 2019

Uh oh!

himkt Oct 10, 2019

Choose a reason for hiding this comment

Uh oh!

niboshi commented Oct 11, 2019

Uh oh!

chainer-ci commented Oct 11, 2019

Uh oh!

niboshi commented Oct 11, 2019

Uh oh!

chainer-ci commented Oct 11, 2019

Uh oh!

niboshi commented Oct 11, 2019

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

himkt commented Oct 4, 2019 •

edited

Loading

himkt commented Oct 6, 2019 •

edited

Loading

himkt Oct 8, 2019 •

edited

Loading