Fix partially unknown Mat#23927
Fix partially unknown Mat#23927asmorkalov merged 1 commit intoopencv:4.xfrom Avasam:partially-unknown-mat
Conversation
| from typing import TYPE_CHECKING, Any | ||
|
|
||
| # Type subscription is not possible in python 3.8 | ||
| if TYPE_CHECKING: |
There was a problem hiding this comment.
Why this check isn't based on NumPy and Python versions like in PR 23838?
There was a problem hiding this comment.
There was a problem hiding this comment.
https://peps.python.org/pep-0484/#version-and-platform-checking
Type checkers are expected to understand simple version and platform checks [...]
If I was to include a numpy version check, static type checkers would see this as two possible assignements of the _NDArray type alias and their exact behaviour will differ with implementation.
I could probably do a Python version check, but a TYPE_CHECKING check was just simpler, works across all versions, and clearly shows that this condition exists purely for static type-checking reasons.
There was a problem hiding this comment.
It's a bit unclear to me on how type checker would behave if library doesn't provide such type (NumPy version check) or/and type subscription is not possible (Python version check).
Can you elaborate on this if you are familiar with it?
There won't be a state when there are 2 alias definitions, because libraries and interpreter versions are known and fixed.
There was a problem hiding this comment.
Can you elaborate on this if you are familiar with it?
Sure thing! I'll provide some examples too with the different scenarios.
It's a bit unclear to me on how type checker would behave if library doesn't provide such type (NumPy version check) or/and type subscription is not possible (Python version check).
Basically in a type-checking-only context (like stubs or a simple TYPE_CHECKING check), type checkers can use the full latest typing ecosystem, because runtime does not matter.*
So by using TYPE_CHECKING instead of sys.version_info >= (3, 9), we're able to let the tooling know the full type for all versions without being restricted by runtime.
With a python version check:
if sys.version_info >= (3, 9):
_NDArray = np.ndarray[Any, np.dtype[np.generic]]
else:
_NDArray = np.ndarray
reveal_type(_NDArray)PS E:\Users\Avasam\Documents\Git\opencv> pyright .\modules\core\misc\python\package\mat_wrapper\__init__.py --pythonversion=3.8
E:\Users\Avasam\Documents\Git\opencv\modules\core\misc\python\package\mat_wrapper\__init__.py
E:\Users\Avasam\Documents\Git\opencv\modules\core\misc\python\package\mat_wrapper\__init__.py:17:13 - information: Type of "_NDArray" is "Type[ndarray[Unknown, Unknown]]"
0 errors, 0 warnings, 1 information
PS E:\Users\Avasam\Documents\Git\opencv> pyright .\modules\core\misc\python\package\mat_wrapper\__init__.py --pythonversion=3.9
E:\Users\Avasam\Documents\Git\opencv\modules\core\misc\python\package\mat_wrapper\__init__.py
E:\Users\Avasam\Documents\Git\opencv\modules\core\misc\python\package\mat_wrapper\__init__.py:17:13 - information: Type of "_NDArray" is "Type[ndarray[Any, dtype[generic]]]"
As you can see, when type-checking against python 3.8, you loose generic type information, because with the version check you are also telling the type-checking to use the non-generic definition for 3.8.
With a TYPE_CHECKING check:
if TYPE_CHECKING:
_NDArray = np.ndarray[Any, np.dtype[np.generic]]
else:
_NDArray = np.ndarray
reveal_type(_NDArray)PS E:\Users\Avasam\Documents\Git\opencv> pyright .\modules\core\misc\python\package\mat_wrapper\__init__.py --pythonversion=3.8
E:\Users\Avasam\Documents\Git\opencv\modules\core\misc\python\package\mat_wrapper\__init__.py
E:\Users\Avasam\Documents\Git\opencv\modules\core\misc\python\package\mat_wrapper\__init__.py:17:13 - information: Type of "_NDArray" is "Type[ndarray[Any, dtype[generic]]]"
0 errors, 0 warnings, 1 information
Now even Python 3.8 users have the full type!
There won't be a state when there are 2 alias definitions, because libraries and interpreter versions are known and fixed.
That is true at runtime, and it is true statically for simple platform/version checks. But type-checkers don't understand something like library version checks.
In this example below I'll use my editor and the Pylance language server to visually show that the "else" code is considered unreachable at runtime (hence type checkers understand simple checks) and demo the type revealed by mypy and pyright:
Non-simple numpy version check:

See how _NDArray isn't grayed out in the second example, and how both type checkers now show a different type? Because it's up to the checker's implementation:
- mypy keeps the first definition and shows an error on the second one

- pyright made a union of both
ndarray. We're back to square 1 with lost generic types. But now also doesn't consider_NDArraya class (because it's a union)

- Other checkers may act differently (pytype, pyre, PyCharm, etc.)
Finally, what if I install numpy v1.19.5 ?
First, runtime isn't affected, the TYPE_CHECKING check still hold and the else clause doesn't use generics. If we only had a simple python version check it would fail at runtime, and I already explained why a numpy version check is problematic

For the type checkers, since there's no type information about np.ndarray, we're back to Any/Unknown, which is exactly the same as before this PR, so no change there.
* There's a tiny exception with checkers written in python that are run using a Python version with older AST and not special-casing it. ie: When running mypy using python 3.7, it won't understand PEP 570's positional-only syntax using /. This isn't an issue if targeting python 3.7 syntax whilst running mypy on python 3.8 for example.
It still wouldn't affect runtime, just the problematic type-checker.
There was a problem hiding this comment.
Well it got merged, that's fine I'll do it in a different PR and reference this one.

Closes #23780 by specifying
ndarray's generic types. The use ofTYPE_CHECKINGand a type alias allows this to keep compatibility with all Python versions.Pull Request Readiness Checklist
See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request
Patch to opencv_extra has the same branch name.