Remove references to deprecated NumPy type aliases. by vrabaud · Pull Request #22965 · opencv/opencv

vrabaud · 2022-12-15T10:23:11Z

This change replaces references to a number of deprecated NumPy type aliases (np.bool, np.int, np.float, np.complex, np.object, np.str) with their recommended replacement (bool, int, float, complex, object, str).

Those types were deprecated in 1.20 and are removed in 1.24, cf numpy/numpy#22607.

Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

I agree to contribute to the project under Apache 2 License.
To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
The PR is proposed to the proper branch
There is a reference to the original bug report and related work

vrabaud · 2022-12-15T10:23:44Z

Do we want to backport to 3.4 ? I am not sure on the Numpy version that is supported there.

asmorkalov · 2022-12-20T08:54:27Z

Yes, backport to 3.4 is useful too.

VadimLevin

It makes sense to keep these types in test_misc.py, but protect them with version check. Otherwise inaccurate refactoring of conversion code might break something for older versions of NumPy, because coverage is reduced.

vrabaud · 2022-12-20T09:41:04Z

@VadimLevin , how would you protect them with version check knowing that np.float (for example) is not defined anymore ?

VadimLevin · 2022-12-20T09:56:07Z

@VadimLevin , how would you protect them with version check knowing that np.float (for example) is not defined anymore ?

Just several thoughts:

numpy is a module and as consequence is a Python object.
Every Python object can be introspected with built-in hasattr/getattr functions.
When all deprecated type aliases will be removed (e.g. numpy.float) you will get AttributeError only when you access these attribute.
This cases might be moved to separate test functions that can be skipped using NumPy version checks or attribute checks in unittest.skipIf

vrabaud · 2022-12-20T12:44:24Z

I believe the build error is unrelated: https://pullrequest.opencv.org/buildbot/builders/precommit_windows64/builds/100389/steps/test_python2/logs/stdio

hawkinsp · 2022-12-20T16:05:45Z

Drive-by comment: you can probably just write bool, float, int etc., since the NumPy types (e.g., np.int etc.) have been aliases for the Python builtins for a very long time. I verified this at least as far back as NumPy 1.15 which is very old at this point.

vrabaud · 2022-12-20T16:43:23Z

@hawkinsp , do you have a link for that please ? It would make this PR simpler. Thx

hawkinsp · 2022-12-20T16:59:19Z

The release notes of NumPy 1.20 note these type aliases have existed for a "long time". I don't know how far back they go!

https://numpy.org/doc/stable/release/1.20.0-notes.html#using-the-aliases-of-builtin-types-like-np-int-is-deprecated

I personally verified the types are aliases as far back as NumPy 1.15.0 which was released in 2018:

Python 3.7.12 (default, Jan 31 2022, 14:15:00)
Type 'copyright', 'credits' or 'license' for more information
IPython 7.31.1 -- An enhanced Interactive Python. Type '?' for help.

In [1]: import numpy as np

In [2]: np.float is float
Out[2]: True

In [3]: np.__version__
Out[3]: '1.15.0'

I don't have an older copy of Python to try older NumPy versions, but I think it's safe to assume they existed before that.

So if I were you I think I'd be quite comfortable just writing float instead of np.float since they have been aliases into antiquity, and you probably shouldn't support NumPy versions older than that anyway (see https://numpy.org/neps/nep-0029-deprecation_policy.html )

vrabaud · 2022-12-20T17:03:22Z

Thx a bunch @hawkinsp ! @VadimLevin , can I revert to what I did initially ? Just np.* -> * ? Thx

asmorkalov · 2022-12-21T06:29:52Z

I checked OpenCV-Python package. The minimum supported Numpy version is 1.13.3. Python 3.6 + Numpy 1.13.3 handles array names without "np." suffix correctly:

Python 3.6.9 (default, Nov 25 2022, 14:10:45) 
[GCC 8.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy as np
>>> q = np.array([1], dtype=float)
>>> print(q)
[ 1.]
>>> np.__version__
'1.13.3'
>>>

So you can just remove package name from types.

VadimLevin · 2022-12-21T07:31:17Z

Thanks @hawkinsp and @asmorkalov. @vrabaud Yep, sounds good. So just remove duplicates from test cases e.g.4.3, np.float(2.5), np.float32(4.25) -> 4.3, np.float(4.25)

modules/python/test/test_misc.py

asmorkalov · 2022-12-21T10:23:18Z

There are more failed tests, if per-module tests are enabled. Command example:

python3.8 ../opencv-master/modules/python/test/test.py -v --repo=/home/alexander/Projects/OpenCV/opencv-master

test_aruco_detector (test_objdetect_aruco.aruco_objdetect_test) ... ERROR
test_aruco_detector_refine (test_objdetect_aruco.aruco_objdetect_test) ... ERROR
test_getDistanceToId (test_objdetect_aruco.aruco_objdetect_test) ... ERROR
test_identify (test_objdetect_aruco.aruco_objdetect_test) ... ERROR
test_idsAccessibility (test_objdetect_aruco.aruco_objdetect_test) ... ERROR
test_peopledetect (test_peopledetect.peopledetect_test) ... ok
test_detect (test_qrcode_detect.qrcode_detector_test) ... ok
test_detect_and_decode (test_qrcode_detect.qrcode_detector_test) ... ok
test_detect_and_decode_multi (test_qrcode_detect.qrcode_detector_test) ... ok
test_detect_multi (test_qrcode_detect.qrcode_detector_test) ... ok
test_simple (test_stitching.stitching_compose_panorama_args) ... ok
test_simple (test_stitching.stitching_compose_panorama_test_no_args) ... ok
test_simple (test_stitching.stitching_detail_test) ... ok
test_simple (test_stitching.stitching_matches_info_test) ... ok
test_simple (test_stitching.stitching_range_matcher_test) ... FAIL
test_simple (test_stitching.stitching_seam_finder_graph_cuts) ... FAIL
test_simple (test_stitching.stitching_test) ... ok
test_lk_homography (test_lk_homography.lk_homography_test) ... ok
test_lk_track (test_lk_track.lk_track_test) ... ok
test_createTracker (test_tracking.tracking_test) ... ok
test_add (test_gapi_core.gapi_core_test) ... ok
test_add_uint8 (test_gapi_core.gapi_core_test) ... ok
test_kmeans (test_gapi_core.gapi_core_test) ... ok
test_kmeans_2d (test_gapi_core.gapi_core_test) ... ERROR
test_kmeans_3d (test_gapi_core.gapi_core_test) ... ERROR

asmorkalov · 2022-12-21T10:30:52Z

Also strange failure in core things:

======================================================================
FAIL: test_native_method_can_be_patched (test_misc.CanUsePurePythonModuleFunction)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/mnt/projects/Projects/OpenCV/opencv-master/modules/python/test/test_misc.py", line 759, in test_native_method_can_be_patched
    self.assertTrue(isinstance(res, Sequence),
AssertionError: False is not true : Overwritten method should return sequence. Got: 10 of type <class 'int'>

asmorkalov · 2022-12-21T10:31:09Z

@VadimLevin could you take a look on it?

asmorkalov · 2022-12-21T10:36:38Z

@vrabaud I added couple of more fixes to your PR.

vrabaud · 2022-12-21T10:38:17Z

@asmorkalov , thx !

asmorkalov · 2022-12-22T08:52:32Z

@vrabaud Please ignore my previous message about errors. I fould yet another OpenCV-Python instance.

modules/gapi/misc/python/test/test_gapi_core.py

modules/objdetect/misc/python/test/test_objdetect_aruco.py

asmorkalov · 2022-12-23T10:34:36Z

@vrabaud @VadimLevin I fixed double/float issue and squashed commits. Please take a look ASAP.

modules/python/test/test_misc.py

This change replaces references to a number of deprecated NumPy type aliases (np.bool, np.int, np.float, np.complex, np.object, np.str) with their recommended replacement (bool, int, float, complex, object, str). Those types were deprecated in 1.20 and are removed in 1.24, cf numpy/numpy#22607.

asmorkalov requested a review from VadimLevin December 15, 2022 13:35

asmorkalov added category: python bindings category: samples labels Dec 19, 2022

vrabaud force-pushed the numpy_fix branch from 39ae1e5 to 4761f1d Compare December 19, 2022 22:08

asmorkalov mentioned this pull request Dec 20, 2022

Pipelines update for FFmpeg 5 and security fixes opencv/opencv-python#756

Merged

asmorkalov requested review from asmorkalov and removed request for VadimLevin December 20, 2022 08:54

vrabaud changed the base branch from 4.x to 3.4 December 20, 2022 09:00

vrabaud changed the base branch from 3.4 to 4.x December 20, 2022 09:00

VadimLevin requested changes Dec 20, 2022

View reviewed changes

vrabaud force-pushed the numpy_fix branch from 4761f1d to 29fa91d Compare December 20, 2022 11:19

vrabaud requested a review from VadimLevin December 20, 2022 12:43

vrabaud force-pushed the numpy_fix branch from 29fa91d to 0a7a9ee Compare December 21, 2022 09:27

asmorkalov reviewed Dec 21, 2022

View reviewed changes

asmorkalov approved these changes Dec 22, 2022

View reviewed changes

VadimLevin reviewed Dec 22, 2022

View reviewed changes

modules/gapi/misc/python/test/test_gapi_core.py Outdated Show resolved Hide resolved

modules/objdetect/misc/python/test/test_objdetect_aruco.py Outdated Show resolved Hide resolved

asmorkalov mentioned this pull request Dec 22, 2022

Move contrib charuco to main objdetect #22986

Merged

6 tasks

asmorkalov added the priority: high label Dec 23, 2022

asmorkalov force-pushed the numpy_fix branch from 1090eab to 50a9844 Compare December 23, 2022 10:30

VadimLevin reviewed Dec 23, 2022

View reviewed changes

modules/python/test/test_misc.py Outdated Show resolved Hide resolved

modules/python/test/test_misc.py Show resolved Hide resolved

asmorkalov force-pushed the numpy_fix branch from 50a9844 to ad568ed Compare December 23, 2022 10:54

VadimLevin approved these changes Dec 23, 2022

View reviewed changes

asmorkalov merged commit 9012e6d into opencv:4.x Dec 23, 2022

vrabaud deleted the numpy_fix branch December 28, 2022 12:16

alalek mentioned this pull request Jan 8, 2023

(5.x) Merge 4.x #23113

Merged

Uh oh!

Conversation

vrabaud commented Dec 15, 2022

Pull Request Readiness Checklist

Uh oh!

vrabaud commented Dec 15, 2022

Uh oh!

asmorkalov commented Dec 20, 2022

Uh oh!

VadimLevin left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

vrabaud commented Dec 20, 2022

Uh oh!

VadimLevin commented Dec 20, 2022

Uh oh!

vrabaud commented Dec 20, 2022

Uh oh!

hawkinsp commented Dec 20, 2022

Uh oh!

vrabaud commented Dec 20, 2022

Uh oh!

hawkinsp commented Dec 20, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

vrabaud commented Dec 20, 2022

Uh oh!

asmorkalov commented Dec 21, 2022

Uh oh!

VadimLevin commented Dec 21, 2022

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

asmorkalov commented Dec 21, 2022

Uh oh!

asmorkalov commented Dec 21, 2022

Uh oh!

asmorkalov commented Dec 21, 2022

Uh oh!

asmorkalov commented Dec 21, 2022

Uh oh!

vrabaud commented Dec 21, 2022

Uh oh!

asmorkalov commented Dec 22, 2022

Uh oh!

Uh oh!

Uh oh!

asmorkalov commented Dec 23, 2022

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

VadimLevin left a comment •

edited

Loading

hawkinsp commented Dec 20, 2022 •

edited

Loading