Skip to content

MAINT: Hide decorator from pytest traceback#28211

Merged
mdhaber merged 1 commit intonumpy:mainfrom
hmaarrfk:patch-3
Mar 13, 2025
Merged

MAINT: Hide decorator from pytest traceback#28211
mdhaber merged 1 commit intonumpy:mainfrom
hmaarrfk:patch-3

Conversation

@hmaarrfk
Copy link
Copy Markdown
Contributor

Currently it points the user to the internal numpy function

kwargs = {'strict': False}, old_name = 'y', new_name = 'desired'

    @functools.wraps(fun)
    def wrapper(*args, **kwargs):
        for old_name, new_name in zip(old_names, new_names):
            if old_name in kwargs:
                if dep_version:
                    end_version = dep_version.split('.')
                    end_version[1] = str(int(end_version[1]) + 2)
                    end_version = '.'.join(end_version)
                    msg = (f"Use of keyword argument `{old_name}` is "
                           f"deprecated and replaced by `{new_name}`. "
                           f"Support for `{old_name}` will be removed "
                           f"in NumPy {end_version}.")
                    warnings.warn(msg, DeprecationWarning, stacklevel=2)
                if new_name in kwargs:
                    msg = (f"{fun.__name__}() got multiple values for "
                           f"argument now known as `{new_name}`")
                    raise TypeError(msg)
                kwargs[new_name] = kwargs.pop(old_name)
>       return fun(*args, **kwargs)
E       AssertionError:
E       Arrays are not equal
E
E       (shapes (2, 2, 200, 200), (2, 2, 800, 800) mismatch)
E        ACTUAL: array([[[[0, 0, 0, ..., 0, 0, 0],
E                [0, 0, 0, ..., 0, 0, 0],
E                [0, 0, 0, ..., 0, 0, 0],...
E        DESIRED: array([[[[0, 0, 0, ..., 0, 0, 0],
E                [0, 0, 0, ..., 0, 0, 0],
E                [0, 0, 0, ..., 0, 0, 0],...

../../miniforge3/envs/dev/lib/python3.10/site-packages/numpy/_utils/__init__.py:85: AssertionError

Whit this PR:

>       np.testing.assert_equal(data, other)
E       AssertionError:
E       Arrays are not equal
E
E       (shapes (2, 2, 200, 200), (2, 2, 800, 800) mismatch)
E        ACTUAL: array([[[[0, 0, 0, ..., 0, 0, 0],
E                [0, 0, 0, ..., 0, 0, 0],
E                [0, 0, 0, ..., 0, 0, 0],...
E        DESIRED: array([[[[0, 0, 0, ..., 0, 0, 0],
E                [0, 0, 0, ..., 0, 0, 0],
E                [0, 0, 0, ..., 0, 0, 0],...

tests/my_test.py:190: AssertionError

Currently it points the user to the internal numpy function

```
kwargs = {'strict': False}, old_name = 'y', new_name = 'desired'

    @functools.wraps(fun)
    def wrapper(*args, **kwargs):
        for old_name, new_name in zip(old_names, new_names):
            if old_name in kwargs:
                if dep_version:
                    end_version = dep_version.split('.')
                    end_version[1] = str(int(end_version[1]) + 2)
                    end_version = '.'.join(end_version)
                    msg = (f"Use of keyword argument `{old_name}` is "
                           f"deprecated and replaced by `{new_name}`. "
                           f"Support for `{old_name}` will be removed "
                           f"in NumPy {end_version}.")
                    warnings.warn(msg, DeprecationWarning, stacklevel=2)
                if new_name in kwargs:
                    msg = (f"{fun.__name__}() got multiple values for "
                           f"argument now known as `{new_name}`")
                    raise TypeError(msg)
                kwargs[new_name] = kwargs.pop(old_name)
>       return fun(*args, **kwargs)
E       AssertionError:
E       Arrays are not equal
E
E       (shapes (2, 2, 200, 200), (2, 2, 800, 800) mismatch)
E        ACTUAL: array([[[[0, 0, 0, ..., 0, 0, 0],
E                [0, 0, 0, ..., 0, 0, 0],
E                [0, 0, 0, ..., 0, 0, 0],...
E        DESIRED: array([[[[0, 0, 0, ..., 0, 0, 0],
E                [0, 0, 0, ..., 0, 0, 0],
E                [0, 0, 0, ..., 0, 0, 0],...

../../miniforge3/envs/dev/lib/python3.10/site-packages/numpy/_utils/__init__.py:85: AssertionError
```
@ngoldbaum
Copy link
Copy Markdown
Member

Seems like an improvement to me.

CI failure is unrelated.

@mdhaber since you did a lot of work recently on these assertion helpers, what do you think about this?

@mdhaber
Copy link
Copy Markdown
Contributor

mdhaber commented Jan 21, 2025

IIUC, it's unfortunate that the use of _rename_parameter obscured the traceback of assertion functions.

But also IIUC, the decorator is currently used only on the assert_array_equal and assert_array_almost_equal functions. It was applied in 2.0, so the change can probably be made permanently (and the decorator removed) in the next release. I already removed the decorator from the assertion functions. I think that will solve the immediate problem reported here. Right? Was the plan to backport this?

It's still probably not a bad idea to hide the decorator traceback from the decorator, though. (Unless the problem actually occurs in the decorator, which would be unusual.)

@ngoldbaum
Copy link
Copy Markdown
Member

Ah good call @hmaarrfk did you check how this error looks right now on main?

@hmaarrfk
Copy link
Copy Markdown
Contributor Author

I already removed the decorator from the assertion functions. I think that will solve the immediate problem reported here. Right? Was the plan to backport this?

Great feel free to close this. I guess it stumped me

@hmaarrfk
Copy link
Copy Markdown
Contributor Author

It won't happen on main. good enough for me.

fyi, you might want to remove the import of

_rename_parameter

and maybe go as far as getting rid of the utility function alltogether?

I once wrote these deprecators with tests: https://pypi.org/project/wabisabi/

I know numpy is all about being lightweight, but without the usage of _rename_parameter it is now an orphaned utility function.

@hmaarrfk hmaarrfk closed this Jan 22, 2025
@hmaarrfk
Copy link
Copy Markdown
Contributor Author

Take this as a little user story. i'm still on numpy 1.26 since some libraries (tensorflow) only just got to 2.X.

So you might want to leave in some deprecations a little longer than you think you should.

@mdhaber
Copy link
Copy Markdown
Contributor

mdhaber commented Jan 22, 2025

So you might want to leave in some deprecations a little longer than you think you should.

This was discussed in the linked issue. I proposed removing it according to the standard policy; I left it to other maintainers to decide whether an exception was needed.

There is always going to be a mismatch between how often developers and users would like updates to happen, so there has to be a compromise somewhere. Users that don't update often enough will miss important warnings and updates. Consider updating in smaller increments if necessary. (Every other version / once per year should be enough to catch all deprecation warnings.)

I did forget to remove the import. Typically linting would catch that sort of thing, but I guess that rule is not enabled. I'd suggest the rule be enabled and the import be removed in that PR.

I don't think the function needs to be removed. It will inevitably be useful later.

@hmaarrfk
Copy link
Copy Markdown
Contributor Author

i'm back here 2 months later wishing that an update to numpy would have resolved this issue.

I generally think that this fix is "in line" with the best practices of numpy. I see little harm in pressing "merge"

Any chance to merge this in?

@hmaarrfk hmaarrfk reopened this Mar 13, 2025
@mdhaber
Copy link
Copy Markdown
Contributor

mdhaber commented Mar 13, 2025

Oops, I see what happened. gh-27738 (which fixes this by removing the decorator from these functions entirely) merged to main before this issue was opened, but that commit wasn't slated to actually be included in NumPy until version 2.3 is released (June-ish).

Since the PR was closed, we didn't merge and backport it with 2.2.3. Let's merge it now and mark it for backport in 2.2.4 (if there is one). The decorator is not applied to these functions in 2.3, so it will definitely be fixed there.

So you might want to leave in some deprecations a little longer than you think you should.

So incidentally, the deprecation was left in place longer than usual!

@mdhaber mdhaber merged commit 2fd2491 into numpy:main Mar 13, 2025
68 checks passed
@mdhaber mdhaber added this to the 2.2.4 release milestone Mar 13, 2025
@mdhaber mdhaber added the 09 - Backport-Candidate PRs tagged should be backported label Mar 13, 2025
@hmaarrfk
Copy link
Copy Markdown
Contributor Author

Thanks!

I think since the plan was to keep the decorators alive it will help in the next round anyway!!!

@charris charris removed the 09 - Backport-Candidate PRs tagged should be backported label Mar 14, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants