Skip to content

Handle deprecation of numpy _validate_lengths#3556

Merged
soupault merged 1 commit intoscikit-image:masterfrom
sciunto:bugfix-3551
Nov 22, 2018
Merged

Handle deprecation of numpy _validate_lengths#3556
soupault merged 1 commit intoscikit-image:masterfrom
sciunto:bugfix-3551

Conversation

@sciunto
Copy link
Copy Markdown
Member

@sciunto sciunto commented Nov 18, 2018

Description

Fixes #3551

I had to make a choice here. I preferred to keep using numpy. I removed the commented code (IMHO, not such a good practice to do that, we can browse numpy's history at any time)

A TODO task is added to remove the conditional function call.

This PR needs to be backported.

Checklist

[It's fine to submit PRs which are a work in progress! But before they are merged, all PRs should provide:]

[For detailed information on these and other aspects see scikit-image contribution guidelines]

References

[If this is a bug-fix or enhancement, it closes issue # ]
[If this is a new feature, it implements the following paper: ]

For reviewers

(Don't remove the checklist below.)

  • Check that the PR title is short, concise, and will make sense 1 year
    later.
  • Check that new functions are imported in corresponding __init__.py.
  • Check that new features, API changes, and deprecations are mentioned in
    doc/release/release_dev.rst.
  • Consider backporting the PR with @meeseeksdev backport to v0.14.x

Copy link
Copy Markdown
Member

@jni jni left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@sciunto I've made a minor suggestion.

My personal preference, though, would be to vendor the code for _as_pairs. The leading underscore means that NumPy is under no obligation to keep that function stable, so we are just setting ourselves up for another bad surprise in the future.

from numpy.lib.arraypad import _as_pairs
except ImportError:
from numpy.lib.arraypad import _validate_lengths
old_numpy = True
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@sciunto, I prefer to have the NumPy version check explicit here:

from distutils import LooseVersion as Version
old_numpy = Version(np.__version__) < Version('1.16')
if old_numpy:
    from numpy.lib.arraypad import _validate_lengths
else:
    from numpy.lib.arraypad import _as_pairs

This makes it clearer (though code) whether 1.15 is inclusive or exclusive.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I made this change :)

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's distutils.version import... ;)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oops, sorry! =\

@sciunto
Copy link
Copy Markdown
Member Author

sciunto commented Nov 19, 2018

I agree with your observation. There are pro and cons. numpy does the job to maintain this code vs they may change again the behavior. I do not have strong feelings on this. my position was to fix it asap first.

@jni
Copy link
Copy Markdown
Member

jni commented Nov 20, 2018

my position was to fix it asap first.

A very good position! =)

Copy link
Copy Markdown
Member

@jni jni left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 I'm happy with this. Thanks @sciunto!

@scikit-image scikit-image deleted a comment from pep8speaks Nov 21, 2018
@soupault soupault changed the title Fixes #3551: _validate_lengths replaced in numpy/master Handle deprecation of numpy _validate_lengths Nov 22, 2018
@soupault soupault merged commit 31d9ecc into scikit-image:master Nov 22, 2018
@lumberbot-app
Copy link
Copy Markdown

lumberbot-app bot commented Nov 22, 2018

Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

  1. Checkout backport branch and update it.
$ git checkout v0.14.x
$ git pull
  1. Cherry pick the first parent branch of the this PR on top of the older branch:
$ git cherry-pick -m1 31d9ecc2f0d8dd3373af3e80b2dcc7887ff2ca24
  1. You will likely have some merge/cherry-pick conflict here, fix them and commit:
$ git commit -am 'Backport PR #3556: Handle deprecation of numpy `_validate_lengths`'
  1. Push to a named branch :
git push YOURFORK v0.14.x:auto-backport-of-pr-3556-on-v0.14.x
  1. Create a PR against branch v0.14.x, I would have named this PR:

"Backport PR #3556 on branch v0.14.x"

And apply the correct labels and milestones.

Congratulation you did some good work ! Hopefully your backport PR will be tested by the continuous integration and merged soon!

If these instruction are inaccurate, feel free to suggest an improvement.

@lumberbot-app lumberbot-app bot added the Still Needs Manual Backport MrMeeseeks-managed label label Nov 22, 2018
@soupault
Copy link
Copy Markdown
Member

Thanks @sciunto !

soupault added a commit to soupault/scikit-image that referenced this pull request Nov 22, 2018
Handle deprecation of numpy `_validate_lengths`
soupault added a commit to soupault/scikit-image that referenced this pull request Nov 22, 2018
Handle deprecation of numpy `_validate_lengths`
soupault added a commit that referenced this pull request Nov 22, 2018
….14.x

Merge pull request #3556 from sciunto/bugfix-3551
@soupault soupault removed the Still Needs Manual Backport MrMeeseeks-managed label label Nov 22, 2018
@sciunto sciunto deleted the bugfix-3551 branch November 22, 2018 21:04
@jriddy
Copy link
Copy Markdown

jriddy commented Jan 14, 2019

Any info on when this will get released on pypi? Numpy 1.16 just came out and broke this in our CI jobs. We can do a custom build of scikit-image internally or just pin numpy's version back for the time being but I prefer to keep with the mainstream of packages when possible

@jni
Copy link
Copy Markdown
Member

jni commented Jan 14, 2019

@jriddy sorry about that! We are aiming to release 0.14.2 and 0.15 (Python 3.5+ only) in the next couple of weeks...

@jriddy
Copy link
Copy Markdown

jriddy commented Jan 14, 2019

@jni No reason to be sorry, we appreciate the work you guys do.

Is the backport branch for 0.14.x currently stable for some value of that word? I can easily clone this and cut an internal 0.14.2-dev1 release that can hold us over until you guys release to PyPI. I just want to know if your dev cycle includes a big QA blitz at the end before a release, or if you generally expect things to work well in your backport branch.

@jni
Copy link
Copy Markdown
Member

jni commented Jan 14, 2019

Oh! Both v0.14.x and master should work great! We try not to merge PRs to either that will break things or require follow up work. (Other than maybe code style or minor speed issues, and even then only rarely.) The last minute pre-release push is just fiddling with release notes and version numbers.

@jriddy
Copy link
Copy Markdown

jriddy commented Jan 14, 2019

Hmmm, is there any guide here for building wheels for this project?

Locally my wheel is working, but inside my dockers I'm getting E ImportError: /usr/local/lib/python3.6/site-packages/skimage/restoration/_unwrap_1d.cpython-36m-x86_64-linux-gnu.so: undefined symbol: PyFPE_jbuf issues.

I'm building in a virtualenv on Ubuntu 18.04 with python 3.6.7.

@stefanv
Copy link
Copy Markdown
Member

stefanv commented Jan 14, 2019 via email

@hmaarrfk
Copy link
Copy Markdown
Member

@jriddy I'm not advocating leaving pypi behind, but scikit-image on conda-forge has the necessary patches to allow a 0.14.1 to be compatible with numpy 1.16.

https://github.com/conda-forge/scikit-image-feedstock/tree/master/recipe

If you can manage to build wheels for 0.14.1, then you should be able to use those patches to get the compatibility you need.

@jriddy
Copy link
Copy Markdown

jriddy commented Jan 14, 2019

It's okay, my original idea was to just mod the version and build a scikit-image-0.14.1.post1 wheel and host it on our internal pypi so it would preempt 0.14.1 but get superseded automatically once you guys release 0.14.2 or 0.15.0, because that would have been the least amount of intervention in the context of my org's CI setup, but as it got more involved, it's just turned out to be simpler to pin numpy to <1.16 in a few of our repos and wait until you guys release.

Thanks for your responses, folks! I really appreciate it.

@pllim
Copy link
Copy Markdown

pllim commented Feb 11, 2019

Hello. I am confused. I thought this is fixed in 0.14.2 but this PR is milestoned to 0.14.3.

@sciunto
Copy link
Copy Markdown
Member Author

sciunto commented Feb 11, 2019

Because I renamed the milestone as several issues were still open after 0.14.2. I shouldn't have done that... Sorry.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

_validate_lengths replaced in numpy 1.16 leading to ImportError exception in scikit-image less than 0.14.2

7 participants