-
-
Notifications
You must be signed in to change notification settings - Fork 2k
Issue 1943 - systematically testing pickling of astropy classes #1947
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
NICE! |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is there an advantage to pickling through a physical file instead of just using dumps and loads? At the least you could use a StringIO object to avoid the caller being required to give a file name testfile.
|
@taldcroft Good points. I've implemented them in my latest commit. Do you think the number of objects included should be expanded? |
|
This is really cool! Out of curiosity, why are we not testing the values of the pickled object? Does a lot break if we do? |
|
@mdboom I had been assuming that if an attribute could be pickled, then it was probably pickled correctly. I just added a fairly generic comparison step, and it hasn't broken anything so far. |
|
@ktchrn - I was having trouble with table Column objects going through pickle and having incorrect attribute values (e.g. losing meta values), so it's definitely a good idea to test the values. |
|
Before recursion, tests weren't catching objects with non-pickleable objects as attributes. Only big change so far is that coordinate systems break because they use Longitude. |
|
This is great! |
|
It looks like this should be combined with #1961, which includes a fix for |
|
Agree this is awesome! Given that #1961 is already in the works, I agree with @astrofrog that it makes sense to rebase this against that once it's merged. But for the other failures, I wonder if maybe we should make new issues and then xfail them, so that this actually gets into the tests now? |
|
@mdboom Okay, I'll start working on it again. Are there any features that it particularly needs, apart from passing the Travis build with older numpy versions and no optional dependencies? |
astropy/tests/helper.py
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This needs a docstring.
|
I think mainly we just need to get the tests passing. I did make one comment about a missing docstring as well. |
|
For me the GitHub UI doesn't show a link to the travis-ci tests for the latest commit: b2405cc !? |
astropy/wcs/tests/test_pickle.py
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think you accidentally added whitespace on this line ... please remove it.
|
@ktchrn - now that 0.4 is out we can consider merging this - could you rebase this and add a changelog entry to the 1.0 entry? |
|
@ktchrn - sorry for the delay, could you rebase this so we can go ahead and merge it? Please include a changelog entry in the 1.0 section. |
|
I don't think this really requires a changelog entry. It's just adding some new tests--users don't care about that. |
afea69b to
6696736
Compare
|
Oops--what happened here? Did you try to rebase? |
|
Yeah. I guess that's all stuff that has happened since the last time I worked on this. If it's more convenient, I can just open a new pull request starting from the project's current state. |
|
@ktchrn - you should be able to clean this up by rebasing - what I suspect is that you may not have rebased against the latest upstream master. Assuming the astropy repo (not your fork) is Let us know if there are any conflicts and you are not sure how to deal with them. Once the rebase is complete, you can force push to the same branch: |
6696736 to
8b76b3d
Compare
|
Thanks! You're right, I was rebasing on my outdated master instead of upstream/master. |
|
Great, that looks a lot better :) |
|
Except of course for the build failing, but we'll get to the bottom of that. It's probably something minor. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@ktchrn - the CartesianPoints class no longer exists, can you remove this line?
|
Looks great--thanks @ktchrn ! |
Issue 1943 - systematically testing pickling of astropy classes
Made sure existing pickling tests included multiple protocols.
Added basic tests for most base classes and some subclasses with additional attributes.
Most of these tests only check whether or not attributes survive pickling, not whether or not they have the same value before and after. Only the NDData test checks for changes to the various arrays it stores.
Some of the failures may be irrelevant (e.g. is constants not being pickleable actually a problem?).
Some of the failures are currently xfails by design (e.g. coordinates.DynamicTransform, which can have a function as an attribute).
Successfully reproduces the Issue 1942 problem.