Hypothesis tests for astropy.units#9211
Conversation
|
@Zac-HD - thanks for the PR! :) |
|
@Zac-HD - we'd definitely need to run some benchmarks before turning SI prefixes into Here, remember that units are almost exclusively used as attributes to |
|
The numerics of units are pretty simple, but parsing Unicode is a classic place to get things disastrously wrong. Perhaps #9348 would be a good place to apply hypothesis? |
|
Looking at past issues, I think a similar test might have caught #8260, if you could state exact properties (even if only for units that do not involve floating-point factors). |
|
@Zac-HD - I'd be quite keen to get some form of hypothesis testing in, and this seems a nice one. I think the main question is to ensure this is not run all the time cc @astropy/coordinators |
|
Why not run it all the time? The performance overhead is tiny, and Hypothesis tests often find bugs that example-based tests don't... but only if they're executed of course! |
|
Will it still be tiny if we do it for every package? But if it is small compared to the other tests, then it is OK. (Sorry, I'm a bit of a proselitizer for reducing power consumption...) |
|
I've milestoned it to 4.1 as we plan to have a discussion and possibly sprint around this at the coordination meeting in December. The overall feeling seems to be that we'll need this, but on the other hand this week is feature freeze, so much of the infrastructural changes and refactors are put on the back burner. |
I would expect that test runtime would still mostly be example-based rather than property-based tests. If PBT is taking too long, you can e.g. turn down the
🎉 If it would be helpful, I'd be delighted to attend any sessions on testing remotely from Australia to resource the conversation. |
f61bf6b to
29123e1
Compare
|
That sounds very useful, thank you. I've added a note to the meeting notes to schedule the discussion that falls into daytime in Canberra (https://docs.google.com/document/d/1pO0giLKGEle5rJeyxhc33rIARY_e_naLiYprScROP84/edit#) |
If hypothesis ends up fully integrated it should probably be put into pytest-astropy as a dependency, and some of this config should move there.
Import hypothesis setup from PR astropy#9211 - If hypothesis ends up fully integrated it should probably be put into pytest-astropy as a dependency, and some of this config should move there. Introduce non-hypothesis example for use in astropy#9579
Import hypothesis setup from PR astropy#9211 - If hypothesis ends up fully integrated it should probably be put into pytest-astropy as a dependency, and some of this config should move there. Introduce non-hypothesis example for use in astropy#9579
|
@bsipocz @astrofrog (@aarchiba) I have a thing in my calendar to phone in to the coordination meeting if you're discussing Hypothesis. Is there a specific time attached to that now? |
|
@Zac-HD , are you able to join http://joinslack.astropy.org/ ? |
|
Yep, I'm in. Taking it over there. |
|
Closing this PR because I think @aarchiba's work is much more convincing about the value of property-based testing for astropy (and astronomy), and the decision to use Hypothesis has been made - it's now provided when you install Thanks to everyone who has been part of these conversations over the last months! |
|
Thank you, @Zac-HD! It will get to units too... |
This PR adds Hypothesis tests for
astropy.units, as proposed in #9017.The test will reliably find floating-point imprecision in the scale factors of various units, which can build up over the course of round-trip conversions. I would propose that all units should be defined with the scale factor from fundamental SI units stored as a
Fraction, so that conversions can be lossless - the overhead is very small as it is a single number in the metadata for an array of any size.I've done my best to list the test-time dependency on
hypothesisin the right place, but would be very happy if anyone had suggestions or feedback on this 😄