Use gufunc fast parser in Time (alternative)#11768
Conversation
|
@mhvk - thanks, I'll have a look! It would certainly help if you could put in a doc string for |
|
@taldcroft - ok, uploaded a version with a lot more comments. Used |
|
@mhvk - Thanks for bearing with me. This looks basically good to me now, with just a question about testing. Does this need a test of inputing a strided array? Beyond that, are there features now enabled by doing this via gufunc that didn't work previously? |
astropy/time/src/parse_times.c
Outdated
| "Returns\n" \ | ||
| "-------\n" \ | ||
| "parser : `~numpy.ufunc`\n" \ | ||
| " Will parse bytes or unicode." |
There was a problem hiding this comment.
Can you clarify what's meant by unicode here? I guess this means numpy UTF4, or ?
There was a problem hiding this comment.
Hmm, that is really a leftover of what I wanted to happen. Right now, it would be possible to extent to UTF4, but little point until unicode is more directly supported for ufuncs. I replaced this text with 'Suitable for use by ~astropy.time.TimeString formats.'
|
Since this code appears to managing Python references, what assurance do you have that there are no leaks for the various cases (including different kinds of failures). |
|
One other thing is that I did build this on my Windows 10 VM using Python 3.7 and confirmed that tests passed. I know that those compiler flags were put in there in response to things failing otherwise, but nice to see that they appear to be no longer needed! |
|
@taldcroft - oops, the mistake you pointed out before came back in with the reversion... I'll fix that tomorrow. On the reference counting: I think I got it right, but note that it is strictly for the ufunc and its associated data with information on start/stop, etc., so this is allocated once and not touched again. The actual call to the parser all goes through numpy's ufunc machinery. On strided arrays, good point, might as well put in some tests with specifically odd strides. p.s. And happy to hear windows worked. I was hopeful since the tests passed. I remain unsure how numpy avoids the problems the ctypes implementation had. |
|
OK, addressed the inline comments and added a test with a transposed array to check ordering is as expected. |
|
This might break pytest collection. I'm getting a pytest error when doing This is the error: On further examination, this is only appears when astropy is installed with |
|
@nstarman - what platform was this on? I do all my testing with pytest like that and didn't run into a problem on Mac and Windows, though I use |
|
I'm actually having trouble reproducing this. I reinstalled astropy from source and it's gone. I'm hazarding a guess that the problem is related to installing astropy in develop mode and then switching to a branch where there's new compiled code. |
|
Yes, that seems likely - the same would have happened with an update to |
Alternative to #11718. @taldcroft - I think this will be closer to what you had in mind.