Use Cython 3.0.x#15402
Conversation
|
Thank you for your contribution to Astropy! 🌌 This checklist is meant to remind the package maintainers who will review this pull request of some common things to look for.
|
|
👋 Thank you for your draft pull request! Do you know that you can use |
| cdef int ndim "nd" | ||
| pass | ||
| # FIXME: doesn't work with recent Numpy API ? | ||
| # cdef int ndim "nd" |
There was a problem hiding this comment.
This wasn't working with the "new" Numpy API. Not sure what it does or why it was needed.
Also maybe it is possible now to import ndarray more directly without resorting to numpy/arrayobject.h.
There was a problem hiding this comment.
This is old code and I'm not sure of the details, but we probably need to understand it a bit before moving forward with this.
There was a problem hiding this comment.
There was a problem hiding this comment.
Actually since this works now without the ndim method, it would seem the entire custom ndarray is no longer needed, meaning basically 37014a9 could be reverted altogether. Against current main this would be
--- a/astropy/table/_column_mixins.pyx
+++ b/astropy/table/_column_mixins.pyx
@@ -25,6 +25,7 @@ Column is itself an array.
import sys
import numpy as np
+cimport numpy as cnp
cdef tuple INTEGER_TYPES = (int, np.integer)
@@ -45,21 +46,18 @@ cdef extern from "Python.h":
PyMappingMethods* tp_as_mapping
-cdef extern from "numpy/arrayobject.h":
- ctypedef class numpy.ndarray [object PyArrayObject]:
- pass
- # FIXME: doesn't work with recent Numpy API ?
- # cdef int ndim "nd"
+ndarray = np.ndarray
+ctypedef cnp.ndarray ndarray_t
ctypedef object (*item_getter)(object, object)
cdef inline object base_getitem(object self, object item, item_getter getitem):
- if (<ndarray>self).ndim > 1 and isinstance(item, INTEGER_TYPES):
+ if (<ndarray_t>self).ndim > 1 and isinstance(item, INTEGER_TYPES):
return self.data[item]
- dtype_kind = (<ndarray>self).dtype.kind
+ dtype_kind = (<ndarray_t>self).dtype.kind
if dtype_kind == 'V' and isinstance(item, STRING_TYPES):
return self.data[item]
This would even restore the current indexing performance in main (tested with numpy 1.26.0, 1.23.5 – 1.22 is breaking pyerfa in my mamba installation).
I suspect there is some more of the boilerplate code @embray mentioned in #4075 that could be removed with Cython 3, but I don't have much deeper insight here...
There was a problem hiding this comment.
@dhomeier - Yes, I came to the same conclusion while having a deeper look earlier today, see 0e21829.
This was probably needed for older version of Cython (at the time the code was added).
I also tried to remove the includes of PyMappingMethods and PyTypeObject but it seems we still need that to access .tp_as_mapping.mp_subscript.
There was a problem hiding this comment.
It seems that __Pyx_PyObject_GetItem would do just that, but I have not found any examples how to use that in Cython (or verify whether that would really give us the base class __getitem__).
In any case as the timings for the 2 and 3d cases indicate, at least in Python 3.11 the overhead from directly calling data[item] is now down to about a factor of 2, so there should not be serious performance repercussions either way.
|
Given the failures in CI, I guess so. |
|
Feel free to cherry pick the commit :) |
Signed-off-by: nstarman <nstarman@users.noreply.github.com>
https://docs.python.org/3/c-api/unicode.html Changed in version 3.7: The return type is now const char * rather of char *.
|
PR is ready (and tests were passing before the rebase). I removed all the NPY_1_7_API_VERSION warnings and a few other ones. I think we will need Cython 3 for Numpy 2.0 so we should get this for the first rc. |
taldcroft
left a comment
There was a problem hiding this comment.
For io.ascii, I honestly don't understand the changes but if it is passing tests then that is fine. For table we need to understand that change to the column mixin which provides the fast column item access.
| cdef int ndim "nd" | ||
| pass | ||
| # FIXME: doesn't work with recent Numpy API ? | ||
| # cdef int ndim "nd" |
There was a problem hiding this comment.
This is old code and I'm not sure of the details, but we probably need to understand it a bit before moving forward with this.
astrofrog
left a comment
There was a problem hiding this comment.
Approving the changes to timeseries.
larrybradley
left a comment
There was a problem hiding this comment.
stats and convolution look good
dhomeier
left a comment
There was a problem hiding this comment.
io.ascii and everything else works, LGTM!
| cdef int ndim "nd" | ||
| pass | ||
| # FIXME: doesn't work with recent Numpy API ? | ||
| # cdef int ndim "nd" |
There was a problem hiding this comment.
It seems that __Pyx_PyObject_GetItem would do just that, but I have not found any examples how to use that in Cython (or verify whether that would really give us the base class __getitem__).
In any case as the timings for the 2 and 3d cases indicate, at least in Python 3.11 the overhead from directly calling data[item] is now down to about a factor of 2, so there should not be serious performance repercussions either way.
|
Ok I think we are good, thanks for the reviews ! |
…402-on-v6.0.x Backport PR #15402 on branch v6.0.x (Use Cython 3.0.x)
With Cython 3.0 we can now remove all the NPY_NO_DEPRECATED_API warnings, I fixed most of them but a few ones remain (wip).
https://cython.readthedocs.io/en/latest/src/userguide/migrating_to_cy30.html
Description
Fixes #15315