-
-
Notifications
You must be signed in to change notification settings - Fork 12.1k
Closed
Description
In 2006, Tim Hochberg questioned the reason why fromiter does not handle object arrays:
/* We would need to alter the memory RENEW code to decrement any
reference counts before just throwing away the memory.
*/
This doesn't seem right. The array that we would be RENEWing is a bunch
of PyObject*s. The reference counts don't reside there, but in the
objects themselves. When we do the RENEW, we don't want the reference
counts to change at all. The one tricky case is if we run out of memory,
I'm not certain that the current setup correctly deals with reference
counts in this case, although it appears likely that it should work
since ret->data should still point to a valid chunk of memory and
decreffing ret should result in the subsequent deallocation of all the
stored objects.
http://mail.scipy.org/pipermail/numpy-discussion/2006-November/024747.html
@teoliphant responded
I'm not sure if you are still thinking about this. I suspect I added
the comment because I was concerned about two things:
1) a resize that deletes memory without first decrefing object counts
--- I suspose this can't happen here, though, because RENEW is always
"growing" memory, right?
2) a resize that grows memory without inserting NULLS. For Object array
manipulation to work, you need initialized (to NULL) "empty" memory.
>The one tricky case is if we run out of memory,
>I'm not certain that the current setup correctly deals with reference
>counts in this case, although it appears likely that it should work
>since ret->data should still point to a valid chunk of memory and
>decreffing ret should result in the subsequent deallocation of all the
>
>
>stored objects.
>
>
As long as you don't store the pointer to new memory until after the
RENEW was successful then this should work.
I think my comment came from the possibility of #1 in general use of
RENEW, but I don't think that is going to happen. Therefore, the only
tweak that seams necessary is to initialize any new memory to NULL for
OBJECT arrays.
http://mail.scipy.org/pipermail/numpy-discussion/2006-November/024841.html
I couldn't find any further discussion of this issue since 2006. Is this analysis of what needs to be done to PyArray_FromIter still sound?
Reactions are currently unavailable