EventHookFunc and SetEventHook need to take a void *user_data argument.
On Jun 15, 2012 1:06 PM, "Thouis (Ray) Jones" <
reply@reply.github.com>
wrote:

This is the simplest intersection of two previous pull requests:
#301
#284

Changes:
Moves PyDataMem_NEW/FREE/RENEW to the external API.
Fixes PyDataMem_NEW/RENEW to return void* instead of char_.
Replaces PyDataMem_NEW/FREE with NpySortArray_malloc/free in sort.c.src
(should be reverted if npysort is moved to be part of multiarraymodule).
Adds PyDataMem_SetEventHook which takes a (PyDataMem_EventHookFunc *) as
an argument,
with signature:
void hook(void *old, void *new, size_t size).
When not NULL, hook will be called at the end of each
PyDataMem_NEW/FREE/RENEW:
result = PyDataMem_NEW(size) -> (_hook(NULL, result, size)
PyDataMem_FREE(ptr) -> (_hook(ptr, NULL, 0)
result = PyDataMem_RENEW(ptr, size) -> (_hook)(ptr, result, size)
Adds tests in multiarray_tests.c.src, driven by tests/test_multiarray.py.

You can merge this Pull Request by running:

git pull https://github.com/thouis/numpy malloc_hooks

Or you can view, comment on it, or merge it online at:

#309

-- Commit Summary --

• ENH: expose PyDataMem_NEW/FREE/RENEW as numpy API functions with an
  event hook.

-- File Changes --

M numpy/core/code_generators/numpy_api.py (4)
M numpy/core/include/numpy/ndarraytypes.h (11)
M numpy/core/src/multiarray/multiarray_tests.c.src (57)
M numpy/core/src/multiarray/multiarraymodule.c (64)
M numpy/core/src/npysort/sort.c.src (18)
M numpy/core/tests/test_multiarray.py (13)
M numpy/lib/src/_compiled_base.c (7)

-- Patch Links --

https://github.com/numpy/numpy/pull/309.patch
https://github.com/numpy/numpy/pull/309.diff

---

Reply to this email directly or view it on GitHub:
#309

...Now that I think about it, there's also a concurrency problem here, since PyDataMem_NEW/FREE/RENEW can be called without the GIL. So PyDataMem_SetHookFunc could race with them, and we need a mutex of some kind to prevent calling into the wrong function pointer or with the wrong user_data. To avoid overhead on the no-hook path though I think it'd be okay to do something like

if (_PyDataMem_eventhook != NULL) {
    acquire_mutex();
    if (_PyDataMem_eventhook != NULL) {
        (*_PyDataMem_eventhook)(...)
    }
    release_mutex();
}

The GIL could be used for the mutex, or else the PyThread_*_lock API could be used directly.

Sorry I keep coming up with extra issues here! You might want to run the API by the list first just to make sure that there's consensus on it before working on it more. (Most people don't see pull requests.)

No problem, I appreciate the feedback. I'll put in a fix for locking, and post to the mailing list.

Limited response on the mailing list (Dag seems to like the idea and possible extensions).

@charris @cournape @teoliphant have all expressed opinions on the other pull requests, so I'll ping them here.

This seems like quite a useful feature that could help debug many programs. I am supportive of its inclusion.

Any more comments on this PR? Mailing list was similarly quiet when asked.

Looks good, I guess! Only thought I've had is from this message http://www.mail-archive.com/numpy-discussion@scipy.org/msg37843.html, maybe an even nicer name would be PyDataMem_SetTraceHook? But that's pretty trivial.

I think github's saying there's a merge conflict ("This pull request cannot be automatically merged"). Can you merge from master or rebase?

I've toyed with different names (Trace, Log, Event), and decided to go with Event because it seemed the most generic.

I'll rebase, and also update the numpy api hash (I had held off on that part, because I knew the NA removal changes were probably coming soon).

I merged master to this branch. If you prefer a cleaner history, I also pushed a malloc_hooks_rebase branch to my repo.

Any chance of this making it into 1.7?

This pull request passes (merged b426539 into 3b9a0fe).

Not sure what's going on with 1.7, but probably? Sorry for all the delays and back and forth here.

Holy cow, this tool is really useful! Is there any documentation for it on http://docs.scipy.org? For that matter, is there any reason it can't be moved out of this tools outpost and into a part of numpy that gets installed by default?