Skip to content

celery: add celery app and task patching#135

Merged
palazzem merged 1 commit into
DataDog:masterfrom
brettlangdon:brettlangdon/patch.celery.sqwished
Feb 23, 2017
Merged

celery: add celery app and task patching#135
palazzem merged 1 commit into
DataDog:masterfrom
brettlangdon:brettlangdon/patch.celery.sqwished

Conversation

@brettlangdon

@brettlangdon brettlangdon commented Dec 16, 2016

Copy link
Copy Markdown
Member

Fixes #87

This PR adds the contrib module for celery and the ability to patch the celery module to trace task execution.

Example:

from ddtrace.contrib.celery import patch
patch()

import celery

app = celery.Celery()

@app.task
def my_task():
    pass

my_task.delay()

I also have some example screenshots of how these traces look.

Task being executed by a worker
image

Task execution started by a Flask request
image

This one is more for fun, since it requires using redis with celery and having the redis library patching enabled.

Redis calls to schedule celery task
image

image

clutchski
clutchski reviewed Dec 19, 2016
View reviewed changes
Comment thread ddtrace/contrib/celery/util.py Outdated

@clutchski clutchski Dec 19, 2016

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is a fair amount of metadata. is it all interesting to you?

@brettlangdon brettlangdon Dec 19, 2016

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tough call, I typically don't dig too deep into this data, but I also usually don't have this data available. Our setup is pretty basic so it is pretty easy to know when/where a task was executed.

It might be worth discussing with someone with a more "complex" setup than us, maybe @miketheman or @JTCunning might have some insight here on what is worth keeping.

@JTCunning JTCunning Dec 19, 2016

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd be interested in most of the identifiers (id/reply_to/correlation_id/task/hostname), and everything inside of delivery_info.

I would say all of the booleans don't really lead to any debugging processes, as they would be either application semantics of widely duplicated results (is_eager/utc are configured globally).

Not sure how @clutchski would like to proceed with the nullable values, but I think it would be a good idea to only display the time related metadata if their fields are populated.

@brettlangdon brettlangdon Dec 22, 2016

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@clutchski curiosity, is there any particular reason to remove some of these tags?

Looking to save on bytes where we can, reduces processing, etc?

@clutchski clutchski Dec 22, 2016

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@brettlangdon exactly. all of this will be collected, stored and serialized millions of times and if nobody cares, better to be light.

@brettlangdon brettlangdon Dec 22, 2016

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense.

We can reasonably drop called_directly, and is_eager. Those should be clear from the trace hierarchy anyways (if they show up under a flask request, they were called directly or were eager... compared to their own trace, which is run by a worker).

We can probably drop the task name, which is the name used for the resource anyways.

We can drop eta, expires, timelimit when they are None (or (None, None)).

Lastly, we can drop utc since it is configured globally.

This will leave us with:

  • correlation_id
  • delivery_info
  • eta - When not None
  • expires - When not None
  • hostname
  • id
  • reply_to
  • retries - Maybe even drop this when it is 0
  • timelimit - When not (None, None)

This seem more reasonable? Anything else in here that seems like a likely candidate to remove?

@palazzem palazzem self-requested a review December 19, 2016 19:11
@brettlangdon brettlangdon force-pushed the brettlangdon/patch.celery.sqwished branch from 289af14 to 64a4312 Compare January 11, 2017 15:52
@brettlangdon

brettlangdon commented Jan 11, 2017

Copy link
Copy Markdown
Member Author

Updated this PR to make the previously discussed changes to the metadata.

  • Remove unnecessary metadata keys
  • Do not send unset metadata values

clutchski
clutchski reviewed Jan 27, 2017
View reviewed changes
Comment thread ddtrace/contrib/celery/app.py Outdated

@clutchski clutchski Jan 27, 2017

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we should probably add a worker type to this or something.

@clutchski clutchski added this to the 0.6.0 milestone Feb 14, 2017
@brettlangdon brettlangdon force-pushed the brettlangdon/patch.celery.sqwished branch from 64a4312 to 707a432 Compare February 15, 2017 20:32
@brettlangdon

brettlangdon commented Feb 15, 2017

Copy link
Copy Markdown
Member Author

@clutchski updated here to use a new worker app type. Let me know if I didn't add the new app type correctly (or if you wanted it named differently).

@jontonsoup

jontonsoup commented Feb 20, 2017

Copy link
Copy Markdown

Hi friends -- I can help this this out on a real app if you need it

@brettlangdon brettlangdon mentioned this pull request Feb 21, 2017
Closed
@palazzem palazzem merged commit d929082 into DataDog:master Feb 23, 2017
@brettlangdon

brettlangdon commented Feb 23, 2017

Copy link
Copy Markdown
Member Author

\0/

@palazzem palazzem mentioned this pull request Feb 23, 2017
Merged
@palazzem

palazzem commented Feb 23, 2017

Copy link
Copy Markdown

Hey @brettlangdon thank you for this PR! It's good to be merged! Actually I did small minor changes that are under review but your work was great! Just in case, if you want to take a look you can find the PR here: #196

And yes @jontonsoup that would be great if you want to use this Celery integration and see if it fits your needs!

@kichooo

kichooo commented Sep 28, 2017

Copy link
Copy Markdown

Is there some example how is this integration supposed to work? I only get traces for a small subset of tasks, and I don't get any interesting traces out of it, namely the apply_async length - but execution length of task would be more interesting?

gh-worker-dd-mergequeue-cf854d Bot pushed a commit that referenced this pull request Apr 23, 2026
@KowalskiThomas
fix(profiling): guard memalloc hook against partially-written saved a…
7114b09 
…llocator (#17664)

## Description

This PR fixes a segmentation fault in the memory allocation profiler that occurs when a hook call races with `memalloc` start/stop operations. The issue arises from concurrent access to the saved allocator struct, which could be partially written while being read, resulting in`NULL` function pointers being dereferenced.  The key indicator in that case is that `#1 0x0000000000000000` frame -- we are trying to execute a null function pointer.

````
Error UnixSignal: Process terminated with SEGV_MAPERR (SIGSEGV)
#0   0x00007ff3c303a8d4  
#1   0x0000000000000000 memalloc_alloc (/go/src/github.com/DataDog/apm-reliability/dd-trace-py/ddtrace/profiling/collector/_memalloc.cpp:68)
#2   0x00007ff39dcb3b20 memalloc_alloc (/go/src/github.com/DataDog/apm-reliability/dd-trace-py/ddtrace/profiling/collector/_memalloc.cpp:68)
#3   0x00007ff39dcb3b20 memalloc_malloc(void*, unsigned long) (/go/src/github.com/DataDog/apm-reliability/dd-trace-py/ddtrace/profiling/collector/_memalloc.cpp:80)
#4   0x00007ff3c3087e1b PyUnicode_New 
#5   0x00007ff3c30889f4  
#6   0x00007ff3c3170c84  
#7   0x00007ff3c316b931  
#8   0x00007ff3c31aaac8  
#9   0x00007ff3c31033ac  
#10  0x00007ff3c310e2a6 PyObject_CallMethodObjArgs 
#11  0x00007ff3c310e46d  
#12  0x00007ff3c31a96c2  
#13  0x00007ff3c3102fd7 PyObject_Vectorcall 
#14  0x00007ff3c32335a2 _PyEval_EvalFrameDefault 
#15  0x00007ff3c323c094  
#16  0x00007ff3c3233dd3 _PyEval_EvalFrameDefault 
#17  0x00007ff3c323c094  
#18  0x00007ff3c30e997d PyObject_CallOneArg 
#19  0x00007ff3c306a480 _PyObject_GenericGetAttrWithDict 
#20  0x00007ff3c30c620d PyObject_GetAttr 
#21  0x00007ff3c32309e7 _PyEval_EvalFrameDefault 
#22  0x00007ff3c323c094  
#23  0x00007ff3c312880e  
#24  0x00007ff3c30e917c _PyObject_MakeTpCall 
#25  0x00007ff3c32335a2 _PyEval_EvalFrameDefault 
#26  0x00007ff3c323c094  
#27  0x00007ff3c312880e  
#28  0x00007ff3c30e917c _PyObject_MakeTpCall 
#29  0x00007ff3c32335a2 _PyEval_EvalFrameDefault 
#30  0x00007ff3c323c094  
#31  0x00007ff3c3233dd3 _PyEval_EvalFrameDefault 
#32  0x00007ff3c323c094  
#33  0x00007ff3c317d0fd  
#34  0x00007ff3c3233dd3 _PyEval_EvalFrameDefault 
#35  0x00007ff3c323c094  
#36  0x00007ff3c3233dd3 _PyEval_EvalFrameDefault 
#37  0x00007ff3c323c094  
#38  0x00007ff3c317d1b5  
#39  0x00007ff3c3102fd7 PyObject_Vectorcall 
#40  0x00007ff3c3232f4a _PyEval_EvalFrameDefault 
#41  0x00007ff3c3240da5  
#42  0x00007ff3c324112d  
#43  0x00007ff3c3233be1 _PyEval_EvalFrameDefault 
#44  0x00007ff3c323c094  
#45  0x00007ff3c317d1b5  
#46  0x00007ff3c3102fd7 PyObject_Vectorcall 
#47  0x00007ff3c3232f4a _PyEval_EvalFrameDefault 
#48  0x00007ff3c323c094  
#49  0x00007ff3c31033ac  
#50  0x00007ff3c310358d PyObject_CallFunctionObjArgs 
#51  0x00007ff3bf7eb91d WraptBoundFunctionWrapper_call (/project/src/wrapt/_wrappers.c:3750)
#52  0x00007ff3c3104055 _PyObject_Call 
#53  0x00007ff3c3233dd3 _PyEval_EvalFrameDefault 
#54  0x00007ff3c323c094  
#55  0x00007ff3c317d23c  
#56  0x00007ff3c3233dd3 _PyEval_EvalFrameDefault 
#57  0x00007ff3c323c094  
#58  0x00007ff3c310416f _PyObject_Call 
#59  0x00007ff3c3233dd3 _PyEval_EvalFrameDefault 
#60  0x00007ff3c3240da5  
#61  0x00007ff3c324112d  
#62  0x00007ff3c3233be1 _PyEval_EvalFrameDefault 
#63  0x00007ff3c323c094  
#64  0x00007ff3c317d1b5  
#65  0x00007ff3c3102fd7 PyObject_Vectorcall 
#66  0x00007ff3c3232f4a _PyEval_EvalFrameDefault 
#67  0x00007ff3c323c094  
#68  0x00007ff3c317d1b5  
#69  0x00007ff3c310416f _PyObject_Call 
#70  0x00007ff3c3233dd3 _PyEval_EvalFrameDefault 
#71  0x00007ff3c323c094  
#72  0x00007ff3c317d1b5  
#73  0x00007ff3c310416f _PyObject_Call 
#74  0x00007ff3c3233dd3 _PyEval_EvalFrameDefault 
#75  0x00007ff3c323c094  
#76  0x00007ff3c31033ac  
#77  0x00007ff3c310358d PyObject_CallFunctionObjArgs 
#78  0x00007ff3bf7eb91d WraptBoundFunctionWrapper_call (/project/src/wrapt/_wrappers.c:3750)
#79  0x00007ff3c30e917c _PyObject_MakeTpCall 
#80  0x00007ff3c32335a2 _PyEval_EvalFrameDefault 
#81  0x00007ff3c323c094  
#82  0x00007ff3c317d518  
#83  0x00007ff3c3155963  
#84  0x00007ff3c315393d  
#85  0x00007ff3c3233dd3 _PyEval_EvalFrameDefault 
#86  0x00007ff3c323c094  
#87  0x00007ff3c317d0fd  
#88  0x00007ff3c3233dd3 _PyEval_EvalFrameDefault 
#89  0x00007ff3c323c094  
#90  0x00007ff3c317d0fd  
#91  0x00007ff3c317d518  
#92  0x00007ff3c3233dd3 _PyEval_EvalFrameDefault 
#93  0x00007ff3c323c094  
#94  0x00007ff3c30e9371 _PyObject_FastCallDictTstate 
#95  0x00007ff3c30e958d _PyObject_Call_Prepend 
#96  0x00007ff3c3109150  
#97  0x00007ff3c3104055 _PyObject_Call 
#98  0x00007ff3c3233dd3 _PyEval_EvalFrameDefault 
#99  0x00007ff3c323c094  
#100 0x00007ff3c30e92f1 _PyObject_FastCallDictTstate 
#101 0x00007ff3c30e958d _PyObject_Call_Prepend 
#102 0x00007ff3c3109150  
#103 0x00007ff3c30e917c _PyObject_MakeTpCall 
#104 0x00007ff3c32335a2 _PyEval_EvalFrameDefault 
#105 0x00007ff3c323c094  
#106 0x00007ff3c30e92f1 _PyObject_FastCallDictTstate 
#107 0x00007ff3c30e958d _PyObject_Call_Prepend 
#108 0x00007ff3c3109150  
#109 0x00007ff3c30e917c _PyObject_MakeTpCall 
#110 0x00007ff3c32335a2 _PyEval_EvalFrameDefault 
#111 0x00007ff3c323c094  
#112 0x00007ff3c30e92f1 _PyObject_FastCallDictTstate 
#113 0x00007ff3c30e958d _PyObject_Call_Prepend 
#114 0x00007ff3c3109150  
#115 0x00007ff3c30e917c _PyObject_MakeTpCall 
#116 0x00007ff3c32335a2 _PyEval_EvalFrameDefault 
#117 0x00007ff3c323c094  
#118 0x00007ff3c30e92f1 _PyObject_FastCallDictTstate 
#119 0x00007ff3c30e958d _PyObject_Call_Prepend 
#120 0x00007ff3c3109150  
#121 0x00007ff3c30e917c _PyObject_MakeTpCall 
#122 0x00007ff3c32335a2 _PyEval_EvalFrameDefault 
#123 0x00007ff3c323c094  
#124 0x00007ff3c30e92f1 _PyObject_FastCallDictTstate 
#125 0x00007ff3c30e958d _PyObject_Call_Prepend 
#126 0x00007ff3c3109150  
#127 0x00007ff3c30e917c _PyObject_MakeTpCall 
#128 0x00007ff3c32335a2 _PyEval_EvalFrameDefault 
#129 0x00007ff3c323c094  
#130 0x00007ff3c30e92f1 _PyObject_FastCallDictTstate 
#131 0x00007ff3c30e958d _PyObject_Call_Prepend 
#132 0x00007ff3c3109150  
#133 0x00007ff3c30e917c _PyObject_MakeTpCall 
#134 0x00007ff3c32335a2 _PyEval_EvalFrameDefault 
#135 0x00007ff3c323c094  
#136 0x00007ff3c30e92f1 _PyObject_FastCallDictTstate 
#137 0x00007ff3c30e958d _PyObject_Call_Prepend 
#138 0x00007ff3c3109150  
#139 0x00007ff3c30e917c _PyObject_MakeTpCall 
#140 0x00007ff3c32335a2 _PyEval_EvalFrameDefault 
#141 0x00007ff3c323c094  
#142 0x00007ff3c30e92f1 _PyObject_FastCallDictTstate 
#143 0x00007ff3c30e958d _PyObject_Call_Prepend 
#144 0x00007ff3c3109150  
#145 0x00007ff3c30e917c _PyObject_MakeTpCall 
#146 0x00007ff3c32335a2 _PyEval_EvalFrameDefault 
#147 0x00007ff3c323c094  
#148 0x00007ff3c30e92f1 _PyObject_FastCallDictTstate 
#149 0x00007ff3c30e958d _PyObject_Call_Prepend 
#150 0x00007ff3c3109150  
#151 0x00007ff3c30e917c _PyObject_MakeTpCall 
#152 0x00007ff3c32335a2 _PyEval_EvalFrameDefault 
#153 0x00007ff3c323c094  
#154 0x00007ff3c30e92f1 _PyObject_FastCallDictTstate 
#155 0x00007ff3c30e958d _PyObject_Call_Prepend 
#156 0x00007ff3c3109150  
#157 0x00007ff3c30e917c _PyObject_MakeTpCall 
#158 0x00007ff3c32335a2 _PyEval_EvalFrameDefault 
#159 0x00007ff3c323c094  
#160 0x00007ff3c30e92f1 _PyObject_FastCallDictTstate 
#161 0x00007ff3c30e958d _PyObject_Call_Prepend 
#162 0x00007ff3c3109150  
#163 0x00007ff3c30e917c _PyObject_MakeTpCall 
#164 0x00007ff3c32335a2 _PyEval_EvalFrameDefault 
#165 0x00007ff3c323c094  
#166 0x00007ff3c30e92f1 _PyObject_FastCallDictTstate 
#167 0x00007ff3c30e958d _PyObject_Call_Prepend 
#168 0x00007ff3c3109150  
#169 0x00007ff3c30e917c _PyObject_MakeTpCall 
#170 0x00007ff3c32335a2 _PyEval_EvalFrameDefault 
#171 0x00007ff3c323c094  
#172 0x00007ff3c30e92f1 _PyObject_FastCallDictTstate 
#173 0x00007ff3c30e958d _PyObject_Call_Prepend 
#174 0x00007ff3c3109150  
#175 0x00007ff3c30e917c _PyObject_MakeTpCall 
#176 0x00007ff3c32335a2 _PyEval_EvalFrameDefault 
#177 0x00007ff3c323c094  
#178 0x00007ff3c30e92f1 _PyObject_FastCallDictTstate 
#179 0x00007ff3c30e958d _PyObject_Call_Prepend 
#180 0x00007ff3c3109150  
#181 0x00007ff3c30e917c _PyObject_MakeTpCall 
#182 0x00007ff3c32335a2 _PyEval_EvalFrameDefault 
#183 0x00007ff3c323c094  
#184 0x00007ff3c30e92f1 _PyObject_FastCallDictTstate 
#185 0x00007ff3c30e958d _PyObject_Call_Prepend 
#186 0x00007ff3c3109150  
#187 0x00007ff3c30e917c _PyObject_MakeTpCall 
#188 0x00007ff3c32335a2 _PyEval_EvalFrameDefault 
#189 0x00007ff3c323c094  
#190 0x00007ff3c3233dd3 _PyEval_EvalFrameDefault 
#191 0x00007ff3c323c094  
#192 0x00007ff3c30e92f1 _PyObject_FastCallDictTstate 
#193 0x00007ff3c30e958d _PyObject_Call_Prepend 
#194 0x00007ff3c3109150  
#195 0x00007ff3c30e917c _PyObject_MakeTpCall 
#196 0x00007ff3c32335a2 _PyEval_EvalFrameDefault 
#197 0x00007ff3c323c094  
#198 0x00007ff3c317d0fd  
#199 0x00007ff3c3233dd3 _PyEval_EvalFrameDefault 
#200 0x00007ff3c323c094  
#201 0x00007ff3c3233dd3 _PyEval_EvalFrameDefault 
#202 0x00007ff3c323c094  
#203 0x00007ff3c3233dd3 _PyEval_EvalFrameDefault 
#204 0x00007ff3c323c094  
#205 0x00007ff3c3233dd3 _PyEval_EvalFrameDefault 
#206 0x00007ff3c323c094  
#207 0x00007ff3c317d23c  
#208 0x00007ff3c31a7ec5  
#209 0x00007ff3c301ac77  
#210 0x00007ff3c357c573  
````

The fix implements two key changes.

1. **Hook functions (`memalloc_alloc`, `memalloc_realloc`)**: Snapshot the allocator struct locally before use and guard indirect function calls with `NULL` checks. This prevents crashes if a partially-written struct is observed during a start/stop race.

2. **Start/stop operations (`memalloc_start`, `memalloc_stop`)**: Use local variables and single assignments when publishing the allocator struct to `global_memalloc_ctx.pymem_allocator_obj`. This ensures concurrent hook calls observe either the old or new struct, never a partially-written intermediate state.

The real root cause is that `PyMem_GetAllocator` is not documented as atomic, and the struct could be read field-by-field while being written to concurrently.  By using local copies and single assignments, we ensure atomicity at the C level and prevent observation of inconsistent state.

Co-authored-by: thomas.kowalski <thomas.kowalski@datadoghq.com>
emmettbutler pushed a commit that referenced this pull request Apr 24, 2026
@KowalskiThomas @emmettbutler
fix(profiling): guard memalloc hook against partially-written saved a…
9579cfe 
…llocator (#17664)

## Description

This PR fixes a segmentation fault in the memory allocation profiler that occurs when a hook call races with `memalloc` start/stop operations. The issue arises from concurrent access to the saved allocator struct, which could be partially written while being read, resulting in`NULL` function pointers being dereferenced.  The key indicator in that case is that `#1 0x0000000000000000` frame -- we are trying to execute a null function pointer.

````
Error UnixSignal: Process terminated with SEGV_MAPERR (SIGSEGV)
#0   0x00007ff3c303a8d4  
#1   0x0000000000000000 memalloc_alloc (/go/src/github.com/DataDog/apm-reliability/dd-trace-py/ddtrace/profiling/collector/_memalloc.cpp:68)
#2   0x00007ff39dcb3b20 memalloc_alloc (/go/src/github.com/DataDog/apm-reliability/dd-trace-py/ddtrace/profiling/collector/_memalloc.cpp:68)
#3   0x00007ff39dcb3b20 memalloc_malloc(void*, unsigned long) (/go/src/github.com/DataDog/apm-reliability/dd-trace-py/ddtrace/profiling/collector/_memalloc.cpp:80)
#4   0x00007ff3c3087e1b PyUnicode_New 
#5   0x00007ff3c30889f4  
#6   0x00007ff3c3170c84  
#7   0x00007ff3c316b931  
#8   0x00007ff3c31aaac8  
#9   0x00007ff3c31033ac  
#10  0x00007ff3c310e2a6 PyObject_CallMethodObjArgs 
#11  0x00007ff3c310e46d  
#12  0x00007ff3c31a96c2  
#13  0x00007ff3c3102fd7 PyObject_Vectorcall 
#14  0x00007ff3c32335a2 _PyEval_EvalFrameDefault 
#15  0x00007ff3c323c094  
#16  0x00007ff3c3233dd3 _PyEval_EvalFrameDefault 
#17  0x00007ff3c323c094  
#18  0x00007ff3c30e997d PyObject_CallOneArg 
#19  0x00007ff3c306a480 _PyObject_GenericGetAttrWithDict 
#20  0x00007ff3c30c620d PyObject_GetAttr 
#21  0x00007ff3c32309e7 _PyEval_EvalFrameDefault 
#22  0x00007ff3c323c094  
#23  0x00007ff3c312880e  
#24  0x00007ff3c30e917c _PyObject_MakeTpCall 
#25  0x00007ff3c32335a2 _PyEval_EvalFrameDefault 
#26  0x00007ff3c323c094  
#27  0x00007ff3c312880e  
#28  0x00007ff3c30e917c _PyObject_MakeTpCall 
#29  0x00007ff3c32335a2 _PyEval_EvalFrameDefault 
#30  0x00007ff3c323c094  
#31  0x00007ff3c3233dd3 _PyEval_EvalFrameDefault 
#32  0x00007ff3c323c094  
#33  0x00007ff3c317d0fd  
#34  0x00007ff3c3233dd3 _PyEval_EvalFrameDefault 
#35  0x00007ff3c323c094  
#36  0x00007ff3c3233dd3 _PyEval_EvalFrameDefault 
#37  0x00007ff3c323c094  
#38  0x00007ff3c317d1b5  
#39  0x00007ff3c3102fd7 PyObject_Vectorcall 
#40  0x00007ff3c3232f4a _PyEval_EvalFrameDefault 
#41  0x00007ff3c3240da5  
#42  0x00007ff3c324112d  
#43  0x00007ff3c3233be1 _PyEval_EvalFrameDefault 
#44  0x00007ff3c323c094  
#45  0x00007ff3c317d1b5  
#46  0x00007ff3c3102fd7 PyObject_Vectorcall 
#47  0x00007ff3c3232f4a _PyEval_EvalFrameDefault 
#48  0x00007ff3c323c094  
#49  0x00007ff3c31033ac  
#50  0x00007ff3c310358d PyObject_CallFunctionObjArgs 
#51  0x00007ff3bf7eb91d WraptBoundFunctionWrapper_call (/project/src/wrapt/_wrappers.c:3750)
#52  0x00007ff3c3104055 _PyObject_Call 
#53  0x00007ff3c3233dd3 _PyEval_EvalFrameDefault 
#54  0x00007ff3c323c094  
#55  0x00007ff3c317d23c  
#56  0x00007ff3c3233dd3 _PyEval_EvalFrameDefault 
#57  0x00007ff3c323c094  
#58  0x00007ff3c310416f _PyObject_Call 
#59  0x00007ff3c3233dd3 _PyEval_EvalFrameDefault 
#60  0x00007ff3c3240da5  
#61  0x00007ff3c324112d  
#62  0x00007ff3c3233be1 _PyEval_EvalFrameDefault 
#63  0x00007ff3c323c094  
#64  0x00007ff3c317d1b5  
#65  0x00007ff3c3102fd7 PyObject_Vectorcall 
#66  0x00007ff3c3232f4a _PyEval_EvalFrameDefault 
#67  0x00007ff3c323c094  
#68  0x00007ff3c317d1b5  
#69  0x00007ff3c310416f _PyObject_Call 
#70  0x00007ff3c3233dd3 _PyEval_EvalFrameDefault 
#71  0x00007ff3c323c094  
#72  0x00007ff3c317d1b5  
#73  0x00007ff3c310416f _PyObject_Call 
#74  0x00007ff3c3233dd3 _PyEval_EvalFrameDefault 
#75  0x00007ff3c323c094  
#76  0x00007ff3c31033ac  
#77  0x00007ff3c310358d PyObject_CallFunctionObjArgs 
#78  0x00007ff3bf7eb91d WraptBoundFunctionWrapper_call (/project/src/wrapt/_wrappers.c:3750)
#79  0x00007ff3c30e917c _PyObject_MakeTpCall 
#80  0x00007ff3c32335a2 _PyEval_EvalFrameDefault 
#81  0x00007ff3c323c094  
#82  0x00007ff3c317d518  
#83  0x00007ff3c3155963  
#84  0x00007ff3c315393d  
#85  0x00007ff3c3233dd3 _PyEval_EvalFrameDefault 
#86  0x00007ff3c323c094  
#87  0x00007ff3c317d0fd  
#88  0x00007ff3c3233dd3 _PyEval_EvalFrameDefault 
#89  0x00007ff3c323c094  
#90  0x00007ff3c317d0fd  
#91  0x00007ff3c317d518  
#92  0x00007ff3c3233dd3 _PyEval_EvalFrameDefault 
#93  0x00007ff3c323c094  
#94  0x00007ff3c30e9371 _PyObject_FastCallDictTstate 
#95  0x00007ff3c30e958d _PyObject_Call_Prepend 
#96  0x00007ff3c3109150  
#97  0x00007ff3c3104055 _PyObject_Call 
#98  0x00007ff3c3233dd3 _PyEval_EvalFrameDefault 
#99  0x00007ff3c323c094  
#100 0x00007ff3c30e92f1 _PyObject_FastCallDictTstate 
#101 0x00007ff3c30e958d _PyObject_Call_Prepend 
#102 0x00007ff3c3109150  
#103 0x00007ff3c30e917c _PyObject_MakeTpCall 
#104 0x00007ff3c32335a2 _PyEval_EvalFrameDefault 
#105 0x00007ff3c323c094  
#106 0x00007ff3c30e92f1 _PyObject_FastCallDictTstate 
#107 0x00007ff3c30e958d _PyObject_Call_Prepend 
#108 0x00007ff3c3109150  
#109 0x00007ff3c30e917c _PyObject_MakeTpCall 
#110 0x00007ff3c32335a2 _PyEval_EvalFrameDefault 
#111 0x00007ff3c323c094  
#112 0x00007ff3c30e92f1 _PyObject_FastCallDictTstate 
#113 0x00007ff3c30e958d _PyObject_Call_Prepend 
#114 0x00007ff3c3109150  
#115 0x00007ff3c30e917c _PyObject_MakeTpCall 
#116 0x00007ff3c32335a2 _PyEval_EvalFrameDefault 
#117 0x00007ff3c323c094  
#118 0x00007ff3c30e92f1 _PyObject_FastCallDictTstate 
#119 0x00007ff3c30e958d _PyObject_Call_Prepend 
#120 0x00007ff3c3109150  
#121 0x00007ff3c30e917c _PyObject_MakeTpCall 
#122 0x00007ff3c32335a2 _PyEval_EvalFrameDefault 
#123 0x00007ff3c323c094  
#124 0x00007ff3c30e92f1 _PyObject_FastCallDictTstate 
#125 0x00007ff3c30e958d _PyObject_Call_Prepend 
#126 0x00007ff3c3109150  
#127 0x00007ff3c30e917c _PyObject_MakeTpCall 
#128 0x00007ff3c32335a2 _PyEval_EvalFrameDefault 
#129 0x00007ff3c323c094  
#130 0x00007ff3c30e92f1 _PyObject_FastCallDictTstate 
#131 0x00007ff3c30e958d _PyObject_Call_Prepend 
#132 0x00007ff3c3109150  
#133 0x00007ff3c30e917c _PyObject_MakeTpCall 
#134 0x00007ff3c32335a2 _PyEval_EvalFrameDefault 
#135 0x00007ff3c323c094  
#136 0x00007ff3c30e92f1 _PyObject_FastCallDictTstate 
#137 0x00007ff3c30e958d _PyObject_Call_Prepend 
#138 0x00007ff3c3109150  
#139 0x00007ff3c30e917c _PyObject_MakeTpCall 
#140 0x00007ff3c32335a2 _PyEval_EvalFrameDefault 
#141 0x00007ff3c323c094  
#142 0x00007ff3c30e92f1 _PyObject_FastCallDictTstate 
#143 0x00007ff3c30e958d _PyObject_Call_Prepend 
#144 0x00007ff3c3109150  
#145 0x00007ff3c30e917c _PyObject_MakeTpCall 
#146 0x00007ff3c32335a2 _PyEval_EvalFrameDefault 
#147 0x00007ff3c323c094  
#148 0x00007ff3c30e92f1 _PyObject_FastCallDictTstate 
#149 0x00007ff3c30e958d _PyObject_Call_Prepend 
#150 0x00007ff3c3109150  
#151 0x00007ff3c30e917c _PyObject_MakeTpCall 
#152 0x00007ff3c32335a2 _PyEval_EvalFrameDefault 
#153 0x00007ff3c323c094  
#154 0x00007ff3c30e92f1 _PyObject_FastCallDictTstate 
#155 0x00007ff3c30e958d _PyObject_Call_Prepend 
#156 0x00007ff3c3109150  
#157 0x00007ff3c30e917c _PyObject_MakeTpCall 
#158 0x00007ff3c32335a2 _PyEval_EvalFrameDefault 
#159 0x00007ff3c323c094  
#160 0x00007ff3c30e92f1 _PyObject_FastCallDictTstate 
#161 0x00007ff3c30e958d _PyObject_Call_Prepend 
#162 0x00007ff3c3109150  
#163 0x00007ff3c30e917c _PyObject_MakeTpCall 
#164 0x00007ff3c32335a2 _PyEval_EvalFrameDefault 
#165 0x00007ff3c323c094  
#166 0x00007ff3c30e92f1 _PyObject_FastCallDictTstate 
#167 0x00007ff3c30e958d _PyObject_Call_Prepend 
#168 0x00007ff3c3109150  
#169 0x00007ff3c30e917c _PyObject_MakeTpCall 
#170 0x00007ff3c32335a2 _PyEval_EvalFrameDefault 
#171 0x00007ff3c323c094  
#172 0x00007ff3c30e92f1 _PyObject_FastCallDictTstate 
#173 0x00007ff3c30e958d _PyObject_Call_Prepend 
#174 0x00007ff3c3109150  
#175 0x00007ff3c30e917c _PyObject_MakeTpCall 
#176 0x00007ff3c32335a2 _PyEval_EvalFrameDefault 
#177 0x00007ff3c323c094  
#178 0x00007ff3c30e92f1 _PyObject_FastCallDictTstate 
#179 0x00007ff3c30e958d _PyObject_Call_Prepend 
#180 0x00007ff3c3109150  
#181 0x00007ff3c30e917c _PyObject_MakeTpCall 
#182 0x00007ff3c32335a2 _PyEval_EvalFrameDefault 
#183 0x00007ff3c323c094  
#184 0x00007ff3c30e92f1 _PyObject_FastCallDictTstate 
#185 0x00007ff3c30e958d _PyObject_Call_Prepend 
#186 0x00007ff3c3109150  
#187 0x00007ff3c30e917c _PyObject_MakeTpCall 
#188 0x00007ff3c32335a2 _PyEval_EvalFrameDefault 
#189 0x00007ff3c323c094  
#190 0x00007ff3c3233dd3 _PyEval_EvalFrameDefault 
#191 0x00007ff3c323c094  
#192 0x00007ff3c30e92f1 _PyObject_FastCallDictTstate 
#193 0x00007ff3c30e958d _PyObject_Call_Prepend 
#194 0x00007ff3c3109150  
#195 0x00007ff3c30e917c _PyObject_MakeTpCall 
#196 0x00007ff3c32335a2 _PyEval_EvalFrameDefault 
#197 0x00007ff3c323c094  
#198 0x00007ff3c317d0fd  
#199 0x00007ff3c3233dd3 _PyEval_EvalFrameDefault 
#200 0x00007ff3c323c094  
#201 0x00007ff3c3233dd3 _PyEval_EvalFrameDefault 
#202 0x00007ff3c323c094  
#203 0x00007ff3c3233dd3 _PyEval_EvalFrameDefault 
#204 0x00007ff3c323c094  
#205 0x00007ff3c3233dd3 _PyEval_EvalFrameDefault 
#206 0x00007ff3c323c094  
#207 0x00007ff3c317d23c  
#208 0x00007ff3c31a7ec5  
#209 0x00007ff3c301ac77  
#210 0x00007ff3c357c573  
````

The fix implements two key changes.

1. **Hook functions (`memalloc_alloc`, `memalloc_realloc`)**: Snapshot the allocator struct locally before use and guard indirect function calls with `NULL` checks. This prevents crashes if a partially-written struct is observed during a start/stop race.

2. **Start/stop operations (`memalloc_start`, `memalloc_stop`)**: Use local variables and single assignments when publishing the allocator struct to `global_memalloc_ctx.pymem_allocator_obj`. This ensures concurrent hook calls observe either the old or new struct, never a partially-written intermediate state.

The real root cause is that `PyMem_GetAllocator` is not documented as atomic, and the struct could be read field-by-field while being written to concurrently.  By using local copies and single assignments, we ensure atomicity at the C level and prevent observation of inconsistent state.

Co-authored-by: thomas.kowalski <thomas.kowalski@datadoghq.com>
emmettbutler pushed a commit that referenced this pull request May 6, 2026
@KowalskiThomas @emmettbutler
fix(profiling): guard memalloc hook against partially-written saved a…
3ae7f5a 
…llocator (#17664)

## Description

This PR fixes a segmentation fault in the memory allocation profiler that occurs when a hook call races with `memalloc` start/stop operations. The issue arises from concurrent access to the saved allocator struct, which could be partially written while being read, resulting in`NULL` function pointers being dereferenced.  The key indicator in that case is that `#1 0x0000000000000000` frame -- we are trying to execute a null function pointer.

````
Error UnixSignal: Process terminated with SEGV_MAPERR (SIGSEGV)
#0   0x00007ff3c303a8d4  
#1   0x0000000000000000 memalloc_alloc (/go/src/github.com/DataDog/apm-reliability/dd-trace-py/ddtrace/profiling/collector/_memalloc.cpp:68)
#2   0x00007ff39dcb3b20 memalloc_alloc (/go/src/github.com/DataDog/apm-reliability/dd-trace-py/ddtrace/profiling/collector/_memalloc.cpp:68)
#3   0x00007ff39dcb3b20 memalloc_malloc(void*, unsigned long) (/go/src/github.com/DataDog/apm-reliability/dd-trace-py/ddtrace/profiling/collector/_memalloc.cpp:80)
#4   0x00007ff3c3087e1b PyUnicode_New 
#5   0x00007ff3c30889f4  
#6   0x00007ff3c3170c84  
#7   0x00007ff3c316b931  
#8   0x00007ff3c31aaac8  
#9   0x00007ff3c31033ac  
#10  0x00007ff3c310e2a6 PyObject_CallMethodObjArgs 
#11  0x00007ff3c310e46d  
#12  0x00007ff3c31a96c2  
#13  0x00007ff3c3102fd7 PyObject_Vectorcall 
#14  0x00007ff3c32335a2 _PyEval_EvalFrameDefault 
#15  0x00007ff3c323c094  
#16  0x00007ff3c3233dd3 _PyEval_EvalFrameDefault 
#17  0x00007ff3c323c094  
#18  0x00007ff3c30e997d PyObject_CallOneArg 
#19  0x00007ff3c306a480 _PyObject_GenericGetAttrWithDict 
#20  0x00007ff3c30c620d PyObject_GetAttr 
#21  0x00007ff3c32309e7 _PyEval_EvalFrameDefault 
#22  0x00007ff3c323c094  
#23  0x00007ff3c312880e  
#24  0x00007ff3c30e917c _PyObject_MakeTpCall 
#25  0x00007ff3c32335a2 _PyEval_EvalFrameDefault 
#26  0x00007ff3c323c094  
#27  0x00007ff3c312880e  
#28  0x00007ff3c30e917c _PyObject_MakeTpCall 
#29  0x00007ff3c32335a2 _PyEval_EvalFrameDefault 
#30  0x00007ff3c323c094  
#31  0x00007ff3c3233dd3 _PyEval_EvalFrameDefault 
#32  0x00007ff3c323c094  
#33  0x00007ff3c317d0fd  
#34  0x00007ff3c3233dd3 _PyEval_EvalFrameDefault 
#35  0x00007ff3c323c094  
#36  0x00007ff3c3233dd3 _PyEval_EvalFrameDefault 
#37  0x00007ff3c323c094  
#38  0x00007ff3c317d1b5  
#39  0x00007ff3c3102fd7 PyObject_Vectorcall 
#40  0x00007ff3c3232f4a _PyEval_EvalFrameDefault 
#41  0x00007ff3c3240da5  
#42  0x00007ff3c324112d  
#43  0x00007ff3c3233be1 _PyEval_EvalFrameDefault 
#44  0x00007ff3c323c094  
#45  0x00007ff3c317d1b5  
#46  0x00007ff3c3102fd7 PyObject_Vectorcall 
#47  0x00007ff3c3232f4a _PyEval_EvalFrameDefault 
#48  0x00007ff3c323c094  
#49  0x00007ff3c31033ac  
#50  0x00007ff3c310358d PyObject_CallFunctionObjArgs 
#51  0x00007ff3bf7eb91d WraptBoundFunctionWrapper_call (/project/src/wrapt/_wrappers.c:3750)
#52  0x00007ff3c3104055 _PyObject_Call 
#53  0x00007ff3c3233dd3 _PyEval_EvalFrameDefault 
#54  0x00007ff3c323c094  
#55  0x00007ff3c317d23c  
#56  0x00007ff3c3233dd3 _PyEval_EvalFrameDefault 
#57  0x00007ff3c323c094  
#58  0x00007ff3c310416f _PyObject_Call 
#59  0x00007ff3c3233dd3 _PyEval_EvalFrameDefault 
#60  0x00007ff3c3240da5  
#61  0x00007ff3c324112d  
#62  0x00007ff3c3233be1 _PyEval_EvalFrameDefault 
#63  0x00007ff3c323c094  
#64  0x00007ff3c317d1b5  
#65  0x00007ff3c3102fd7 PyObject_Vectorcall 
#66  0x00007ff3c3232f4a _PyEval_EvalFrameDefault 
#67  0x00007ff3c323c094  
#68  0x00007ff3c317d1b5  
#69  0x00007ff3c310416f _PyObject_Call 
#70  0x00007ff3c3233dd3 _PyEval_EvalFrameDefault 
#71  0x00007ff3c323c094  
#72  0x00007ff3c317d1b5  
#73  0x00007ff3c310416f _PyObject_Call 
#74  0x00007ff3c3233dd3 _PyEval_EvalFrameDefault 
#75  0x00007ff3c323c094  
#76  0x00007ff3c31033ac  
#77  0x00007ff3c310358d PyObject_CallFunctionObjArgs 
#78  0x00007ff3bf7eb91d WraptBoundFunctionWrapper_call (/project/src/wrapt/_wrappers.c:3750)
#79  0x00007ff3c30e917c _PyObject_MakeTpCall 
#80  0x00007ff3c32335a2 _PyEval_EvalFrameDefault 
#81  0x00007ff3c323c094  
#82  0x00007ff3c317d518  
#83  0x00007ff3c3155963  
#84  0x00007ff3c315393d  
#85  0x00007ff3c3233dd3 _PyEval_EvalFrameDefault 
#86  0x00007ff3c323c094  
#87  0x00007ff3c317d0fd  
#88  0x00007ff3c3233dd3 _PyEval_EvalFrameDefault 
#89  0x00007ff3c323c094  
#90  0x00007ff3c317d0fd  
#91  0x00007ff3c317d518  
#92  0x00007ff3c3233dd3 _PyEval_EvalFrameDefault 
#93  0x00007ff3c323c094  
#94  0x00007ff3c30e9371 _PyObject_FastCallDictTstate 
#95  0x00007ff3c30e958d _PyObject_Call_Prepend 
#96  0x00007ff3c3109150  
#97  0x00007ff3c3104055 _PyObject_Call 
#98  0x00007ff3c3233dd3 _PyEval_EvalFrameDefault 
#99  0x00007ff3c323c094  
#100 0x00007ff3c30e92f1 _PyObject_FastCallDictTstate 
#101 0x00007ff3c30e958d _PyObject_Call_Prepend 
#102 0x00007ff3c3109150  
#103 0x00007ff3c30e917c _PyObject_MakeTpCall 
#104 0x00007ff3c32335a2 _PyEval_EvalFrameDefault 
#105 0x00007ff3c323c094  
#106 0x00007ff3c30e92f1 _PyObject_FastCallDictTstate 
#107 0x00007ff3c30e958d _PyObject_Call_Prepend 
#108 0x00007ff3c3109150  
#109 0x00007ff3c30e917c _PyObject_MakeTpCall 
#110 0x00007ff3c32335a2 _PyEval_EvalFrameDefault 
#111 0x00007ff3c323c094  
#112 0x00007ff3c30e92f1 _PyObject_FastCallDictTstate 
#113 0x00007ff3c30e958d _PyObject_Call_Prepend 
#114 0x00007ff3c3109150  
#115 0x00007ff3c30e917c _PyObject_MakeTpCall 
#116 0x00007ff3c32335a2 _PyEval_EvalFrameDefault 
#117 0x00007ff3c323c094  
#118 0x00007ff3c30e92f1 _PyObject_FastCallDictTstate 
#119 0x00007ff3c30e958d _PyObject_Call_Prepend 
#120 0x00007ff3c3109150  
#121 0x00007ff3c30e917c _PyObject_MakeTpCall 
#122 0x00007ff3c32335a2 _PyEval_EvalFrameDefault 
#123 0x00007ff3c323c094  
#124 0x00007ff3c30e92f1 _PyObject_FastCallDictTstate 
#125 0x00007ff3c30e958d _PyObject_Call_Prepend 
#126 0x00007ff3c3109150  
#127 0x00007ff3c30e917c _PyObject_MakeTpCall 
#128 0x00007ff3c32335a2 _PyEval_EvalFrameDefault 
#129 0x00007ff3c323c094  
#130 0x00007ff3c30e92f1 _PyObject_FastCallDictTstate 
#131 0x00007ff3c30e958d _PyObject_Call_Prepend 
#132 0x00007ff3c3109150  
#133 0x00007ff3c30e917c _PyObject_MakeTpCall 
#134 0x00007ff3c32335a2 _PyEval_EvalFrameDefault 
#135 0x00007ff3c323c094  
#136 0x00007ff3c30e92f1 _PyObject_FastCallDictTstate 
#137 0x00007ff3c30e958d _PyObject_Call_Prepend 
#138 0x00007ff3c3109150  
#139 0x00007ff3c30e917c _PyObject_MakeTpCall 
#140 0x00007ff3c32335a2 _PyEval_EvalFrameDefault 
#141 0x00007ff3c323c094  
#142 0x00007ff3c30e92f1 _PyObject_FastCallDictTstate 
#143 0x00007ff3c30e958d _PyObject_Call_Prepend 
#144 0x00007ff3c3109150  
#145 0x00007ff3c30e917c _PyObject_MakeTpCall 
#146 0x00007ff3c32335a2 _PyEval_EvalFrameDefault 
#147 0x00007ff3c323c094  
#148 0x00007ff3c30e92f1 _PyObject_FastCallDictTstate 
#149 0x00007ff3c30e958d _PyObject_Call_Prepend 
#150 0x00007ff3c3109150  
#151 0x00007ff3c30e917c _PyObject_MakeTpCall 
#152 0x00007ff3c32335a2 _PyEval_EvalFrameDefault 
#153 0x00007ff3c323c094  
#154 0x00007ff3c30e92f1 _PyObject_FastCallDictTstate 
#155 0x00007ff3c30e958d _PyObject_Call_Prepend 
#156 0x00007ff3c3109150  
#157 0x00007ff3c30e917c _PyObject_MakeTpCall 
#158 0x00007ff3c32335a2 _PyEval_EvalFrameDefault 
#159 0x00007ff3c323c094  
#160 0x00007ff3c30e92f1 _PyObject_FastCallDictTstate 
#161 0x00007ff3c30e958d _PyObject_Call_Prepend 
#162 0x00007ff3c3109150  
#163 0x00007ff3c30e917c _PyObject_MakeTpCall 
#164 0x00007ff3c32335a2 _PyEval_EvalFrameDefault 
#165 0x00007ff3c323c094  
#166 0x00007ff3c30e92f1 _PyObject_FastCallDictTstate 
#167 0x00007ff3c30e958d _PyObject_Call_Prepend 
#168 0x00007ff3c3109150  
#169 0x00007ff3c30e917c _PyObject_MakeTpCall 
#170 0x00007ff3c32335a2 _PyEval_EvalFrameDefault 
#171 0x00007ff3c323c094  
#172 0x00007ff3c30e92f1 _PyObject_FastCallDictTstate 
#173 0x00007ff3c30e958d _PyObject_Call_Prepend 
#174 0x00007ff3c3109150  
#175 0x00007ff3c30e917c _PyObject_MakeTpCall 
#176 0x00007ff3c32335a2 _PyEval_EvalFrameDefault 
#177 0x00007ff3c323c094  
#178 0x00007ff3c30e92f1 _PyObject_FastCallDictTstate 
#179 0x00007ff3c30e958d _PyObject_Call_Prepend 
#180 0x00007ff3c3109150  
#181 0x00007ff3c30e917c _PyObject_MakeTpCall 
#182 0x00007ff3c32335a2 _PyEval_EvalFrameDefault 
#183 0x00007ff3c323c094  
#184 0x00007ff3c30e92f1 _PyObject_FastCallDictTstate 
#185 0x00007ff3c30e958d _PyObject_Call_Prepend 
#186 0x00007ff3c3109150  
#187 0x00007ff3c30e917c _PyObject_MakeTpCall 
#188 0x00007ff3c32335a2 _PyEval_EvalFrameDefault 
#189 0x00007ff3c323c094  
#190 0x00007ff3c3233dd3 _PyEval_EvalFrameDefault 
#191 0x00007ff3c323c094  
#192 0x00007ff3c30e92f1 _PyObject_FastCallDictTstate 
#193 0x00007ff3c30e958d _PyObject_Call_Prepend 
#194 0x00007ff3c3109150  
#195 0x00007ff3c30e917c _PyObject_MakeTpCall 
#196 0x00007ff3c32335a2 _PyEval_EvalFrameDefault 
#197 0x00007ff3c323c094  
#198 0x00007ff3c317d0fd  
#199 0x00007ff3c3233dd3 _PyEval_EvalFrameDefault 
#200 0x00007ff3c323c094  
#201 0x00007ff3c3233dd3 _PyEval_EvalFrameDefault 
#202 0x00007ff3c323c094  
#203 0x00007ff3c3233dd3 _PyEval_EvalFrameDefault 
#204 0x00007ff3c323c094  
#205 0x00007ff3c3233dd3 _PyEval_EvalFrameDefault 
#206 0x00007ff3c323c094  
#207 0x00007ff3c317d23c  
#208 0x00007ff3c31a7ec5  
#209 0x00007ff3c301ac77  
#210 0x00007ff3c357c573  
````

The fix implements two key changes.

1. **Hook functions (`memalloc_alloc`, `memalloc_realloc`)**: Snapshot the allocator struct locally before use and guard indirect function calls with `NULL` checks. This prevents crashes if a partially-written struct is observed during a start/stop race.

2. **Start/stop operations (`memalloc_start`, `memalloc_stop`)**: Use local variables and single assignments when publishing the allocator struct to `global_memalloc_ctx.pymem_allocator_obj`. This ensures concurrent hook calls observe either the old or new struct, never a partially-written intermediate state.

The real root cause is that `PyMem_GetAllocator` is not documented as atomic, and the struct could be read field-by-field while being written to concurrently.  By using local copies and single assignments, we ensure atomicity at the C level and prevent observation of inconsistent state.

Co-authored-by: thomas.kowalski <thomas.kowalski@datadoghq.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@palazzem palazzem Awaiting requested review from palazzem

2 more reviewers

@clutchski clutchski clutchski left review comments

@JTCunning JTCunning JTCunning left review comments

Reviewers whose approvals may not affect merge requirements

Assignees

@palazzem palazzem

Labels

integrations

Projects

None yet

Milestone

0.6.0

Development

Successfully merging this pull request may close these issues.

7 participants

@brettlangdon @jontonsoup @palazzem @kichooo @clutchski @JTCunning @ufoot