Skip to content

Add a convenience for making local streams in python#3355

Merged
angeloskath merged 4 commits into
mainfrom
local-stream
Apr 3, 2026
Merged

Add a convenience for making local streams in python#3355
angeloskath merged 4 commits into
mainfrom
local-stream

Conversation

@angeloskath

@angeloskath angeloskath commented Apr 2, 2026

Copy link
Copy Markdown
Member

This PR started as a small PR to make it easy to create thread local streams in Python. But I think we had the teardown logic quite wrong but it didn't show up until running the server which does the computation in a background dedicated thread.

Basically, thread local objects are torn down when the thread exits which is likely to be before the interpreter exits. In that case the destructors of C++ objects will run and try to clean up Python objects without holding the GIL. So the fix is the standard hold the GIL and clean up in destructor situation.

Regarding the main thread though, the interpreter will probably be torn down before the thread local objects. So when the destructors run there will be no python and no GIL and segfault. So we register exit handlers that clean up and the C++ objects check if there is cleanup to do or do nothing.

The above means that correct usage of MLX requires either manual clean up of some kind, or all threads to join the main thread before exiting.

@angeloskath angeloskath requested a review from zcbenz April 2, 2026 06:04
@angeloskath angeloskath merged commit 6a9a121 into main Apr 3, 2026
16 checks passed
@angeloskath angeloskath deleted the local-stream branch April 3, 2026 01:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants