Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 82.65%. Comparing base (53744b2) to head (42a54f0).
⚠️ Report is 3 commits behind head on main.

The documentation preview is ready to be viewed at http://preview.awkward-array.org.s3-website.us-east-1.amazonaws.com/PR3829

@ikrommyd - it looks like the two high level arrays that supposed to be immutable have been moved — does it mean that awkward is not thread-safe?

The high-level arrays are "kinda" mutable when it comes to __setitem__ (which uses ak.with_field). Also the python layouts are immutable but their buffers are not. And we are not in control of what jax does to the buffers internally.

Well awkward is not thread-safe in the same sense that numpy is not thread-safe. You can have multiple threads modifying a buffer and there are no locks inside numpy protecting you. The python part of awkward should be thread safe but that doesn't mean you should share arrays between threads and start doing in place mutations to the buffers.

Also the issue may be from jax here. I don't think you should spawn threads that do stuff to the same jax buffer either. We hit the error when we tried to get the unsafe_buffer_pointer of a jax buffer because something caused the buffer to be "sharded". The buffers themselves are not thread-safe if multiple threads try to do something to them.

Awkward is thread safe in the sense that you can spawn multiple python threads running awkward code without a problem. That's what we test in the test with pytest-run-parallel (that we can run the test functions in parallel without a problem). This is the same kind of testing that numpy has and the same "thread-safety".

https://numpy.org/doc/stable/reference/thread_safety.html