-
Notifications
You must be signed in to change notification settings - Fork 291
Add pinned byte array builtin #5832
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.
dolio
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The code looks fine to me.
My only question has to do with e.g. fillBuf. For the non-socket versions, they don't take a size, and just use the whole buffer. Should there be sized versions, too? I guess the normal situation with a buffer is that you just use it for uniform chunking, and don't reuse a larger buffer than necessary for multiple chunk sizes. But I thought I'd bring it up.
pchiusano
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Cool.
For fillBuf, I kind of like the idea of it taking a Nat to control how many bytes it's waiting for. Even if 99% of the time you just pass in the buffer size, it seems more flexible.
|
I decided to just fill the buffer so we wouldn't have to do a range check, since in the most common case you're just filling the buffer. Taking a |
|
But then I guess the |
|
We can just provide both, I think that's the move. |
Overview
This change adds
PinnedByteArraysupport to Unison, enabling efficient pinned memory management for low-level I/O operations and system calls. Previously, Unison only hadMutableByteArrayfor mutable byte operations, but lacked the ability to create pinned arrays that prevent the garbage collector from moving memory during critical operations.Old behavior: Users could only work with regular mutable byte arrays that could be moved by the garbage collector, making them unsuitable for system calls, certain I/O operations, or FFI that require stable memory addresses.
New behavior: Users can now create pinned byte arrays using e.g.
IO.pinnedByteArrayensuring memory remains at a fixed address during critical operations. They can be cast to mutable arrays usingPinnedByteArray.cast, so wherever aMutableByteArraycan be used, aPinnedByteArraycan also be used after casting. Operations that require pinned memory usePinnedByteArraydirectly. Crucially, aMutableByteArraycannot be cast to aPinnedByteArray, which otherwise would cause segfaults.Example usage:
This is particularly useful for:
Fixed buffer I/O
This PR also adds
SocketandHandleI/O operations that use pinned memory:IO.socketSendBuf- Sends a number of bytes from aPinnedByteArrayto aSocket.IO.socketReceiveBuf- Blocking read of some number of bytes from aSocketinto aPinnedByteArray.IO.putBuf- Writes a number of bytes from aPinnedByteArrayto aHandle.IO.getBufSome- Reads whatever bytes are available on aHandleto aPinnedByteArray. Waits only if no bytes are available.IO.fillBuf- Waits until enough bytes are available on aHandleto fill aPinnedByteArray, then reads them into the array.Test coverage
Operations on pinned arrays are exercised via transcripts.