Add generic FileRegion support in io_uring stream channel (#16571)#16677
Merged
Conversation
`AbstractIoUringStreamChannel` only handles `DefaultFileRegion` (via splice). Any other `FileRegion` implementation hits `UnsupportedOperationException` from the parent's `filterOutboundMessage`. This is inconsistent with EPOLL and NIO, which accept any `FileRegion` via a `transferTo` fallback. Frameworks like Apache Spark wrap `DefaultFileRegion` inside composite `FileRegion` implementations (e.g., `MessageWithHeader`) that work on NIO and EPOLL but crash on io_uring. See #16570 For generic `FileRegion` (non-`DefaultFileRegion` or when splice is unsupported), let it pass through `filterOutboundMessage` into the outbound buffer, then convert it to a direct `ByteBuf` in **64 KB chunks** via `FileRegion.transferTo` in the write path (`scheduleWriteSingle` / `writeComplete0`). - `DefaultFileRegion` + splice path is unchanged. - Data is chunked to limit memory usage — each chunk is read via `transferTo` into a capped `ByteBufWritableByteChannel`, then submitted as an io_uring async send. - Partial sends and EAGAIN preserve the chunk buffer for retry; unrecoverable errors release it and fail the write. - Empty FileRegions (count = 0) submit a 0-byte send so the completion path can remove them. Also enabled `testCustomFileRegion` in `IoUringSocketFileRegionTest` and `IoUringBufferRingSocketFileRegionTest`. Any `FileRegion` implementation is now writable on io_uring, consistent with EPOLL and NIO. | Transport | `DefaultFileRegion` | Generic `FileRegion` | |---|---|---| | NIO | `FileChannel.transferTo` | `region.transferTo` | | EPOLL | native sendfile | `region.transferTo` fallback | | io_uring (before) | splice | **UnsupportedOperationException** | | io_uring (after) | splice (unchanged) | chunked `transferTo` → ByteBuf → async send | --------- Co-authored-by: Norman Maurer <norman_maurer@apple.com>
LuciferYang
approved these changes
Apr 22, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
AbstractIoUringStreamChannelonly handlesDefaultFileRegion(viasplice). Any other
FileRegionimplementation hitsUnsupportedOperationExceptionfrom the parent'sfilterOutboundMessage.This is inconsistent with EPOLL and NIO, which accept any
FileRegionvia a
transferTofallback. Frameworks like Apache Spark wrapDefaultFileRegioninside compositeFileRegionimplementations (e.g.,MessageWithHeader) that work on NIO and EPOLL but crash on io_uring.See #16570
For generic
FileRegion(non-DefaultFileRegionor when splice isunsupported), let it pass through
filterOutboundMessageinto theoutbound buffer, then convert it to a direct
ByteBufin 64 KBchunks via
FileRegion.transferToin the write path(
scheduleWriteSingle/writeComplete0).DefaultFileRegion+ splice path is unchanged.transferTointo a cappedByteBufWritableByteChannel, then submittedas an io_uring async send.
unrecoverable errors release it and fail the write.
path can remove them.
Also enabled
testCustomFileRegioninIoUringSocketFileRegionTestandIoUringBufferRingSocketFileRegionTest.Any
FileRegionimplementation is now writable on io_uring, consistentwith EPOLL and NIO.
DefaultFileRegionFileRegionFileChannel.transferToregion.transferToregion.transferTofallbacktransferTo→ ByteBufCo-authored-by: Norman Maurer norman_maurer@apple.com