Motivation:

In our testsuite we often just wrapped a byte[] and passed it to the write method. While this works it also might end up not testing often with direct buffers. We should make this a bit more random and so cover direct and heap buffers. The idea popped up when reviewing #15231

Modifications:

- Add new static helper method that will either return a wrapped buffer or a direct buffers in a random fashion
- Use this method to allocate buffers

Result:

More random testing of heap / direct buffers

…5281)

Motivation:

In our testsuite we often just wrapped a byte[] and passed it to the
write method. While this works it also might end up not testing often
with direct buffers. We should make this a bit more random and so cover
direct and heap buffers. The idea popped up when reviewing
#15231

Modifications:

- Add new static helper method that will either return a wrapped buffer
or a direct buffers in a random fashion
- Use this method to allocate buffers

Result:

More random testing of heap / direct buffers

Motivation:
In netty#14943 we disabled Unsafe by default on Java 24+, to avoid producing warnings from the JVM.
However, disabling the use of Unsafe also disables the hacks we use to access `ByteBuffer` cleaners to eagerly deallocate them.
This can have dire performance consequences, particularly for busy applications.

Disabling the warnings from Unsafe also did nothing to stem the warnings produced by our JNI code, so many systems still see warnings anyway.

We should instead revert the changed default, and just tolerate the warning in Netty 4.1.

In Netty 4.2 we have netty#15231 as a long-term fix, where we avoid using Unsafe but will have a way to eagerly release the memory of our native ByteBuffer instances.

Modification:
Change the default for our Unsafe usage to no longer be disabled on Java 24+.

Result:
More consistent performance across Java versions.

See https://github.com/netty/netty/wiki/Java-24-and-sun.misc.Unsafe for
more information.

Motivation:
In #14943 we disabled Unsafe by
default on Java 24+, to avoid producing warnings from the JVM. However,
disabling the use of Unsafe also disables the hacks we use to access
`ByteBuffer` cleaners to eagerly deallocate them. This can have dire
performance consequences, particularly for busy applications.

Disabling the warnings from Unsafe also did nothing to stem the warnings
produced by our JNI code, so many systems still see warnings anyway.

We should instead revert the changed default, and just tolerate the
warning in Netty 4.1.

In Netty 4.2 we have #15231 as a
long-term fix, where we avoid using Unsafe but will have a way to
eagerly release the memory of our native ByteBuffer instances.

Modification:
Change the default for our Unsafe usage to no longer be disabled on Java
24+.

Result:
More consistent performance across Java versions.

Motivation:
Currently, the preference if a direct buffer should be allocated by, for
example, `ByteBufAllocator.buffer()` or `ByteBufAllocator.ioBuffer()`
depends on the fact if unsafe is available. It seems like this was
originally introduced as only with unsafe it was possible to get the
cleaner of a ByteBuffer to clean it. However, with the changes
introduced in #15231, it doesn't make sense anymore to depend on this as
unsafe could be unavailable but MemorySegments will still be freeable.

Modification:
Switch the check for unsafe to a check that validates if the current
cleaner implementation is able to free the allocated buffers (each
implementation can do this except the no-op one). This ensures that
direct buffers are actually preferred when they're backed by something
cleanable and still unpreferred when unsafe is unavailable and buffers
cannot be freed in a controlled way.

Result:
Direct buffers will be preferred when they're cleanable in a controlled
way unrelated to the fact if unsafe is available.

…5281)

Motivation:

In our testsuite we often just wrapped a byte[] and passed it to the
write method. While this works it also might end up not testing often
with direct buffers. We should make this a bit more random and so cover
direct and heap buffers. The idea popped up when reviewing
#15231

Modifications:

- Add new static helper method that will either return a wrapped buffer
or a direct buffers in a random fashion
- Use this method to allocate buffers

Result:

More random testing of heap / direct buffers

#15395)

…5281)

Motivation:

In our testsuite we often just wrapped a byte[] and passed it to the
write method. While this works it also might end up not testing often
with direct buffers. We should make this a bit more random and so cover
direct and heap buffers. The idea popped up when reviewing
#15231

Modifications:

- Add new static helper method that will either return a wrapped buffer
or a direct buffers in a random fashion
- Use this method to allocate buffers

Result:

More random testing of heap / direct buffers

…ing to Netty change

### What changes were proposed in this pull request?

This PR removes `-Dio.netty.noUnsafe=true` from the default JVM args in `values.yaml`.

### Why are the changes needed?

Netty 4.2.2+ have a correct implementation instead of `Unsafe` usage for Java 25+ like the following instead of the legacy `Unsafe` operation.

- netty/netty#15231
- netty/netty#15338

### Does this PR introduce _any_ user-facing change?

No.

### How was this patch tested?

Pass the CIs with the existing tests.

### Was this patch authored or co-authored using generative AI tooling?

Generated-by: Claude Code (claude-4-opus)

Closes #593 from dongjoon-hyun/SPARK-56229.

Authored-by: Dongjoon Hyun <dongjoon@apache.org>
Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>