Skip to content

Add XXHash3 and XXHash64#316

Merged
dain merged 2 commits intoairlift:masterfrom
dain:xxhash3
Feb 3, 2026
Merged

Add XXHash3 and XXHash64#316
dain merged 2 commits intoairlift:masterfrom
dain:xxhash3

Conversation

@dain
Copy link
Member

@dain dain commented Jan 31, 2026

New APIs

XXHash3 (XxHash3Native):

  • 64-bit and 128-bit one-shot hashing
  • Streaming (incremental) hashing with XxHash3Hasher / XxHash3Hasher128
  • Seeded variants for all methods

XXHash64 (XxHash64Hasher):

  • One-shot and streaming hashing
  • Both native (FFM) and pure Java (VarHandle) implementations
  • Automatically selects native when available

Performance Summary

Benchmarked on Apple M1 (aarch64), throughput in ops/ms (higher is better):

Input Size xxhash3_64 xxhash3_128 xxhash64
8 bytes 366,000 76,000 284,000
64 bytes 265,000 56,000 145,000
1 KB 39,000 23,000 16,000
16 KB 2,700 2,600 1,000
64 KB 670 660 250

Key observations:

  • XXHash3 is ~2.5x faster than XXHash64 for large inputs (1KB+)
  • XXHash3 64-bit has minimal FFM overhead and is faster at all sizes
  • XXHash3 128-bit has FFM struct-return overhead at small inputs (<512 bytes), but converges with 64-bit at larger sizes (8KB+)
  • For 128-bit hashes on small inputs, the overhead is the cost of the simpler API—no known workarounds exist
  • XXHash64 native, Java, and legacy Unsafe implementations all have equivalent performance (not shown)

Implementation Notes

  • Internal Unsafe-based XXHash64 in zstd package retained for Zstd Java implementation compatibility

Adds XXH3 hash function with 64-bit and 128-bit variants via FFM bindings.
Includes streaming support for incremental hashing.
@dain dain requested review from electrum and martint January 31, 2026 03:48
Provides XxHash64Hasher interface with both VarHandle-based Java
implementation and FFM native bindings. The internal Unsafe-based
implementation still exists to serve the Zstd Java code.
@dain dain merged commit fecdef9 into airlift:master Feb 3, 2026
2 checks passed
@dain dain deleted the xxhash3 branch February 3, 2026 17:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants