Now that we have implemented optimized native scorers for all data types, across our 2 supported architectures (x64 and ARM64), both single and "bulk", we need to fine tune them.
The native functions internally have different implementations/parameters that can be changed to optimize them: bulk size, prefetching, using of different SIMD instructions, specialized implementations for higher tier HW (SVE/AVX-512), etc. We also have different bulk algorithms and implementations, different unrolling levels and mechanisms. We should assess which ones are the most effectives, and adopt them across the codebase, to make code more readable, maintenable, and consistently more efficient.

Related tasks/issues:

Add missing benchmarks/tests:

• Add bare-bones int8 bulk operation benchmark #145173
• Add native int4 operation benchmarks; fix JDK22+ test guards #145096

Consolidate/fix implementations:

• Refactor x64 bulk scoring: shared template, masked AVX-512 tail, inlinable inner ops #145310
• Fix bulk scoring to process last batch instead of falling through to scalar tail #145316

Optimizations:

• Optimize ARM sqri8/sqri7u kernels using vabdq + vdotq #145116
• [Native] int4 x86 SIMD optimizations #144649
• [Native] Use vdotq_s32 for int7u/int8 distances on ARM #144505
• Use native vector scorers for on-heap data during indexing #143040
• AVX-512 int8 kernels with cascade unrolling #145683
• (disk)BBQ optimized kernels (especially bulk) with AVX-512