…15133)

Reduce MGET / MSET latency by overlapping the dict-lookup memory accesses
across the keys of a single multi-key command. Builds on the cross-command
batched prefetch framework introduced in #14017 and the dict-prefetch state
machine in `memory_prefetch.c`, and lifts the kvobject-aware bits out of the
state machine into two new `dictType` callbacks so the same machinery can
be reused for other dict-encoded types later (hash hashtable, sets, sorted
sets) without paying for `kvobj`-specific code paths in the core loop.

Bundles the work originally proposed in #14899 (MGET prefetch framework,
by @mpozniak95) and #15043 (MSET batch prefetch).

## Design

Two new optional callbacks on `dictType`:

```c
typedef struct dictType {
    ...
    /* Bring the entry's key payload into cache before keyCompare runs.
     * Returns the address to prefetch, or NULL if the entry alone is enough. */
    void *(*prefetchEntryKey)(const dictEntry *de);

    /* Called only after a key match. Returns the value-side payload to
     * prefetch (or NULL). */
    void *(*prefetchEntryValue)(const dictEntry *de);
} dictType;
```

`dbDictType` registers both. The kv-aware logic — the `dictEntryIsKey()`
shortcut for embedded kvobjs, and `kv->ptr` for `OBJ_STRING` /
`OBJ_ENCODING_RAW` values — now lives in two small helpers in
`server.c`:

```c
static void *dbDictPrefetchEntryKey(const dictEntry *de) {
    return dictEntryIsKey(de) ? NULL : dictGetKey(de);
}

static void *dbDictPrefetchEntryValue(const dictEntry *de) {
    kvobj *kv = dictGetKey(de);
    return (kv->type == OBJ_STRING && kv->encoding == OBJ_ENCODING_RAW)
            ? kv->ptr : NULL;
}
```

The `PrefetchGetValueDataFunc` typedef and the per-call `get_val_data`
parameter on `dictPrefetchKeys()` / `dictPrefetch()` are removed — the
dict's own type drives both ends. This also removes the foot-gun where
callers (like `mgetCommand`) had to remember whether to pass
`prefetchGetObjectValuePtr` or `NULL`. `memory_prefetch.c` no longer
references `kvobj`, `kvobjGetKey`, or any specific value layout.

## State machine

Two file-local types in `memory_prefetch.c`:

| Type | Role |
|---|---|
| `dictPrefetchLookup` | Per-key snapshot of an in-flight,
software-pipelined `dictFind` (mirrors the locals a synchronous
`dictFind` would carry across one bucket walk). |
| `dictPrefetcher` | Driver that advances a batch of
`dictPrefetchLookup`s through the FSM, yielding to the next in-flight
lookup each time a prefetch is issued. |

Five-stage lifecycle for each lookup, driven by the prefetcher:

```text
                                                           │
                                                         start
                                                           │
                                                  ┌────────▼─────────┐
                                       ┌─────────►│  PREFETCH_BUCKET ├────►────────┐
                                       │          └────────┬─────────┘            no more tables
                                       │             bucket│found                  │
                                       │                   │                       │
        entry not found - goto next table         ┌────────▼────────┐              │
                                       └────◄─────┤ PREFETCH_ENTRY  │              ▼
                                    ┌────────────►└────────┬────────┘              │
                                    │                 entry│found                  │
                                    │                      │                       │
                                    │          ┌───────────▼─────────────┐         │
                                    │          │   PREFETCH_ENTRY_KEY    │ ◄── dictType->prefetchEntryKey(de)
                                    │          └───────────┬─────────────┘         │
                                    │                      │                       │
        key mismatch - goto next entry                     │                       │
                                    │          ┌───────────▼─────────────┐         │
                                    └──────◄───│   PREFETCH_ENTRY_VALUE  │ ◄── keyCompare; on match,
                                               └───────────┬─────────────┘     dictType->prefetchEntryValue(de)
                                                           │                       │
                                                 ┌─────────▼─────────────┐         │
                                                 │     PREFETCH_DONE     │◄────────┘
                                                 └───────────────────────┘
```

`PREFETCH_BUCKET` first picks `ht_table[0]`, then flips to `ht_table[1]`
if the dict is mid-rehash, then transitions to `PREFETCH_DONE` if no
more tables remain.

`memory_prefetch.c` exposes a small lifecycle that any caller can drive:

```c
dictPrefetcherInit(p, max_keys);                  /* one-shot heap alloc of lookups[] */
dictPrefetcherReset(p, dicts, keys, nkeys);       /* configure for one batch */
dictPrefetcherRun(p);                             /* drive FSM until all PREFETCH_DONE */
dictPrefetcherFree(p);                            /* release */
```

Each FSM stage is a named static function (`dictPrefetchBucket`,
`dictPrefetchEntry`, `dictPrefetchEntryKey`, `dictPrefetchEntryValue`),
so the `dictPrefetcherRun` driver is a four-line `switch` over the
state.

The state machine is dict-pure: no `kvobj` field on
`dictPrefetchLookup`,
no `kvobjGetKey` reach-through. Round-robin advance semantics — a state
only advances the cursor if a prefetch was actually issued — are
preserved, so the embedded-kvobj fast path
(`dictEntryIsKey(de) == 1` → callback returns NULL) still skips the
extra prefetch and falls straight into the compare on the next loop
iteration.

The cross-command path (`prefetchCommands` / `PrefetchCommandsBatch`)
embeds a `dictPrefetcher` initialized once at startup and reset per
batch, so cross-command prefetching no longer allocates per call.

## Intra-command API

```c
void dictPrefetchKeys(dict **dicts, void **keys, size_t nkeys);
```

A single multi-key command (e.g. MGET) can prefetch dict data for a
batch of its own keys, reusing the same state machine that the
cross-command path uses. Single-key calls (`nkeys <= 1`) early-return —
nothing to interleave with. The implementation stack-allocates a
fixed-size lookup array bounded by `DICT_PREFETCH_MAX_SIZE = 64` (no
VLA, predictable stack usage), so the intra-command path doesn't touch
the heap.

## Notes on the call sites

A shared helper picks the next prefetch batch and warms it via
`dictPrefetchKeys`:

```c
/* Pick the next prefetch batch starting at argv[start] and warm it via
 * dictPrefetchKeys. 'stride' is 1 for keys-only args (MGET) or 2 for
 * key/value pairs (MSET). Returns the chosen batch size in items. */
static int prefetchKeysBatch(client *c, int slot, int start, int stride);
```

Adaptive batch sizing inside the helper: if at least two full batches
(`PREFETCH_BATCH_SIZE * 2 = 32` items) remain, take one batch
(`PREFETCH_BATCH_SIZE = 16`); otherwise take all remaining items in one
call. This generalizes the small-request fast path so the trailing
batch of a large request also gets the single-call benefit.

- **MGET (`mgetCommand`)** — gated by
`do_prefetch = server.prefetch_batch_max_size && !already_prefetched && numkeys > 1`,
with `already_prefetched = c->current_pending_cmd &&
(c->current_pending_cmd->flags & PENDING_CMD_KEYS_PREFETCHED)`.
  When `do_prefetch` is set, each iteration calls
  `prefetchKeysBatch(c, slot, j, 1)` and then sequentially
  `lookupKeyRead`s + replies the chosen batch. When `do_prefetch` is
  clear (cross-command path already warmed the keys, or batch
  prefetching is off), the loop takes all remaining items in one go
  and skips the prefetch.

- **MSET / MSETNX (`msetGenericCommand`)** — same `do_prefetch` gate as
  MGET with `stride = 2`. For the NX flag the NX-check loop runs
  `lookupKeyWrite` (which already warmed everything via
  `prefetchKeysBatch`); the SET loop then disables further prefetch
  (`do_prefetch &&= !nx`) so we don't re-prefetch on the second pass.
  Going through the full state machine (rather than bucket-only) means
  `dbDictType`'s `prefetchEntryValue` callback runs on a key match —
  warming the old kvobj's payload, which `setKey -> dbReplaceValue ->
  updateKeysizesHist(oldlen, newlen)` then reads to compute the
  histogram delta. The slot dict is re-fetched per batch — in cluster
  mode the slot dict can be freed mid-MSET (`KVSTORE_FREE_EMPTY_DICTS`
  + `expireIfNeeded`), so a cached pointer would otherwise dangle.

- **Cross-command batch path (`addCommandToBatch`)** — sets
  `PENDING_CMD_KEYS_PREFETCHED` on every command added to the batch,
  even on partial-batch overflow (was: only when ALL keys fit). The
  intra-command path then uniformly skips supplemental prefetching for
  any command the batch touched. Rationale: running both paths
  (cross-command warm + intra-command supplement) caused a measured
  −9.6 % regression on x86 with pipeline-10, and the partial cross-
  command warmup is sufficient for the head of the keyset; the cold
  tail goes through normal lookup, which is still cheaper than running
  the FSM a second time on already-warm keys.

- **Future types**: each dict's `dictType` can register its own
  `prefetchEntryKey` / `prefetchEntryValue` (e.g. for the hashtable hash
  encoding, the field-sds and value-sds payloads), without touching
  `memory_prefetch.c`.

## Benchmark validation

On x86, performance improvements are significant for larger batch sizes:
  - 5Mkeys-string-mget-10B-100keys-pipeline-10: +89.44%
  - 5Mkeys-string-mget-100B-100keys: +37.33%
  - 5Mkeys-string-mget-100B-30keys: +22.40%
On ARM (Graviton4), the gains are even more pronounced:
  - 5Mkeys-string-mget-10B-100keys-pipeline-10: +128.34%
  - 5Mkeys-string-mget-100B-100keys-pipeline-10: +46.76%
Overall, the improvement scales with batch size, while a few small-batch cases show marginal gains or slight regressions.

---------

Co-authored-by: Marcin Poźniak <marcin.pozniak@intel.com>
Co-authored-by: Yuan Wang <yuan.wang@redis.com>