performance issue analysis:
We identified the performance issue in many of the 43 ClickBench queries with ClickHouse on the 2x240 vCPUs system, particularly high page faults. For a deeper investigation, let's consider Q35. Q35 exhibited around 40% __handle_mm_fault hotspot in cycles on the GNR 2x240 vCPUs platform. We discovered that the high page faults stem from MADV_DONTNEED with the perf event.
In Query 35, there are 256 memory reallocations from 4KB to 16KB for each arena. Jemalloc recycles and coalesces many of these 256 16K memory blocks into a larger one when ClickHouse frees them. Subsequently, the large memory space in the dirty ecache cannot be reused for the next following 16K request, as the maximum allowed memory space before splitting is 16 * 64 KB. When a new memory request is made, Jemalloc locates an existing extent larger than the requested size in the dirty ecache and splits it to fit the actual requested size, maximizing memory reuse. There is a boundary for the requested extent size and the existing extent size, with a maximum ratio of 64 to minimize memory fragmentation. Consequently, Jemalloc would find possible memory space from the retained ecache (already MADV_DONTNEED and lacking physical pages) or through mmap, resulting in page faults and high RSS.

What does the patch do:
The idea is to reuse more dirty ecache extents in allocation/deallocation and also make sure that coalesce all the possible memory pieces in the purge/decay. There are two coalescing timings for large extents (>= 16KB) in Jemalloc: moving the extent during different states of ecaches (active -> dirty in our case) or decay/purging the existing extent in the dirty/muzzy ecache (dirty -> muzzy/retained) with background thread. Given the boundary "opt_lg_extent_max_active_fit" when requesting a new extent, we can apply the same boundary when coalescing existing extents if memory blocks return from ClickHouse to Jemalloc, enabling the reuse of most large extents in subsequent requests. This approach will not increase memory fragmentation, as there is no limit at the decay/purging stage. Jemalloc will merge all memory extents as much as possible and remove them from dirty to muzzy retained.

Result:
Test the patch with Q35 of Clickbench with ClickHouse on 2x240vCPUs system. The performance improvement after applying this patch can be seen in the table below.

We have also developed a microbench, which has 256 reallocations from 4KB to 16KB in each arena. Let's run 1000 iterations to get the average performance improvement. For 1 arena, the time cost is only 19.3% as before with the patch.

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <time.h>
#include <jemalloc/jemalloc.h>

#define ARENA_COUNT 1
#define ALLOC_COUNT 256
#define INITIAL_SIZE (4 * 1024)
#define RESIZE_SIZE (16 * 1024)
#define ITERATIONS 1000

int main() {
    void **ptrs;
    unsigned *arena_ids;
    clock_t start, end;
    double total_time = 0.0;

    const char *version;
    size_t sz = sizeof(version);
    mallctl("version", &version, &sz, NULL, 0);
    printf("Using jemalloc version: %s\n", version);

    ptrs = malloc(ARENA_COUNT * ALLOC_COUNT * sizeof(void*));
    arena_ids = malloc(ARENA_COUNT * sizeof(unsigned));
    if (!ptrs || !arena_ids) {
        fprintf(stderr, "Failed to allocate pointer arrays\n");
        return 1;
    }

    for (unsigned i = 0; i < ARENA_COUNT; i++) {
        unsigned arena_id;
        size_t sz = sizeof(unsigned);
        if (mallctl("arenas.create", &arena_id, &sz, NULL, 0) != 0) {
            fprintf(stderr, "Failed to create arena %u\n", i);
            return 1;
        }
        arena_ids[i] = arena_id;
    }

    for (int iter = 0; iter < ITERATIONS; iter++) {
        //printf("Starting iteration %d...\n", iter+1);
        start = clock();

        for (unsigned i = 0; i < ARENA_COUNT; i++) {
            for (unsigned j = 0; j < ALLOC_COUNT; j++) {
                unsigned idx = i * ALLOC_COUNT + j;
                int flags = MALLOCX_ARENA(arena_ids[i]);
                ptrs[idx] = mallocx(INITIAL_SIZE, flags);
                if (!ptrs[idx]) {
                    fprintf(stderr, "Memory allocation failed\n");
                    return 1;
                }
                memset(ptrs[idx], 1, INITIAL_SIZE);
            }
        }

        for (unsigned i = 0; i < ARENA_COUNT; i++) {
            for (unsigned j = 0; j < ALLOC_COUNT; j++) {
                unsigned idx = i * ALLOC_COUNT + j;
                int flags = MALLOCX_ARENA(arena_ids[i]);
                void *new_ptr = rallocx(ptrs[idx], RESIZE_SIZE, flags);
                if (!new_ptr) {
                    fprintf(stderr, "Memory resize failed\n");
                    return 1;
                }
                ptrs[idx] = new_ptr;
                memset((char*)ptrs[idx] + INITIAL_SIZE, 2, RESIZE_SIZE - INITIAL_SIZE);
            }
        }

        for (unsigned i = 0; i < ARENA_COUNT; i++) {
            for (unsigned j = 0; j < ALLOC_COUNT; j++) {
                unsigned idx = i * ALLOC_COUNT + j;
                dallocx(ptrs[idx], 0);
            }
        }

        end = clock();
        double time_taken = ((double)(end - start)) / CLOCKS_PER_SEC;
        total_time += time_taken;
	//printf("Iteration %d took %.6f seconds\n", iter+1, time_taken);
    }

    printf("Total time: %.6f seconds. Average time per loop: %.6f seconds\n", total_time, total_time / ITERATIONS);

    free(ptrs);
    free(arena_ids);

    return 0;
}

This pull request introduces a new max_size parameter to the extent coalescing logic in src/extent.c. The changes aim to improve memory management by limiting the maximum size of coalesced extents, particularly for large extents in the dirty ecache. The most important changes include adding the max_size parameter to relevant functions, updating the coalescing logic to respect this limit, and documenting the rationale behind the new behavior.

Changes to extent coalescing logic:

• Addition of max_size parameter:

  • The extent_try_coalesce_impl and related functions (extent_try_coalesce and extent_try_coalesce_large) were updated to include a max_size parameter, which restricts the maximum size of coalesced extents. [1] [2]

• Conditional checks for coalescing:

  • Added checks in both forward and backward coalescing logic to skip merging extents if their combined size exceeds max_size. This ensures that overly large extents are not created, improving memory reuse efficiency. [1] [2]

Documentation and rationale:

• Detailed comments on max_size behavior:
  • Added extensive comments explaining the purpose of the max_size parameter for large extents in the dirty ecache. The documentation highlights how this change improves dirty ecache reuse efficiency while maintaining flexibility during decay/purge operations.