-
Notifications
You must be signed in to change notification settings - Fork 8.3k
Add jemalloc profiling web UI for ClickHouse Keeper #100544
Copy link
Copy link
Open
Labels
comp-keeperClickHouse Keeper (Raft-based coordination layer; ZooKeeper replacement) and ZooKeeper client lib...ClickHouse Keeper (Raft-based coordination layer; ZooKeeper replacement) and ZooKeeper client lib...feature
Description
Problem
ClickHouse Server (26.2+) has a built-in jemalloc web UI at /jemalloc that provides one-click heap profiling with in-browser flame graph generation. Keeper has none of this — diagnosing memory issues requires SSH access, manual jmfp 4LW commands, installing jeprof, and running flamegraph.pl offline.
Related PRs
- #97077 — Added jemalloc stats web UI and
system.jemalloc_statsfor ClickHouse Server (26.2) - #85438 — Store jemalloc samples in
trace_log, introduced per-thread profiling model, deprecatedSYSTEM JEMALLOC ENABLE/DISABLE PROFILE - #97218 — Added
system.jemalloc_profile_textfor querying heap profiles from SQL - #98232 — Made jemalloc profiling cover all threads when enabled via config
Proposal
Add a /jemalloc web UI to Keeper's HTTP control endpoint (keeper_server.http_control.port) that provides:
- Live memory stats with auto-refresh charts (allocated, active, resident, mapped)
- Heap profile collection in collapsed/symbolized/raw format via a "Get Profile" button
- In-browser flame graph generation (reusing the same JS-based
flamegraph.plport from the server)
Since Keeper has no SQL engine, the implementation adds dedicated HTTP endpoints (/jemalloc/profile, /jemalloc/stats, /jemalloc/status) that call the jemalloc C API directly and return plain text/JSON. The HTML page fetches from these endpoints instead of sending SQL.
Out of scope
- Per-query profiling (Keeper has no queries or
system.trace_log) - Historical profiling via
trace_log/ automatic flush on OOM (separate effort) - Enabling/disabling profiling at runtime (deprecated globally; uses config
jemalloc_enable_global_profiler)
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
comp-keeperClickHouse Keeper (Raft-based coordination layer; ZooKeeper replacement) and ZooKeeper client lib...ClickHouse Keeper (Raft-based coordination layer; ZooKeeper replacement) and ZooKeeper client lib...feature