Summary

Fixes #1623 (v0.4.0 regression: a second large model loads side-by-side instead of switching, over-committing memory and failing the next prefill).

The pre-load admission check in EnginePool.get_engine() gated on live memory only:

current = max(mx.get_active_memory(), get_phys_footprint())
projected = current + entry.estimated_size
if projected <= ceiling: break   # admit, no eviction

It never consulted the tracked accumulator _current_model_memory. After a model settles or idles, both mx.get_active_memory() and the process footprint can read well below the model's true resident size (Metal releases pooled memory; the SSD hot cache isn't counted), while _current_model_memory still reflects the committed total. So loading a second large model projected under the ceiling and was admitted without evicting the first — pushing memory to 57+ GB and failing the next request with a prefill memory-guard error.

This is a v0.4.0 regression: 0.3.7's _ensure_memory_available() (removed along with max_model_memory in the memory_guard_tier refactor) evicted using the accumulator.

Fix

Add self._current_model_memory to the max():

current = max(
    mx.get_active_memory(),
    get_phys_footprint(),
    self._current_model_memory,
)

The accumulator is a floor on committed model memory that live reads can dip below. This also makes pre-load admission consistent with the request-time prefill-eviction path (_evict_idle_lru_for_prefill), which already takes max(active, phys, _current_model_memory) — the pre-load gate was the lone path missing it.

Test plan

• New regression test test_eviction_when_live_memory_undercounts: with active = phys = 0 but the accumulator showing the pair over-commits, the second model must still evict the first. Fails on main (model-a not evicted), passes with the fix.
• pytest tests/test_engine_pool.py — 83 passed (incl. the existing test_eviction_before_load, unaffected: its fixture already proxies phys to the accumulator, so max(0, acc, acc) == acc).
• pytest tests/test_process_memory_enforcer.py tests/test_scheduler_admission.py tests/test_scheduler_prefill_memory_guard.py tests/test_memory_monitor.py tests/test_server_prefill_memory_handler.py — 177 passed.
• No new ruff findings.