Sorry I'm late to the PR, trying to understand the "deadlock" and wanted to double check if my understanding is correct. This is my interaction with Claude:

Your understanding is correct for a single xorb
If only the "N tasks waiting on the write lock for xorb X" problem existed in isolation, it's a throughput degradation, not a deadlock. Eventually the one task that holds the write lock finishes downloading, the 63 waiters see the cached data and return, CAS permits are released, and everything proceeds.

The buffer makes it worse, but still not a strict deadlock
In the old code, get_data_task acquires the CAS permit before returning the DataFuture and before that future is registered with the writer. So the main loop's flow is:

acquire buffer_permit ← blocks if buffer full
await get_data_task() ← acquires CAS permit here, then returns DataFuture
writer.enqueue(data_future, buffer_permit)
If the main loop is stuck waiting for a CAS permit, it's holding a buffer permit without having enqueued anything to the writer. The writer can't drain that slot (it was never registered). So buffer capacity is reduced by the number of terms stuck waiting for CAS permits — but since the main loop is sequential, that's at most 1 at a time.

What makes it an effective indefinite stall in practice
The real issue is the combination on slow connections:

Say buffer = 64 slots, CAS pool = 64 permits
Main loop fills all 64 buffer slots with terms for the same large xorb X (common in GGUF files where one xorb block is referenced by hundreds of file terms). All 64 CAS permits consumed.
1 task holds the write lock and downloads xorb X over a slow connection (hundreds of MB, minutes to download)
63 tasks hold CAS permits waiting for the write lock — doing nothing useful
Writer is waiting for all 64 futures to complete — it can't release buffer permits until they do
Main loop is blocked: buffer is full, no new work can be enqueued
The system isn't in a true circular deadlock — the write lock holder is making progress. But the effective parallelism has collapsed to 1 active download at any time, and that download may be large. On the slow connection used to reproduce the bug, this means minutes of no observable progress, which is indistinguishable from a deadlock to the user.

When the write lock holder eventually finishes, all 63 waiters immediately return (cached data), 64 buffer permits are released simultaneously, and the main loop floods the next 64 terms — many of which might again be for the same xorb, restarting the cycle.

Why the fix actually helps
The OnceCell single-flight means only 1 task ever acquires a CAS permit per xorb block. The other 63 tasks just wait on the OnceCell future without holding any permits. So:

CAS permits reflect actual concurrent downloads, not write-lock waiters
Buffer permits aren't wasted on tasks stuck waiting for CAS permits (since the permit is acquired inside the spawned task, not in the main loop)
True download parallelism is restored across different xorb blocks
So "deadlock" in the PR description is slightly loose — it's really a throughput collapse to near-zero caused by CAS permit starvation, which on a slow connection looks and feels like a deadlock.