The memory budget is kv.transaction.max_refresh_spans_bytes. Once a transaction exceeds it, it can't refresh any more. Combined with a short closed timestamp duration, this means that such transactions are likely to have to refresh.
Instead, we should do what we do for write footprint tracking, and start collapsing adjacent spans when we're running out of budget. This can introduce false conflicts, but it'd gives a chance to succeed at refreshing (hopefully, a good one in common cases).

cc @ajwerner @nvanbenschoten