Use case

Thresholds for spilling data to disk should be flexible and based on current memory consumption to avoid exhausting memory with multiple queries, requiring a lot of memory, but capable of spilling data to disk.

Describe the solution you'd like
Thresholds for spilling to disk are computed at the start of every query. The limit should be based on current memory consumption and a configurable ratio. For example, if there is only 1 GB left at the moment of a query start. Then the limit for spilling to disk should be set to 500MB = 0.5 * 1GB, where the ratio 0.5 (default) is configurable.

Describe alternatives you've considered

It is clear that the solution is not ideal. For example, if many queries are started simultaneously they will still suffer from MEMORY_LIMIT_EXCEEDED errors. But in a "steady state" this solution is good, so it will cover 90% percent of cases. Here is the solution #41887 that should cover 99% of cases, but it is much more complicated and should be addressed after we implement this simple solution