Use case

Don't delay inserts if they are not frequent.
Enable asynchronous inserts by default, but without the degradation of latency in normal scenarios.

Describe the solution you'd like

We will use (something like) exponentially smoothed insert rate to choose the wait time in some interval.

There will be settings to control the minimum and maximum wait times: async_insert_busy_timeout_min_ms for the minimum and the existing setting async_insert_busy_timeout_ms for the maximum. We can set the minimum to something like 50 ms and the maximum to something like 1000 ms. We will also have settings for increase rate (e.g. 2 by default) and decrease rate (e.g. 2 by default).

For every asynchronous insert queue, we will record the time of the previous insert and the previously selected wait time.
The selected wait time is adjusted dynamically between the minimum and maximum wait time.

If the time after the previous insert is more than the maximum wait time, the insert will be processed immediately (synchronously), and the selected wait time will be divided by the decrease rate and clamped to the interval between minimum and maximum wait time.

If the time after the previous insert is less than the maximum wait time, the selected wait time will be multiplied by the increase rate and clamped to the interval between minimum and maximum wait time, and the insert will be put in the queue with the selected wait time.

What to assume about this algorithm

If the inserts happen less frequently than the maximum wait time, they will be processed immediately with no delay.
If the same is true with some exceptions (e.g. most of the time, you have less than one insert per second, but occasionally two/tree in a short sequence), they will be delayed, but not much. And if you have many frequent inserts it will collect them in batches during the desired maximum wait time.

It does not use information about how long inserts take. But it looks like we don't need it.