@cxljs thank you for updating the Readme. I am still not convinced we need that big of a buffer to be allocated on startup. If you have the time, would you investigate how the reader will compare when reading with 8kb chunks? The reader will need some refactoring, but the writer should be fine, since we know how much data we should write.

Hello @ndyakov , I agree that the default size of 256KB is a bit large for most scenarios. The default size of the receive buffer for TCP sockets is 128KB, so in most scenarios, a value <= 128KB (or 64KB?) should be sufficient.

would you investigate how the reader will compare when reading with 8kb chunks?

I'll find some time to test it.

I write a simple test: set a 1KB-32KB value, and use 10000 goroutines to concurrent get it.

when value size = 8KB, the result:

Concurrent get: 10000 times
ReadBufferSize: 4 KB, WriteBufferSize: 256 KB, Time taken: 871.862295ms, avg: 87.186µs
ReadBufferSize: 8 KB, WriteBufferSize: 256 KB, Time taken: 439.96525ms, avg: 43.996µs
ReadBufferSize: 16 KB, WriteBufferSize: 256 KB, Time taken: 439.775098ms, avg: 43.977µs
ReadBufferSize: 32 KB, WriteBufferSize: 256 KB, Time taken: 420.656181ms, avg: 42.065µs
ReadBufferSize: 64 KB, WriteBufferSize: 256 KB, Time taken: 500.629106ms, avg: 50.062µs
ReadBufferSize: 128 KB, WriteBufferSize: 256 KB, Time taken: 398.798537ms, avg: 39.879µs
ReadBufferSize: 256 KB, WriteBufferSize: 256 KB, Time taken: 393.967161ms, avg: 39.396µs
ReadBufferSize: 512 KB, WriteBufferSize: 256 KB, Time taken: 361.235162ms, avg: 36.123µs
ReadBufferSize: 1024 KB, WriteBufferSize: 256 KB, Time taken: 434.438915ms, avg: 43.443µs

It seems that let the read buffer size >= the average size of the value is sufficient.

@cxljs from the example I can see the WriteBufferSize was always 256? Is this a mistake when printing or you left the write buffer the same?
p.s. looks like we can aim for 32kb as a default buffer size, what do you think?

you left the write buffer the same?

Yes, and the size of WriteBufferSize won't affect the test results.

looks like we can aim for 32kb as a default buffer size

I agree it.

@cxljs would you like to work on this in this branch or I can merge this one and you (or I) can work on using 32k as default and fix all docs / comments related to it ?

I'll update it directly in this PR.