Increasing read chunk size could improve performance for large responses #135
Description
Coming from discussion on Gitter with @dalf…
Currently we are reading response data in chunks of 4kB…
httpcore/httpcore/_async/http11.py
Line 26 in f4240b6
httpcore/httpcore/_async/http2.py
Line 29 in f4240b6
Benchmarking using @dalf's pyhttp-benchmark tool led us to see that increasing this number to 64kB could lead 2-3x execution time improvement for large responses (typically > 256kB).
My rationale would be that reading N bytes in one go via a syscall is faster than reading n = N/k bytes k times — mostly because the kernel is way faster than Python.