Skip to content

Read buffer size in Data.Streaming.Network is hardcoded to 4096 #22

@kdkeyser

Description

@kdkeyser

I think it would make sense to make the buffer size configurable. I would be happy to look into this and provide a pull request if you think this would be helpful too.

Main use-case is for high-throughput servers: if you need to saturate a 40 Gb/s link, the recvfrom calls to the kernel become a bottleneck when using 4kiB reads. Since these servers often have plenty of memory (64-128 GB is not exception) there is room to increase memory consumption for these buffers if it would result in improved throughput.

FYI, the Go 1.5 runtime uses 32 kiB buffers. I started looking into the performance difference between a very basic data pump benchmark program in Go vs Haskell/Conduit and noticed this to be the cause of the Go test program outperforming the Haskell one when using message sizes larger than 4 kiB (in a simple request/response protocol).

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions