Skip to content

remote write 2.0 - implement additional compressions #13366

@cstyan

Description

@cstyan

With the introduction of remote write 2.0 and content negotiation to select or enforce a protocol version, that can be extended to allow for support of additional compression algorithms. In our benchmarks so far it doesn't look like replacing snappy (or klauspost/compress's snappy/s2 package) will lead to that much of an improvement. The reduction in duplicated data from the 2.0 format is already a huge win, and zstd/flate seem to be at least 2x slower in terms of compression CPU time.

However, with queue configurations such as sending less frequently but with larger batch sizes (or the opposite) it's possible that other compression algorithms could be beneficial. This is likely more relevant for the receiving end, at least in our case we see that decompression and reconstruction of structures from proto is not insignificant in terms of CPU.

So, TL; DR prometheus will mostly likely continue to use snappy by default and may provide support for other compression algorithms via content negotiation.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions