Skip to content

[RW2] metadata.send feature parity with v1 #17191

@bwplotka

Description

@bwplotka

Proposal

Recently we discovered that RW1 flow has a side channel for metadata directly from scrape cache, which completely avoids `metadata-wal-records feature (and its issues).

PRW1 spec does not contain metadata support, but more users depends on experimental PRW1 support (e.g. Mimir users).

PRW2 has a native, official metadata support in spec. The PRW2 metadata is "connected/attached" to each series we send making it easier to consume. The PRW1 flow is disabled for PRW2 because of this fundamental difference - old PRW1 metadata were disconnected and per metric family. We can naively use existing metadataWatcher.

Instead Prometheus offers (1) metadata-wal-records feature and newer (2) type-and-unit-labels that allows PRW2 to provide metadata to consumers. (1) has known limitations and overhead, (2) does not support (or plan to) help string, require potentially extra relabelling to strip labels and is generally a new option (not released now).

This issue is to decide what (if anything) should be done to support feature parity around PRW1 metadata flow.
This is, practically speaking, only required for Mimir users using Prometheus, which is of considerable size.

Some ideas:

A: Focus on (2) type-and-unit-labels and provide examples or relabelling for stripping unit and type labels, document the behaviour
B: Implement metadata.send=true parity by direct lookup in scrape cache OR a second cache in queue_manager
C: Mix of (A) and (B) -- only do (B) for help.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions