-
Notifications
You must be signed in to change notification settings - Fork 10.2k
Description
Proposal
Recently we discovered that RW1 flow has a side channel for metadata directly from scrape cache, which completely avoids `metadata-wal-records feature (and its issues).
PRW1 spec does not contain metadata support, but more users depends on experimental PRW1 support (e.g. Mimir users).
PRW2 has a native, official metadata support in spec. The PRW2 metadata is "connected/attached" to each series we send making it easier to consume. The PRW1 flow is disabled for PRW2 because of this fundamental difference - old PRW1 metadata were disconnected and per metric family. We can naively use existing metadataWatcher.
Instead Prometheus offers (1) metadata-wal-records feature and newer (2) type-and-unit-labels that allows PRW2 to provide metadata to consumers. (1) has known limitations and overhead, (2) does not support (or plan to) help string, require potentially extra relabelling to strip labels and is generally a new option (not released now).
This issue is to decide what (if anything) should be done to support feature parity around PRW1 metadata flow.
This is, practically speaking, only required for Mimir users using Prometheus, which is of considerable size.
Some ideas:
A: Focus on (2) type-and-unit-labels and provide examples or relabelling for stripping unit and type labels, document the behaviour
B: Implement metadata.send=true parity by direct lookup in scrape cache OR a second cache in queue_manager
C: Mix of (A) and (B) -- only do (B) for help.