Proposal: Add type and unit metadata labels by dashpole · Pull Request #39 · prometheus/proposals

dashpole · 2024-10-24T15:20:13Z

Some screenshots from the Prometheus UI after this PoC: prometheus/prometheus#15683

The PoC updates all PromQL tests to demonstrate that adding type and unit labels doesn't break existing queries.

Querying for `unit`

Signed-off-by: David Ashpole <dashpole@google.com>

ArthurSens

Great start! I think we need to discuss a bit more whether to handle __* labels in the PromQL engine or the UI. Maybe there's a chance we're doing a breaking change...?

proposals/2024-09-25_metadata-labels.md

Signed-off-by: David Ashpole <dashpole@google.com>

beorn7

Thank you very much for proposing this. I love it in general, and as said elsewhere, I had a very similar idea a while ago for completely different reasons (mostly to avoid mixed series with native histograms and float-based sample types). Finding a similar solution for unrelated problems seems like a signal that this could be actually useful. However, I have also run into some conundrums, which was the reason why I put the whole idea on the backburner. I'll try to sketch out my train of thought here:

The __name__ label is the precedent here for "special labels", and as such it gives us a hint of what the issues might be. It's special power is not just about how to display it. It is treated specially for label matching and aggregation, and it is removed by most operations, following the argument that a rate of process_cpu_seconds_total is not process_cpu_seconds_total anymore. We have to ask ourselves the same questions for __unit__ and __type__.

In the brave new world, process_cpu_seconds_total would probably look like process_cpu{__type__="counter", __unit__="seconds"}. If we rate that, the outcome is not a counter anymore (but a gauge), and it is actually unit-less (seconds per second, i.e. it is the CPU usage ratio. In wishful thinking, the outcome should be process_cpu_usage{__type__="gauge", __unit__="ratio"}. Of course, we cannot easily come up with a general procedure to make up new names (which is the reason why current PromQL simply removes the name), and even changing the unit is tough. Changing the type might actually be feasible (we explored that a bit with native histograms, which "know" if they are a counter histogram or a gauge histogram). So maybe we can come up with "type translation rules" for all PromQL operations, and then maybe drop the unit in doubt (when calculating a rate or multiplying) and keep it when it makes sense (a sum aggregation or adding). But you will notice how deep we are in the woods here already.

The next thing is aggregation and label matching. As said, __name__ is also special here, but we probably need to be special in different ways for __unit__ and __type__. For example if we simply do a + b, we probably want to include unit and type in the label match (only add gauges to gauges, only add seconds to seconds etc.). However, a * b is already very different. b might be a scaling factor, i.e. a unit-less gauge. a could be counters or histograms. So in this case, we would like to exclude the unit and type from the label match.

It gets very confusing quickly. I wish we could just add the __unit__ and __type__ as an experiment without further special treatment and see how people cope with it. But right now my worry is it will hit too many road bumps…

proposals/2024-09-25_metadata-labels.md

beorn7 · 2024-11-12T10:37:40Z

It gets very confusing quickly. I wish we could just add the __unit__ and __type__ as an experiment without further special treatment and see how people cope with it. But right now my worry is it will hit too many road bumps…

Still in brainstorming mode, here is an idea how maybe a step zero could look like: Add the __unit__ and __type__ upon ingestion, but mostly ignore them in PromQL operations, maybe with the following exceptions:

__unit__ and __type__ can be used in label selectors. This allows the filtering (listed as a goal in this doc) and could also be used to increase readability of a PromQL expression.
__unit__ and __type__ can still be added via label_replace, e.g. to improve display of a result or to give the result of recording rules the unit and type information.

However, aggregations and label matches would ignore __unit__ and __type__ and would therefore work as before, and any operation would remove the __unit__ and __type__ label (with the exception of label_replace), both meant to circumvent any of the issues laid out above.

From there, we could cautiously move forward with more ideas. A step one might then be to handle __type__ in a more "native" fashion, i.e. attaching the "correct" __type__ label depending on the operation, still allowing the user to override via label_replace (like hard-casting a type in C), but maybe enforce a histogram type on native histograms (and prevent such a type on floats), so that we would avoid mixed series. Also, taking __type__ into account during aggregations might be possible.

Signed-off-by: David Ashpole <dashpole@google.com>

bwplotka · 2024-11-12T21:09:49Z

unit and type can still be added via label_replace, e.g. to improve display of a result or to give the result of recording rules the unit and type information.

Hm, I think it would be also nice to display them when collision actually occurs 🤔

Signed-off-by: David Ashpole <dashpole@google.com>

bwplotka

Nice work! Looking good, some comments.

proposals/2024-09-25_metadata-labels.md

Signed-off-by: David Ashpole <dashpole@google.com>

dashpole · 2024-11-26T03:47:47Z

@beorn7 I've added "Aggregations and label matches ignore __unit__ and __type__ and any operation would remove the __unit__ and __type__ label (with the exception of label_replace)." based on your suggestion above. I've also added "Handle type and unit in PromQL operations" as a potential future extension.

beorn7 · 2024-11-26T15:46:05Z

Quick note about federation: To my knowledge, federation still completely ignores metadata, so the exposition format it generates has all metric types as "untyped" and no units and help strings anyway, simply because Prometheus doesn't know about metadata post-ingestion. So being able to funnel unit and type through the Prometheus TSDB via labels would be a direct improvement for federation.

ArthurSens · 2024-11-26T22:54:11Z

Quick note about federation: To my knowledge, federation still completely ignores metadata, so the exposition format it generates has all metric types as "untyped" and no units and help strings anyway, simply because Prometheus doesn't know about metadata post-ingestion. So being able to funnel unit and type through the Prometheus TSDB via labels would be a direct improvement for federation.

I think it would be better to enable metadata post-scrape/federation than adding type and unit as labels in the exposition format. See #39 (comment)

beorn7 · 2024-11-27T11:58:02Z

Quick note about federation: To my knowledge, federation still completely ignores metadata, so the exposition format it generates has all metric types as "untyped" and no units and help strings anyway, simply because Prometheus doesn't know about metadata post-ingestion. So being able to funnel unit and type through the Prometheus TSDB via labels would be a direct improvement for federation.

I think it would be better to enable metadata post-scrape/federation than adding type and unit as labels in the exposition format. See #39 (comment)

The problem is that federation breaks the assumption that all metrics of the same name (one metric family) all have the same metadata.

bwplotka · 2024-11-27T12:57:13Z

The problem is that federation breaks the assumption that all metrics of the same name (one metric family) all have the same metadata.

Yea, I wonder if we could lift this with OM 2.0. WDYT @beorn7 ?

Related discussion: #39 (comment)

beorn7 · 2024-11-27T13:28:05Z

The problem is that federation breaks the assumption that all metrics of the same name (one metric family) all have the same metadata.

Yea, I wonder if we could lift this with OM 2.0. WDYT @beorn7 ?

The current structure of both protobuf and text versions of OM doesn't really lend itself to multiple different type, unit, or help entries for metrics with the same name. So we needed a new structural way of specifying unit and type (let's ignore help for now) per metric (rather than per metric family). Ironically, we already have a place for things that are per metric. We call it labels. So let's put type and metric into labels? Wait… that's what this proposal actually proposes. 😁

bwplotka · 2024-11-27T15:06:45Z

Well yes, but we just discussed that type and unit per label will not be easily parsable in OM/text, so putting it somewhere else actually helps. There might be a big questions there:

Is Metric Family concept actually useful these days? Should OM 2.0 simply redefine it, and flat that out? Personally it's only confusing, we don't use this semantic on the storage or PromQL (AFAIK). 💣
What's wrong to keep metric families but make them unique per type and unit? (not only name)?

beorn7 · 2024-11-27T16:23:39Z

Well yes, but we just discussed that type and unit per label will not be easily parsable in OM/text, so putting it somewhere else actually helps.

I wouldn't say that this is actually the case. The way we parse the labels doesn't really create a lot of parsing cost if there are more labels. It does create more payload, but not so much in relative terms. The OM text format is designed to have a lot of repetitive information anyway. If we want to avoid repetitive information, we needed a fundamentally different structure.

Is Metric Family concept actually useful these days? Should OM 2.0 simply redefine it, and flat that out? Personally it's only confusing, we don't use this semantic on the storage or PromQL (AFAIK). 💣

It correctly models the per-scrape data model. It avoids (a part of the) repetition. Of course you can flat that out, but then you get the repetition that was marked above as "not easily parseable". My claim is that "flattening that out" is pretty much equivalent to "adding that as labels".

What's wrong to keep metric families but make them unique per type and unit? (not only name)?

Well, the text format originally was designed to not be ordered in any way. So every line was keyed by the metric family name, i.e. a metric line starts with the metric family name, and all the metadata lines contain the metric family name as the 2nd token after the hash. OM became somewhat dependent on order, but not consequently so (you could avoid a lot of repetition if you did that) (another of the design problems I see with OM). With the original idea of the text format structure, you needed to repeat not just the metric family name on each line, but the combination of metric family name, type, and unit (which, you guessed it, is equivalent to putting type and unit into a label). Alternatively, we could redesign the text format fundamentally and make it really depend on the order everywhere, but then we should do it thoroughly and avoid other repetitions, too (at the very least the metric family name).

Protobuf is a bit different as it is structured, and all the family members are a repeated Metric message inside a MetricFamily message. So keying on more than just the name is relatively easy to accomplish. This is another reason why I think we should "simply" solve the protobuf parsing performance issue and then go back to "if you want efficiency, use protobuf – if you want simplicity, use text" (simplicity as in "easy to create in a valid form", which means kicking out many of the requirements OM text introduced on top of the original text format, like order, whitespace, …).

bwplotka · 2024-11-28T13:10:15Z

The OM text format is designed to have a lot of repetitive information anyway. If we want to avoid repetitive information, we needed a fundamentally different structure.

I thought it's more the fact we usually need to know what's the metric type (and metric family name) ahead of time (e.g. histograms have a different flow). But you are right, perhaps we could make label based approach work fine. Then it's only human readability question and what's defined in SDK and what's queried.

It correctly models the per-scrape data model.

I think I got it now, FAIK this only makes a difference for unstructured types like classic histogram and summary where metric name you define in SDKs (aka metric family name) is != resulting metric name. In the word with native histograms and perhaps native summaries one day, it would make no different, right?

My claim is that "flattening that out" is pretty much equivalent to "adding that as labels".

Agree.

Well, the text format originally was designed to not be ordered in any way. So every line was keyed by the metric family name, i.e. a metric line starts with the metric family name, and all the metadata lines contain the metric family name as the 2nd token after the hash. OM became somewhat dependent on order, but not consequently so (you could avoid a lot of repetition if you did that) (another of the design problems I see with OM). With the original idea of the text format structure, you needed to repeat not just the metric family name on each line, but the combination of metric family name, type, and unit (which, you guessed it, is equivalent to putting type and unit into a label). Alternatively, we could redesign the text format fundamentally and make it really depend on the order everywhere, but then we should do it thoroughly and avoid other repetitions, too (at the very least the metric family name).

Got it, thank you for detailed explanation! I think we should discuss those options in a clear proposal, make some decision on this in OM 2.0, will add issue for this.

Protobuf is a bit different as it is structured, and all the family members are a repeated Metric message inside a MetricFamily message. So keying on more than just the name is relatively easy to accomplish. This is another reason why I think we should "simply" solve the prometheus/prometheus#14668 and then go back to "if you want efficiency, use protobuf – if you want simplicity, use text" (simplicity as in "easy to create in a valid form", which means kicking out many of the requirements OM text introduced on top of the original text format, like order, whitespace, …).

Nice! Proposing as an intention in our OM 2.0 doc

beorn7

I'm generally fine with the content. My comments are just about improving wording or fix some technicalities that do not change the core message of this document.

proposals/2024-09-25_metadata-labels.md

Co-authored-by: Björn Rabenstein <github@rabenste.in> Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

Signed-off-by: bwplotka <bwplotka@gmail.com>

bwplotka · 2025-04-23T10:08:39Z

Addressed all, thanks for an amazing review! @beorn7 @ArthurSens @carrieedwards @fionaliao @krajorama

@fionaliao Overall LGTM, but I had the same question as Arthur around whether user-provided type and unit labels are always removed like in the case of RW 1.0

Yes, I updated one section to make it clear and clarified things in #39 (comment)

ArthurSens

LGTM

proposals/2024-09-25_metadata-labels.md

Co-authored-by: Arthur Silva Sens <arthursens2005@gmail.com> Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

proposals/2024-09-25_metadata-labels.md

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

Mentioned in DMs it's a LGTM once things were addressed, and they are now.

bwplotka · 2025-04-24T17:20:39Z

I think we should be good to go here! 👍🏽

Bartek Plotka 16:06
Just to schedule on your todo list (again): Are we good to merge #39 (review)? (type and unit proposal)

Björn Rabenstein 17:29 @beorn7
I'm not sure when I get to it again. But all my comments were not changing anything in the core, they were just about some peripheral understanding, so if you feel you have addressed my comments, you don't have to wait for me to verify.

Experimental implementation of prometheus/proposals#39 Previous (unmerged) experiments: * main...dashpole:prometheus:type_and_unit_labels * #16025 Signed-off-by: bwplotka <bwplotka@gmail.com>

Experimental implementation of prometheus/proposals#39 Previous (unmerged) experiments: * main...dashpole:prometheus:type_and_unit_labels * #16025 Signed-off-by: bwplotka <bwplotka@gmail.com> feature: type-and-unit-labels (extended MetricIdentity) Experimental implementation of prometheus/proposals#39 Previous (unmerged) experiments: * main...dashpole:prometheus:type_and_unit_labels * #16025 Signed-off-by: bwplotka <bwplotka@gmail.com>

* feature: type-and-unit-labels (extended MetricIdentity) Experimental implementation of prometheus/proposals#39 Previous (unmerged) experiments: * main...dashpole:prometheus:type_and_unit_labels * #16025 Signed-off-by: bwplotka <bwplotka@gmail.com> feature: type-and-unit-labels (extended MetricIdentity) Experimental implementation of prometheus/proposals#39 Previous (unmerged) experiments: * main...dashpole:prometheus:type_and_unit_labels * #16025 Signed-off-by: bwplotka <bwplotka@gmail.com> * Fix compilation errors Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com> Lint Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com> Revert change made to protobuf 'Accept' header Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com> Fix compilation errors for 'dedupelabels' tag Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com> * Rectored into schema.Metadata Signed-off-by: bwplotka <bwplotka@gmail.com> * texparse: Added tests for PromParse Signed-off-by: bwplotka <bwplotka@gmail.com> * add OM tests. Signed-off-by: bwplotka <bwplotka@gmail.com> * add proto tests Signed-off-by: bwplotka <bwplotka@gmail.com> * Addressed comments. Signed-off-by: bwplotka <bwplotka@gmail.com> * add schema label tests. Signed-off-by: bwplotka <bwplotka@gmail.com> * addressed comments. Signed-off-by: bwplotka <bwplotka@gmail.com> * fix tests. Signed-off-by: bwplotka <bwplotka@gmail.com> * add promql tests. Signed-off-by: bwplotka <bwplotka@gmail.com> * lint Signed-off-by: bwplotka <bwplotka@gmail.com> * Addressed comments. Signed-off-by: bwplotka <bwplotka@gmail.com> --------- Signed-off-by: bwplotka <bwplotka@gmail.com> Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com> Co-authored-by: Arthur Silva Sens <arthursens2005@gmail.com>

Experimental implementation of prometheus/proposals#39 Previous (unmerged) experiments: * prometheus/prometheus@main...dashpole:prometheus:type_and_unit_labels * prometheus#16025 Signed-off-by: bwplotka <bwplotka@gmail.com>

* MAINTAINERS: add Jan-Otto Kröpke as an discovery/azure maintainer Signed-off-by: Jan-Otto Kröpke <mail@jkroepke.de> * [REFACTOR] PromQL: DRY `stddev` and `stdvar` functions (#16451) * DRY stddev and stdvar --------- Signed-off-by: Neeraj Gartia <neerajgartia211002@gmail.com> * feat: Support 'NoTranslation' mode in OTLP endpoint (#16441) * feat: Support 'NoTranslation' mode in OTLP endpoint Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com> --------- Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com> * promql: histogram_fraction for bucket histograms (#16095) * promql: histogram_fraction for bucket histograms This PR extends the histogram_fraction function to also work with classic bucket histograms. This is beneficial because it allows expressions like sum(increase(my_bucket{le="0.5"}[10m]))/sum(increase(my_total[10m])) to be written without knowing the actual values of the "le" label, easing the transition to native histograms later on. It also feels natural since histogram_quantile also can deal with classic histograms. Signed-off-by: Michael Hoffmann <mhoffmann@cloudflare.com> * promql: histogram_fraction for bucket histograms * Add documentation and reduce code duplication * Fix a bug in linear interpolation between bucket boundaries * Add more PromQL tests Signed-off-by: Michael Hoffmann <mhoffmann@cloudflare.com> * Update docs/querying/functions.md Co-authored-by: Björn Rabenstein <github@rabenste.in> Signed-off-by: Michael Hoffmann <mhoffm@posteo.de> --------- Signed-off-by: Michael Hoffmann <mhoffmann@cloudflare.com> Signed-off-by: Michael Hoffmann <mhoffm@posteo.de> Co-authored-by: Björn Rabenstein <github@rabenste.in> * Add primitive support for ingesting OTLP delta metrics as-is (#16360) * Add simple delta support Signed-off-by: Fiona Liao <fiona.liao@grafana.com> * Rename delta2cumulative part Signed-off-by: Fiona Liao <fiona.liao@grafana.com> * Whoops bad refactor Signed-off-by: Fiona Liao <fiona.liao@grafana.com> * Add example yml Signed-off-by: Fiona Liao <fiona.liao@grafana.com> * Feature flag instead and histogram hint handling Signed-off-by: Fiona Liao <fiona.liao@grafana.com> * Delete otel_delta.yml - outdated Signed-off-by: Fiona Liao <fiona.liao@grafana.com> * Renaming to native delta support Signed-off-by: Fiona Liao <fiona.liao@grafana.com> * Add more explanatory comments Signed-off-by: Fiona Liao <fiona.liao@grafana.com> * Add more explanation to histograms Signed-off-by: Fiona Liao <fiona.liao@grafana.com> * Correct comment on d2c consumer Signed-off-by: Fiona Liao <fiona.liao@grafana.com> * Add tests for counters and fix bug Signed-off-by: Fiona Liao <fiona.liao@grafana.com> * Add histogram tests Signed-off-by: Fiona Liao <fiona.liao@grafana.com> * Add docs Signed-off-by: Fiona Liao <fiona.liao@grafana.com> * Sort series to make test deterministic Signed-off-by: Fiona Liao <fiona.liao@grafana.com> * More formatting Signed-off-by: Fiona Liao <fiona.liao@grafana.com> * Change flag name to ingestion Signed-off-by: Fiona Liao <fiona.liao@grafana.com> * Explain where rate calculation can go wrong Signed-off-by: Fiona Liao <fiona.liao@grafana.com> * Add warning about duplicate timestamps Signed-off-by: Fiona Liao <fiona.liao@grafana.com> * Update docs/feature_flags.md Co-authored-by: Arthur Silva Sens <arthursens2005@gmail.com> Signed-off-by: Fiona Liao <fiona.liao@grafana.com> * Fix tests Signed-off-by: Fiona Liao <fiona.liao@grafana.com> * Remove unnecessary if Signed-off-by: Fiona Liao <fiona.liao@grafana.com> * Add warning to d2c section Signed-off-by: Fiona Liao <fiona.liao@grafana.com> * Make unknown type error when getting temporality Signed-off-by: Fiona Liao <fiona.liao@grafana.com> * Correct type comment - not planning to add delta metric metadata type Signed-off-by: Fiona Liao <fiona.liao@grafana.com> * Remove unused param for empty type Signed-off-by: Fiona Liao <fiona.liao@grafana.com> * Rewrite temporality logic to be clearer Signed-off-by: Fiona Liao <fiona.liao@grafana.com> * Change spurious to unnecessary - better description Signed-off-by: Fiona Liao <fiona.liao@grafana.com> --------- Signed-off-by: Fiona Liao <fiona.liao@grafana.com> Co-authored-by: Arthur Silva Sens <arthursens2005@gmail.com> * Add documentation for 'NoTranslation' mode (#16470) * Add documentation for 'NoTranslation' mode Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com> * Update docs/configuration/configuration.md Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com> Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com> --------- Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com> Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com> * fix conversion from time range in seconds to time duration at parser Signed-off-by: Zhang Zhanpeng <zhangzhanpeng.zzp@alibaba-inc.com> * promql: histogram_stddev and histogram_stdvar should use arithmetic mean for custom buckets Signed-off-by: amanycodes <amanycodes@gmail.com> * stable and definitely unique card UI key in /alerts Signed-off-by: amanycodes <amanycodes@gmail.com> * fix-ci-errors Signed-off-by: amanycodes <amanycodes@gmail.com> * stable and unique key for /rules page and key fixes Signed-off-by: amanycodes <amanycodes@gmail.com> * removed comments Signed-off-by: amanycodes <amanycodes@gmail.com> * Fix PRW2 data diff error message typo (#16481) Signed-off-by: SungJin1212 <tjdwls1201@gmail.com> * fix typo (#16480) Signed-off-by: Yuchen Wang <yuchen.wang@databricks.com> * Strip newline in error when reading from remote client (#16487) Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com> * chore(deps): bump golang.org/x/net Bumps [golang.org/x/net](https://github.com/golang/net) from 0.37.0 to 0.38.0. - [Commits](https://github.com/golang/net/compare/v0.37.0...v0.38.0) --- updated-dependencies: - dependency-name: golang.org/x/net dependency-version: 0.38.0 dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> * chore(deps): bump github.com/aws/aws-sdk-go from 1.55.6 to 1.55.7 Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.55.6 to 1.55.7. - [Release notes](https://github.com/aws/aws-sdk-go/releases) - [Changelog](https://github.com/aws/aws-sdk-go/blob/main/CHANGELOG_PENDING.md) - [Commits](https://github.com/aws/aws-sdk-go/compare/v1.55.6...v1.55.7) --- updated-dependencies: - dependency-name: github.com/aws/aws-sdk-go dependency-version: 1.55.7 dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * chore(deps): bump github.com/docker/docker Bumps [github.com/docker/docker](https://github.com/docker/docker) from 28.0.4+incompatible to 28.1.1+incompatible. - [Release notes](https://github.com/docker/docker/releases) - [Commits](https://github.com/docker/docker/compare/v28.0.4...v28.1.1) --- updated-dependencies: - dependency-name: github.com/docker/docker dependency-version: 28.1.1+incompatible dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * chore(deps): bump github.com/Azure/azure-sdk-for-go/sdk/azidentity Bumps [github.com/Azure/azure-sdk-for-go/sdk/azidentity](https://github.com/Azure/azure-sdk-for-go) from 1.8.2 to 1.9.0. - [Release notes](https://github.com/Azure/azure-sdk-for-go/releases) - [Changelog](https://github.com/Azure/azure-sdk-for-go/blob/main/documentation/release.md) - [Commits](https://github.com/Azure/azure-sdk-for-go/compare/sdk/azidentity/v1.8.2...sdk/azcore/v1.9.0) --- updated-dependencies: - dependency-name: github.com/Azure/azure-sdk-for-go/sdk/azidentity dependency-version: 1.9.0 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * chore(deps): bump github.com/prometheus/client_model from 0.6.1 to 0.6.2 Bumps [github.com/prometheus/client_model](https://github.com/prometheus/client_model) from 0.6.1 to 0.6.2. - [Release notes](https://github.com/prometheus/client_model/releases) - [Commits](https://github.com/prometheus/client_model/compare/v0.6.1...v0.6.2) --- updated-dependencies: - dependency-name: github.com/prometheus/client_model dependency-version: 0.6.2 dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * chore(deps): bump google.golang.org/api from 0.229.0 to 0.230.0 Bumps [google.golang.org/api](https://github.com/googleapis/google-api-go-client) from 0.229.0 to 0.230.0. - [Release notes](https://github.com/googleapis/google-api-go-client/releases) - [Changelog](https://github.com/googleapis/google-api-go-client/blob/main/CHANGES.md) - [Commits](https://github.com/googleapis/google-api-go-client/compare/v0.229.0...v0.230.0) --- updated-dependencies: - dependency-name: google.golang.org/api dependency-version: 0.230.0 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * chore(deps): bump github.com/gophercloud/gophercloud/v2 Bumps [github.com/gophercloud/gophercloud/v2](https://github.com/gophercloud/gophercloud) from 2.6.0 to 2.7.0. - [Release notes](https://github.com/gophercloud/gophercloud/releases) - [Changelog](https://github.com/gophercloud/gophercloud/blob/v2.7.0/CHANGELOG.md) - [Commits](https://github.com/gophercloud/gophercloud/compare/v2.6.0...v2.7.0) --- updated-dependencies: - dependency-name: github.com/gophercloud/gophercloud/v2 dependency-version: 2.7.0 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * chore(deps): bump google.golang.org/grpc from 1.71.1 to 1.72.0 Bumps [google.golang.org/grpc](https://github.com/grpc/grpc-go) from 1.71.1 to 1.72.0. - [Release notes](https://github.com/grpc/grpc-go/releases) - [Commits](https://github.com/grpc/grpc-go/compare/v1.71.1...v1.72.0) --- updated-dependencies: - dependency-name: google.golang.org/grpc dependency-version: 1.72.0 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * chore(deps): bump github.com/linode/linodego from 1.48.1 to 1.49.0 Bumps [github.com/linode/linodego](https://github.com/linode/linodego) from 1.48.1 to 1.49.0. - [Release notes](https://github.com/linode/linodego/releases) - [Commits](https://github.com/linode/linodego/compare/v1.48.1...v1.49.0) --- updated-dependencies: - dependency-name: github.com/linode/linodego dependency-version: 1.49.0 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * chore(deps): bump github.com/miekg/dns from 1.1.64 to 1.1.65 Bumps [github.com/miekg/dns](https://github.com/miekg/dns) from 1.1.64 to 1.1.65. - [Changelog](https://github.com/miekg/dns/blob/master/Makefile.release) - [Commits](https://github.com/miekg/dns/compare/v1.1.64...v1.1.65) --- updated-dependencies: - dependency-name: github.com/miekg/dns dependency-version: 1.1.65 dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * chore(deps): bump github.com/prometheus/client_golang Bumps [github.com/prometheus/client_golang](https://github.com/prometheus/client_golang) from 1.21.1 to 1.22.0. - [Release notes](https://github.com/prometheus/client_golang/releases) - [Changelog](https://github.com/prometheus/client_golang/blob/main/CHANGELOG.md) - [Commits](https://github.com/prometheus/client_golang/compare/v1.21.1...v1.22.0) --- updated-dependencies: - dependency-name: github.com/prometheus/client_golang dependency-version: 1.22.0 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * chore(deps): bump github.com/digitalocean/godo from 1.142.0 to 1.144.0 Bumps [github.com/digitalocean/godo](https://github.com/digitalocean/godo) from 1.142.0 to 1.144.0. - [Release notes](https://github.com/digitalocean/godo/releases) - [Changelog](https://github.com/digitalocean/godo/blob/main/CHANGELOG.md) - [Commits](https://github.com/digitalocean/godo/compare/v1.142.0...v1.144.0) --- updated-dependencies: - dependency-name: github.com/digitalocean/godo dependency-version: 1.144.0 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * chore(deps): bump github.com/prometheus/client_golang Bumps [github.com/prometheus/client_golang](https://github.com/prometheus/client_golang) from 1.21.1 to 1.22.0. - [Release notes](https://github.com/prometheus/client_golang/releases) - [Changelog](https://github.com/prometheus/client_golang/blob/main/CHANGELOG.md) - [Commits](https://github.com/prometheus/client_golang/compare/v1.21.1...v1.22.0) --- updated-dependencies: - dependency-name: github.com/prometheus/client_golang dependency-version: 1.22.0 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * [BUILD] Remove Go toolchain line in internal/tools (#16516) * Upgraded npm dependencies pre 3.4.0-rc.0 (#16493) * Volunteer Bryan Boreham as Shepherd for Release 3.5 Signed-off-by: Bryan Boreham <bjboreham@gmail.com> * Add `label_selector` to hetzner service discovery Allows to filter the servers when sending the listing request to the API. This feature is only available when using the `role=hcloud`. See https://docs.hetzner.cloud/#label-selector for details on how to use the label selector. Signed-off-by: Jonas Lammler <jonas.lammler@hetzner-cloud.de> * Clarify the docs of promtool --{min,max}-time options State explicitely what kind of timestamps are expected for the --min-time and --max-time options of promtool tsdb commands. This is especially important for the dump-openmetrics command as users could otherwise mistakenly think it would be in seconds, like the OpenMetrics timestamps themselves. Signed-off-by: Nicolas Peugnet <nicolas.peugnet@lip6.fr> * Labels: simpler/faster stringlabels encoding (#16069) Instead of using varint to encode the size of each label, use a single byte for size 0-254, or a flag value of 255 followed by the size in 3 bytes little-endian. This reduces the amount of code, and also the number of branches in commonly-executed code, so it runs faster. The maximum allowed label name or value length is now 2^24 or 16MB. Memory used by labels changes as follows: * Labels from 0 to 127 bytes length: same * From 128 to 254: 1 byte less * From 255 to 16383: 2 bytes more * From 16384 to 2MB: 1 byte more * From 2MB to 16MB: same Labels: panic on string too long. Slightly more user-friendly than encoding bad data and finding out when we decode. Clarify that Labels.Bytes() encoding can change --------- Signed-off-by: Bryan Boreham <bjboreham@gmail.com> * [BUGFIX] Top-level: Update GOGC before loading TSDB (#16491) We should use the configured value, or Prometheus' default of 75%, while initializing and loading the WAL. Since the Go default is 100%, most Prometheus users would experience higher memory usage before the value is configured. Also: move Go runtime params earlier in initialization. E.g. if a module starting up looks at GOMAXPROCS to size something, we need to have set it already. --------- Signed-off-by: Bryan Boreham <bjboreham@gmail.com> * promtool: Optional fuzzy float64 comparison in rules unittests (#16395) Make fuzzy compare opt-in via fuzzy_compare boolean in each unittest file. Signed-off-by: Graham Reed <greed@hypervolt.co.uk> * chore(deps): bump github/codeql-action from 3.28.13 to 3.28.16 Bumps [github/codeql-action](https://github.com/github/codeql-action) from 3.28.13 to 3.28.16. - [Release notes](https://github.com/github/codeql-action/releases) - [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md) - [Commits](https://github.com/github/codeql-action/compare/1b549b9259bda1cb5ddde3b41741a82a2d15a841...28deaeda66b76a05916b6923827895f2b14ab387) --- updated-dependencies: - dependency-name: github/codeql-action dependency-version: 3.28.16 dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * chore(deps): bump github.com/ionos-cloud/sdk-go/v6 from 6.3.3 to 6.3.4 Bumps [github.com/ionos-cloud/sdk-go/v6](https://github.com/ionos-cloud/sdk-go) from 6.3.3 to 6.3.4. - [Release notes](https://github.com/ionos-cloud/sdk-go/releases) - [Changelog](https://github.com/ionos-cloud/sdk-go/blob/master/docs/CHANGELOG.md) - [Commits](https://github.com/ionos-cloud/sdk-go/compare/v6.3.3...v6.3.4) --- updated-dependencies: - dependency-name: github.com/ionos-cloud/sdk-go/v6 dependency-version: 6.3.4 dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * chore(deps): bump the go-opentelemetry-io group across 2 directories with 5 updates Bumps the go-opentelemetry-io group with 4 updates in the / directory: [go.opentelemetry.io/collector/component](https://github.com/open-telemetry/opentelemetry-collector), [go.opentelemetry.io/collector/consumer](https://github.com/open-telemetry/opentelemetry-collector), [go.opentelemetry.io/collector/processor](https://github.com/open-telemetry/opentelemetry-collector) and [go.opentelemetry.io/collector/semconv](https://github.com/open-telemetry/opentelemetry-collector). Bumps the go-opentelemetry-io group with 2 updates in the /documentation/examples/remote_storage directory: [go.opentelemetry.io/collector/pdata](https://github.com/open-telemetry/opentelemetry-collector) and [go.opentelemetry.io/collector/semconv](https://github.com/open-telemetry/opentelemetry-collector). Updates `go.opentelemetry.io/collector/component` from 1.30.0 to 1.31.0 - [Release notes](https://github.com/open-telemetry/opentelemetry-collector/releases) - [Changelog](https://github.com/open-telemetry/opentelemetry-collector/blob/main/CHANGELOG-API.md) - [Commits](https://github.com/open-telemetry/opentelemetry-collector/compare/pdata/v1.30.0...pdata/v1.31.0) Updates `go.opentelemetry.io/collector/consumer` from 1.30.0 to 1.31.0 - [Release notes](https://github.com/open-telemetry/opentelemetry-collector/releases) - [Changelog](https://github.com/open-telemetry/opentelemetry-collector/blob/main/CHANGELOG-API.md) - [Commits](https://github.com/open-telemetry/opentelemetry-collector/compare/pdata/v1.30.0...pdata/v1.31.0) Updates `go.opentelemetry.io/collector/pdata` from 1.30.0 to 1.31.0 - [Release notes](https://github.com/open-telemetry/opentelemetry-collector/releases) - [Changelog](https://github.com/open-telemetry/opentelemetry-collector/blob/main/CHANGELOG-API.md) - [Commits](https://github.com/open-telemetry/opentelemetry-collector/compare/pdata/v1.30.0...pdata/v1.31.0) Updates `go.opentelemetry.io/collector/processor` from 1.30.0 to 1.31.0 - [Release notes](https://github.com/open-telemetry/opentelemetry-collector/releases) - [Changelog](https://github.com/open-telemetry/opentelemetry-collector/blob/main/CHANGELOG-API.md) - [Commits](https://github.com/open-telemetry/opentelemetry-collector/compare/pdata/v1.30.0...pdata/v1.31.0) Updates `go.opentelemetry.io/collector/semconv` from 0.124.0 to 0.125.0 - [Release notes](https://github.com/open-telemetry/opentelemetry-collector/releases) - [Changelog](https://github.com/open-telemetry/opentelemetry-collector/blob/main/CHANGELOG-API.md) - [Commits](https://github.com/open-telemetry/opentelemetry-collector/compare/v0.124.0...v0.125.0) Updates `go.opentelemetry.io/collector/pdata` from 1.30.0 to 1.31.0 - [Release notes](https://github.com/open-telemetry/opentelemetry-collector/releases) - [Changelog](https://github.com/open-telemetry/opentelemetry-collector/blob/main/CHANGELOG-API.md) - [Commits](https://github.com/open-telemetry/opentelemetry-collector/compare/pdata/v1.30.0...pdata/v1.31.0) Updates `go.opentelemetry.io/collector/semconv` from 0.124.0 to 0.125.0 - [Release notes](https://github.com/open-telemetry/opentelemetry-collector/releases) - [Changelog](https://github.com/open-telemetry/opentelemetry-collector/blob/main/CHANGELOG-API.md) - [Commits](https://github.com/open-telemetry/opentelemetry-collector/compare/v0.124.0...v0.125.0) --- updated-dependencies: - dependency-name: go.opentelemetry.io/collector/component dependency-version: 1.31.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: go-opentelemetry-io - dependency-name: go.opentelemetry.io/collector/consumer dependency-version: 1.31.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: go-opentelemetry-io - dependency-name: go.opentelemetry.io/collector/pdata dependency-version: 1.31.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: go-opentelemetry-io - dependency-name: go.opentelemetry.io/collector/processor dependency-version: 1.31.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: go-opentelemetry-io - dependency-name: go.opentelemetry.io/collector/semconv dependency-version: 0.125.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: go-opentelemetry-io - dependency-name: go.opentelemetry.io/collector/pdata dependency-version: 1.31.0 dependency-type: indirect update-type: version-update:semver-minor dependency-group: go-opentelemetry-io - dependency-name: go.opentelemetry.io/collector/semconv dependency-version: 0.125.0 dependency-type: indirect update-type: version-update:semver-minor dependency-group: go-opentelemetry-io ... Signed-off-by: dependabot[bot] <support@github.com> * chore(deps): bump github.com/digitalocean/godo from 1.144.0 to 1.145.0 Bumps [github.com/digitalocean/godo](https://github.com/digitalocean/godo) from 1.144.0 to 1.145.0. - [Release notes](https://github.com/digitalocean/godo/releases) - [Changelog](https://github.com/digitalocean/godo/blob/main/CHANGELOG.md) - [Commits](https://github.com/digitalocean/godo/compare/v1.144.0...v1.145.0) --- updated-dependencies: - dependency-name: github.com/digitalocean/godo dependency-version: 1.145.0 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * chore(deps): bump google.golang.org/api from 0.230.0 to 0.231.0 Bumps [google.golang.org/api](https://github.com/googleapis/google-api-go-client) from 0.230.0 to 0.231.0. - [Release notes](https://github.com/googleapis/google-api-go-client/releases) - [Changelog](https://github.com/googleapis/google-api-go-client/blob/main/CHANGES.md) - [Commits](https://github.com/googleapis/google-api-go-client/compare/v0.230.0...v0.231.0) --- updated-dependencies: - dependency-name: google.golang.org/api dependency-version: 0.231.0 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * chore(cmd): Add info log for automemlimit config Signed-off-by: machine424 <ayoubmrini424@gmail.com> * Prepare release 3.4.0-rc.0 (#16517) Co-authored-by: Ayoub Mrini <ayoubmrini424@gmail.com> * Fix linting issues found by golangci-lint v2.0.2 (#16368) * Fix linting issues found by golangci-lint v2.0.2 --------- Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * chore(deps): bump github.com/open-telemetry/opentelemetry-collector-contrib/processor/deltatocumulativeprocessor Bumps [github.com/open-telemetry/opentelemetry-collector-contrib/processor/deltatocumulativeprocessor](https://github.com/open-telemetry/opentelemetry-collector-contrib) from 0.124.1 to 0.125.0. - [Release notes](https://github.com/open-telemetry/opentelemetry-collector-contrib/releases) - [Changelog](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/CHANGELOG-API.md) - [Commits](https://github.com/open-telemetry/opentelemetry-collector-contrib/compare/v0.124.1...v0.125.0) --- updated-dependencies: - dependency-name: github.com/open-telemetry/opentelemetry-collector-contrib/processor/deltatocumulativeprocessor dependency-version: 0.125.0 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * Bump golangci-lint to v2.1.5 (#16545) Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com> * chore(deps): bump actions/setup-node from 4.3.0 to 4.4.0 (#16533) Bumps [actions/setup-node](https://github.com/actions/setup-node) from 4.3.0 to 4.4.0. - [Release notes](https://github.com/actions/setup-node/releases) - [Commits](https://github.com/actions/setup-node/compare/cdca7365b2dadb8aad0a33bc7601856ffabcc48e...49933ea5288caeca8642d1e84afbd3f7d6820020) --- updated-dependencies: - dependency-name: actions/setup-node dependency-version: 4.4.0 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * promql: don't emit a value from `histogram_fraction` or `histogram_quantile` if classic and native histograms are present at the same timestamp Signed-off-by: Charles Korn <charles.korn@grafana.com> * Add global config option for always_scrape_classic_histograms Addresses https://github.com/prometheus/prometheus/issues/16371 This will help with migrating to native histograms with `convert_classic_histograms_to_nhcb` since users may still need to keep the classic histograms during a migration Signed-off-by: chardch <otwordsne@gmail.com> * fix promql-test error message and readme Signed-off-by: Neeraj Gartia <neerajgartia211002@gmail.com> * chore(scripts/sync_repo_files): add link to source script Signed-off-by: machine424 <ayoubmrini424@gmail.com> * tsdb/errors.MultiError: support errors.As (#16544) * tsdb/errors.MultiError: implement Unwrap the multierror was hiding some errors in Mimir. I also added unit tests because I had them handy from a similar change I and yuri did in XXX and some time ago --------- Signed-off-by: Dimitar Dimitrov <dimitar.dimitrov@grafana.com> Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com> * test(cmd): add test for GOGC setting As suggested in https://github.com/prometheus/prometheus/pull/14176#issuecomment-2150308054 Signed-off-by: machine424 <ayoubmrini424@gmail.com> * [PERF] TSDB: Pass down label value limit into implementation (#16158) * allow limiting label values calls Signed-off-by: Andre Branchizio <andrejbranch@gmail.com> * update fix configuration.md Signed-off-by: dongjiang <dongjiang1989@126.com> * Add a test for aggregation wrapped in ParenExpr Signed-off-by: Lukasz Mierzwa <l.mierzwa@gmail.com> * fix(parser): parenthesis around aggregate expression parsing The position range of nested aggregate expression was wrong, for the expression "(sum(foo))" the position of "sum(foo)" should be 1-9, but the parser could not decide the end of the expression on pos 9, instead it read ahead to pos 10 and then emitted the aggregate. But we only kept the last closing position (10) and wrote that into the aggregate. The reason for this is that the parser cannot know from "(sum(foo)" alone if the aggregate is finished. It could be finished as in "(sum(foo))" but equally it could continue with group modifier as "(sum(foo) by (bar))". The fix is to track ")" tokens in a stack in addition to the lastClosing position, which is overloaded with other things like offset number tracking. Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com> * Fix position in TestPreprocessAndWrapWithStepInvariantExpr This is failing after 26088d01b9518b3315805186dcee534b986bf79f and it seems that currently tested position is incorrect, fix it Signed-off-by: Lukasz Mierzwa <l.mierzwa@gmail.com> * docs: Fix `metric_name_escaping_scheme` config parameter This is a fix for documentation update done in https://github.com/prometheus/prometheus/pull/16066, setting correct name for a configuration value. Signed-off-by: Martin Danko <46035688+zepellin@users.noreply.github.com> * chore(engine): add simple NHCB benchmark Copy the benchmark for native histograms with exponential buckets and adopt to native histograms with custom buckets. Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com> * perf(chunkenc): intern the custom values for native histograms The custom values are the "le" bucket boundaries of native histograms with custom buckets. They are never modified. It is ok to not copy them when iterating a chunk, just reference them. If we will ever have a function that modifies the custom values, like 'trim' for example. That function will have to make a copy on write. Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com> * UI: optimize pool searching in /target page Signed-off-by: Ryan Wu <rongjun0821@gmail.com> * refactor: use slices.Contains to simplify code Signed-off-by: hardlydearly <799511800@qq.com> * promql: Refactor some functions to make them more DRY (#16532) Signed-off-by: Neeraj Gartia <neerajgartia211002@gmail.com> * promql: support variable scalar parameter in aggregations in range queries (#16404) This fixes the regression introduced in https://github.com/prometheus/prometheus/issues/15971 while preserving the performance improvements. Signed-off-by: Neeraj Gartia <neerajgartia211002@gmail.com> * Try fixing potential deadlocks in discovery Manager.ApplyConfig() uses multiple locks: - Provider.mu - Manager.targetsMtx Manager.cleaner() uses the same locks but in the opposite order: - First it locks Manager.targetsMtx - The it locks Provider.mu I've seen a few strange cases of Prometheus hanging up on shutdown and never compliting that shutdown. From a few traces I was given it appears that while Prometheus is still running only discovery.Manager and notifier.Manager are running running. From that trace it also seems like they are stuck on a lock from two functions: - cleaner waits on a RLock() - ApplyConfig waits on a Lock() I cannot reproduce it but I suspect this is a race between locks. Imagine this scenario: - Manager.ApplyConfig() is called - Manager.ApplyConfig locks Provider.mu.Lock() - at the same time cleaner() is called on the same Provider instance and it calls Manager.targetsMtx.Lock() - Manager.ApplyConfig() now calls Manager.targetsMtx.Lock() but that lock is already held by cleaner() function so ApplyConfig() hangs there - at the same time cleaner() now wants to lock Provider.mu.Rlock() but that lock is already held by Manager.ApplyConfig() - we end up with both functions locking each other out without any way to break that lock Re-order lock calls to try to avoid this scenario. I tried writing a test case for it but couldn't hit this issue. Signed-off-by: Lukasz Mierzwa <l.mierzwa@gmail.com> * Optimize memoization and search debouncing on /targets page (#16589) Moving the debouncing of the search field to the parent component and then memoizing the ScrapePoolsList component prevents a lot of superfluous re-renders of the entire scrape pools list that previously got triggered immediately when you typed in the search box or even just collapsed a pool. (While the computation of what data to show was already memoized in the ScrapePoolList component, the component itself still had to re-render a lot with the same data.) Discovered this problem + verified fix using react-scan. Signed-off-by: Julius Volz <julius.volz@gmail.com> * Fix full-page re-rendering when opening status nav menu (#16590) When opening the status pages menu while already viewing one of the status pages, the whole page would be re-rendered because the menu target's default action of following the current page's URL was not prevented. Also, we don't need to use a NavLink component for the menu target when we are not viewing a status page, because then the component won't need to be highlighted anyways. Discovered + fixed with the help of react-scan. Signed-off-by: Julius Volz <julius.volz@gmail.com> * fix(config): respect GOGC environment variable if no "runtime" block exists Fixes: https://github.com/prometheus/prometheus/issues/16334 Related to: - https://github.com/prometheus/prometheus/pull/15238 - https://github.com/prometheus/prometheus/pull/16052 Currently, when the GOGC environment variable is set -- and no `runtime` block is set in the Prometheus config file -- it is ignored and the default value of 75% is always used. However, if there is an empty runtime block (e.g. `runtime: {}`), _then_ the GOGC environment variable is checked. This PR changes this behavior to consistently check and use the GOGC environment variable when it is set (unless the `gogc` field is set in the `runtime` block of the loaded config file, in which case it still gives that precedence). Co-authored-by: Adam Rambo <arambo@protonmail.com> Signed-off-by: Will Hegedus <whegedus@akamai.com> * test(cmd): enable test cases for GOGC environment variable Signed-off-by: Will Hegedus <whegedus@akamai.com> * Clean up codeboxes and headings in docs The new docs site will have syntax highlighting, so this adds language tags to code boxes that are currently missing them. I didn't add `promql` as a language yet since the highlighter doesn't support it yet, plus a lot of the PromQL codeboxes in our docs aren't strictly valid PromQL, they are more like multiple expressions listed in the same code box on multiple lines. So I'm leaving that for sometime later. In the HTTP API page, I moved the curl examples from the JSON codeboxes to their own ones above the JSON output. I considered putting an "Output:" text between the curl + JSON output, but I think the way it currently looks without it is probably fine. I also fixed a number of headings which were at the wrong level relative to their nesting in the document. I also removed `go` as a language from the Go template language examples, because the Go template language isn't Go at all. I also adjusted the indent on one codebox to be more reasonable (2 spaces instead of 8). And then finally, my editor made a bunch of whitespace changes automatically, like removing trailing spaces. Signed-off-by: Julius Volz <julius.volz@gmail.com> Signed-off-by: Julius Volz <julius.volz@gmail.com> * Move m.targetsMtx.Lock down into the loop Make sure the order of locks is always the same in all functions. In ApplyConfig() we have m.targetsMtx.Lock() after provider is locked, so replicate the same in allGroups(). Signed-off-by: Lukasz Mierzwa <l.mierzwa@gmail.com> * Fixup make proto (#16603) Use `common-` prefix for `make proto` so downstream projects like client_golang can implement their own `make proto`. Signed-off-by: SuperQ <superq@gmail.com> * Add golangci-lint fmt (#16602) With golangci-lint v2, it now has "formatters" that can be configured. Add `golangci-lint fmt` to the `make format` in Makefile.common. * Enable goimports formatter. Signed-off-by: SuperQ <superq@gmail.com> * SD UI: Better total target count display when using `keep_dropped_targets` option Fixes https://github.com/prometheus/prometheus/issues/16586 Signed-off-by: Julius Volz <julius.volz@gmail.com> * Add health & text filtering on the /rules page Addresses part of https://github.com/prometheus/prometheus/issues/16515 For now, I'm adding very similar filtering to the /rules page as we have on the /alerts page, with the difference being: * The state filter filters by rule health (ok/warn/unknown) instead of alert state (firing/pending/inactive) * We don't collect & show detailed stats on the different state counts as we do on the /alerts page There is a lot of copied / very similar code between those two pages (and also some others) around filtering and pagination, so maybe there is an opportunity for more code sharing in the future here. Signed-off-by: Julius Volz <julius.volz@gmail.com> * docs: fix typo in operators.md Signed-off-by: marcoderama <marcoderamagit@gmail.com> * Release 3.4.0 (#16600) * Adjust release date in CHANGELOG.md Signed-off-by: Jan-Otto Kröpke <mail@jkroepke.de> * Merge release-3.4 into main (#16609) Co-authored-by: Ayoub Mrini <ayoubmrini424@gmail.com> * Fix flakiness in TestOTLPWriteHandler (#16608) * feature: type-and-unit-labels (PROM-39 implementation) (#16228) * feature: type-and-unit-labels (extended MetricIdentity) Experimental implementation of https://github.com/prometheus/proposals/pull/39 Previous (unmerged) experiments: * https://github.com/prometheus/prometheus/compare/main...dashpole:prometheus:type_and_unit_labels * https://github.com/prometheus/prometheus/pull/16025 Signed-off-by: bwplotka <bwplotka@gmail.com> feature: type-and-unit-labels (extended MetricIdentity) Experimental implementation of https://github.com/prometheus/proposals/pull/39 Previous (unmerged) experiments: * https://github.com/prometheus/prometheus/compare/main...dashpole:prometheus:type_and_unit_labels * https://github.com/prometheus/prometheus/pull/16025 Signed-off-by: bwplotka <bwplotka@gmail.com> * Fix compilation errors Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com> Lint Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com> Revert change made to protobuf 'Accept' header Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com> Fix compilation errors for 'dedupelabels' tag Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com> * Rectored into schema.Metadata Signed-off-by: bwplotka <bwplotka@gmail.com> * texparse: Added tests for PromParse Signed-off-by: bwplotka <bwplotka@gmail.com> * add OM tests. Signed-off-by: bwplotka <bwplotka@gmail.com> * add proto tests Signed-off-by: bwplotka <bwplotka@gmail.com> * Addressed comments. Signed-off-by: bwplotka <bwplotka@gmail.com> * add schema label tests. Signed-off-by: bwplotka <bwplotka@gmail.com> * addressed comments. Signed-off-by: bwplotka <bwplotka@gmail.com> * fix tests. Signed-off-by: bwplotka <bwplotka@gmail.com> * add promql tests. Signed-off-by: bwplotka <bwplotka@gmail.com> * lint Signed-off-by: bwplotka <bwplotka@gmail.com> * Addressed comments. Signed-off-by: bwplotka <bwplotka@gmail.com> --------- Signed-off-by: bwplotka <bwplotka@gmail.com> Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com> Co-authored-by: Arthur Silva Sens <arthursens2005@gmail.com> * util/httputil: Always add Vary header in SetCORS Closes #15406 Signed-off-by: jub0bs <jcretel-infosec+github@protonmail.com> * refactor: use the built-in max/min to simplify the code Signed-off-by: carrychair <linghuchong404@gmail.com> * refactor(endpointslice): use service cache.Indexer to achieve better iteration performance (#16365) * refactor(endpointslice): use cache.Indexer to index endpointslices by LabelServiceName so not have to iterate over all endpoint objects. Signed-off-by: Ryan Wu <rongjun0821@gmail.com> * check the type and error early and add 'TestEndpointSliceDiscoveryWithUnrelatedServiceUpdate' unit test to give a regression test Signed-off-by: Ryan Wu <rongjun0821@gmail.com> * make service indexer namespaced Signed-off-by: Ryan Wu <rongjun0821@gmail.com> * remove unneeded test func Signed-off-by: Ryan Wu <rongjun0821@gmail.com> * Apply suggestions from code review Co-authored-by: Ayoub Mrini <ayoubmrini424@gmail.com> Signed-off-by: Ryan Wu <rongjun0821@gmail.com> --------- Signed-off-by: Ryan Wu <rongjun0821@gmail.com> Co-authored-by: Ayoub Mrini <ayoubmrini424@gmail.com> * feat(tsdb): introduce --use-uncached-io feature flag and allow using it for chunks writing (#15365) Signed-off-by: machine424 <ayoubmrini424@gmail.com> Signed-off-by: Ayoub Mrini <ayoubmrini424@gmail.com> * test(cmd/main/TestRuntimeGOGCConfig): add checks on reloads as well Signed-off-by: machine424 <ayoubmrini424@gmail.com> * chore(config): add guidelines for adding a new RuntimeConfig field based on learnings from GoGC addition Signed-off-by: machine424 <ayoubmrini424@gmail.com> * fix: add reproducer for a dangling-reference issue in parsers Signed-off-by: machine424 <ayoubmrini424@gmail.com> * fix(model/textparse): Labels(): copy the input to avoid dangling references Signed-off-by: machine424 <ayoubmrini424@gmail.com> * fix metric_name_escaping_scheme option (#16625) * remote_write azure auth : allow empty client_id to suppport system assigned managed identity (#16421) * squash (#1) * remote-write: allow empty azure client_id to support system assigned managed identity * add blank line for tests * remote-write: allow empty azure client_id to support system assigned managed identity Signed-off-by: Kaveesh Dubey <kadubey@microsoft.com> * add blank line for tests Signed-off-by: Kaveesh Dubey <kadubey@microsoft.com> --------- Signed-off-by: Kaveesh Dubey <kadubey@microsoft.com> * treat empty client_id as system-assigned identity; this is a valid case Signed-off-by: Kaveesh Dubey <kadubey@microsoft.com> * rename file Signed-off-by: bragi92 <kadubey@microsoft.com> --------- Signed-off-by: Kaveesh Dubey <kadubey@microsoft.com> Signed-off-by: bragi92 <kadubey@microsoft.com> * ci: address zizmor gh action vulnerabilities (#16530) * ci: address zizmor gh action vulnerabilities --------- Signed-off-by: Joe Harvey <51208233+jharvey10@users.noreply.github.com> Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Add support for promoting all OTel resource attributes (#16426) Add support for promoting all OTel resource attributes via `promote_all_resource_attributes`, except for those ignored using 'ignore_resource_attributes'. --------- Signed-off-by: Antonio Jimenez <antonjim@thousandEyes.com> Signed-off-by: Antonio Jimenez <123171955+antonjim-te@users.noreply.github.com> * Add parse alerting for rules files (#16601) Builds over https://github.com/prometheus/prometheus/pull/16462 Addresses comments, adds invalid rules file Signed-off-by: subhramit <subhramit.bb@live.in> Co-authored-by: marcodebba <marcodebonis74@gmail.com> * chore: Adjust test and add comment about DNS resolution issue for failing tests (#16200) * chore: Add comment about DNS resolution issue for failing tests Signed-off-by: zhengkezhou1 <madzhou1@gmail.com> * remove unexported-return Signed-off-by: zhengkezhou1 <madzhou1@gmail.com> --------- Signed-off-by: zhengkezhou1 <madzhou1@gmail.com> * [BACKPORT-3.4] fix: add reproducer for a dangling-reference issue in parsers and fix (#16633) * chore: fix some comments Signed-off-by: tongjicoder <tongjicoder@icloud.com> * refactor: Move 'for' state restoration metrics to defer block Signed-off-by: Anand Rajagopal <anrajag@amazon.com> * tsdb: fix issue where a new segment file is created for every chunk if `WithSegmentSize` not called (#16635) * tsdb: fix issue where a new segment file is created for every chunk Signed-off-by: Charles Korn <charles.korn@grafana.com> * Address PR feedback Signed-off-by: Charles Korn <charles.korn@grafana.com> --------- Signed-off-by: Charles Korn <charles.korn@grafana.com> * Standardize doc page title handling See https://groups.google.com/g/prometheus-developers/c/cwL3cM66Em8 Signed-off-by: Julius Volz <julius.volz@gmail.com> * fix(test): fall back to default direct I/O requirements in tests when statx isn't supported by using a higher lever util Signed-off-by: machine424 <ayoubmrini424@gmail.com> * Update otlptranslator with new API (#16626) * Update otlptranslator with new API Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com> --------- Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com> * Release 3.4.1 (#16652) * test(promql): histogram_count inconsistent Ref: https://github.com/prometheus/prometheus/issues/16681 Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com> * fix(promql): histogram_count inconsistent The problem is in the counter reset detection. The code that loads the samples is matrixIterSlice which uses the typed Buffer iterator, which will preload the integer histogram samples, however the last sample is always(!) loaded as a float histogram sample in matrixIterSlice and the optimized iterator fails to detect counter resets in that case. Also the iterator does not reset lastH, lastFH properly. Ref: https://github.com/prometheus/prometheus/issues/16681 Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com> * Make translatorMetricFromOtelMetric public (#16683) Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com> * fix(promql): do not loose information about buckets when doing the detect Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com> * bugfix: Default OTLP config not respected when `otlp:` block is unset (#16693) Co-authored-by: Jan-Otto Kröpke <mail@jkroepke.de> * promqltest: Adjust more tests to "left-open" ranges This was an oversight because the old tests still happened to pass with the new behavior, but important test data was excluded at the left end of the interval, rendering some tests not actually testing what we want to test. In the past, we also applied different strategies to adjust the test (extend the range from 1m to 2m, or set the evaluation timestamp to 45s). This commit unifies things and reduces redundancy. Signed-off-by: beorn7 <beorn@grafana.com> * promql: Simplify avg aggregation and avg_over_time As it turns out, if we combine Kahan summation and incremental mean calculation properly, it works quite well and we do not need to switch between simple mean calculation and incremental calculation based on overflow. This simplifies the code quite a bit. Signed-off-by: beorn7 <beorn@grafana.com> * Export QuerySamples StartTimestamp and Interval to enable custom statistics implementations to populate these fields as needed. Signed-off-by: Innokentii Konstantinov <innokenty.konstantinov@grafana.com> * chore(docs/kubernetes SD): add a note about Endpoints API being deprecated in kubernetes 1.33+ (#16684) * chore(docs/kubernetes SD): add a note about Endpoints API being deprecated in kubernetes 1.33+ Signed-off-by: machine424 <ayoubmrini424@gmail.com> * chore(discovery/kubernetes): add Endpoints API deprecation comment Signed-off-by: machine424 <ayoubmrini424@gmail.com> --------- Signed-off-by: machine424 <ayoubmrini424@gmail.com> * PROM-39: Provide PromQL info annotations when rate()/increase() over series without `__type__`="counter" label (#16632) * Provide PromQL info annotations when rate()/increase() over series without counter label Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com> * Address comments Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com> --------- Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com> * [Refactor] PromQL: Simplify detectHistogramStatsDecoding Restarting the depth-first walk on each leg of a binary expression is convoluted. ISTM the correct logic is to walk the path backwards to the first relevant function. Signed-off-by: Bryan Boreham <bjboreham@gmail.com> * Add more avg_over_time test cases with extreme values These tests were initially created by @crush-on-anechka. I modified them slightly. Signed-off-by: beorn7 <beorn@grafana.com> * Enhancement: Reload all scrape pools concurrently (#16595) * Reload all scrape pools concurrently At the moment all scrape pools that need to be reloaded are reloaded one by one. While reloads are ongoing mtxScrape is locked. For each pool that's being reloaded we need to wait until all targets are updated. This whole process can take a while and the more scrape pools to reload the longer. At the same time all pools are independent and there's no real reason to do them one-by-one. Reload each pool in a seperate goroutine so we finish config reload as ASAP as possible and unlock the mtxScrape. Signed-off-by: Lukasz Mierzwa <l.mierzwa@gmail.com> * Address PR review feedback Signed-off-by: Lukasz Mierzwa <l.mierzwa@gmail.com> --------- Signed-off-by: Lukasz Mierzwa <l.mierzwa@gmail.com> * promql: Deactivate three failing tests for the time being These tests fail on darwin/arm64. One is expected, because the test demonstrates the limits of the numerical accuracy of our methods, and different inaccurate outcomes on different hardware are expected. The other two are mysterious at the moment, see https://github.com/prometheus/prometheus/issues/16714 for detailed discussion and debugging. Signed-off-by: beorn7 <beorn@grafana.com> * promql: typo fix in comment Signed-off-by: beorn7 <beorn@grafana.com> * engine: fix PossibleNonCounterInfo annotation for rate and increase. (#16718) * Comment exposed field Signed-off-by: Innokentii Konstantinov <innokenty.konstantinov@grafana.com> * Remove invalid test Signed-off-by: Innokentii Konstantinov <innokenty.konstantinov@grafana.com> * Fix PossibleNonCounterInfo annotation. Signed-off-by: Innokentii Konstantinov <innokenty.konstantinov@grafana.com> * lint Signed-off-by: Innokentii Konstantinov <innokenty.konstantinov@grafana.com> --------- Signed-off-by: Innokentii Konstantinov <innokenty.konstantinov@grafana.com> * fix(nhcb): do not loose NHCB after exponential with different labels The problem happens when we parse a standalone native histogram, which sets the p.lastHistogramExponential state flag. We never unset it. Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com> * refactor(nhcb): get rid of extra lastHistogramExponential state and make it a regular state of inhibiting NHCB conversion. Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com> * Enhance TestDropOldTimeSeries to test both v1 and v2 protocols (#16709) - Wrapped existing test logic in a loop to run with both protocol versions - Ensures consistent behavior across protocol versions for dropping old time series Signed-off-by: AxcelXander <tyz666@bu.edu> Co-authored-by: AxcelXander <tyz666@bu.edu> * mantine-ui: fix /alerts accordian theme colors (#16043) * fix-alerts-accordion-theme Signed-off-by: amanycodes <amanycodes@gmail.com> * fixes Signed-off-by: amanycodes <amanycodes@gmail.com> * added changes to /rules page Signed-off-by: amanycodes <amanycodes@gmail.com> * more fixes Signed-off-by: amanycodes <amanycodes@gmail.com> --------- Signed-off-by: amanycodes <amanycodes@gmail.com> Signed-off-by: Aman <95525722+amanycodes@users.noreply.github.com> * Fetch and display full query stats in hover tooltip in table query tab (#16723) Fixes https://github.com/prometheus/prometheus/issues/5857 Signed-off-by: Julius Volz <julius.volz@gmail.com> * promql: add ts_of_(max,min,last)_over_time functions This commit adds the ts_of_(max,min,last)_over_time functions behind the experimental feature flag. Signed-off-by: Michael Hoffmann <mhoffmann@cloudflare.com> * promql: Fix unary operator precedence in duration expressions Split unary operator handling in duration expressions into two specific cases to fix precedence conflicts: - Handle unary operators with number literals directly - Handle unary operators with parenthesized expressions separately This prevents unary minus from incorrectly binding to subsequent operators in expressions like `foo offset -1^1`, ensuring it parses as `(foo offset -1) ^ 1` rather than `foo offset (-1^1)`. Fixes #16711 Signed-off-by: Julien Pivotto <291750+roidelapluie@users.noreply.github.com> * Update duration expression docs Signed-off-by: Julien Pivotto <291750+roidelapluie@users.noreply.github.com> * promql: fix ts_of_last_over_time for histogram vectors This PR fixes a bug in ts_of_last_over_time where the float samples where used when computing the last timestamp of the histogram samples. Signed-off-by: Michael Hoffmann <mhoffmann@cloudflare.com> * config: fix typo in comments (#16734) Signed-off-by: Owen Williams <owen.williams@grafana.com> * discovery: add STACKIT SD (#16401) * Fix unit for Scrape failures Signed-off-by: datamuc <m@rbfh.de> * PromQL: Fix native histogram last_over_time with offset Signed-off-by: 🌲 Harry 🌊 John 🏔 <johrry@amazon.com> * OTLP: Support including scope metadata as metric labels (#16730) Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * promql: reuse `histogramStatsIterator` where possible, and expose it for other implementations to use (#16686) * Expose type * Add `Reset` method --------- Signed-off-by: Charles Korn <charles.korn@grafana.com> * Add script for converting PromQL tests to new syntax format (#16562) Signed-off-by: Kapil Lamba <kapillamba4@gmail.com> Co-authored-by: Neeraj Gartia <80708727+NeerajGartia21@users.noreply.github.com> * mantine-ui: add query warnings display to graph view Signed-off-by: Julien Pivotto <291750+roidelapluie@users.noreply.github.com> * fix: prevent invalid array access in aggregate expression This commit fixes the evaluation of invalid expressions like `sum(rate(`. Before that, it would trigger a panic in the PromQL engine because it tried to access an index which is out of range. The bug was probably introduced by 06d0b063ea. Signed-off-by: Simon Pasquier <spasquie@redhat.com> * mantine-ui: add query info warnings display to graph view Followup of https://github.com/prometheus/prometheus/pull/16753 Signed-off-by: Julien Pivotto <291750+roidelapluie@users.noreply.github.com> * OTLP receiver: Rename `otlp.convert_scope_metadata` to `otlp.promote_scope_metadata` (#16760) * Rename otlp.convert_scope_metadata to otlp.promote_scope_metadata --------- Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> Co-authored-by: Julius Hinze <juliusmh@proton.me> * [BUILD] Ran make update-go-deps Signed-off-by: Bryan Boreham <bjboreham@gmail.com> * [BUILD] Wind back otlptranslator update which has breaking change Signed-off-by: Bryan Boreham <bjboreham@gmail.com> * [BUILD] Wind back dependencies which require Go 1.24 We need Prometheus to build with the last 2 versions of Go. Remove Go toolchain line - it is redundant with the `go`` line above, and could mislead readers. Also don't update `fsnotify` which is pinned to an older version. Signed-off-by: Bryan Boreham <bjboreham@gmail.com> * [TESTS] Rules: remove brittle TestNewGroup It broke on different implementation of 'NewNopLogger'. Signed-off-by: Bryan Boreham <bjboreham@gmail.com> * [BUILD] Docker SD: Fix up deprecated types Signed-off-by: Bryan Boreham <bjboreham@gmail.com> * Add tests for relabeling of type and unit labels (#16743) Signed-off-by: Carrie Edwards <edwrdscarrie@gmail.com> * Add documentation for custom_values usage in NHCB (#16757) Signed-off-by: rishi-jat <rishijat098@gmail.com> Signed-off-by: Rishi Jat <rishijat098@gmail.com> Co-authored-by: George Krajcsovits <krajorama@users.noreply.github.com> * fix(promql): histogram_quantile and histogram_fraction NaN observed in native histogram (#16724) * fix(promql): histogram_quantile NaN observed in native histogram Fixes: #16578 See the issue for detailed explanation. When a histogram had only NaN observations and no normal observations, we returned 0 from the quantile, which is completely wrong. If there were normal observations but we went over them, we returned the upper bound of the existing buckets, however that contradicts expectations on histogram_fraction. Now we return NaN if the quantile is calculated to be over all normal observations, falling into NaNs (in a virtual +Inf bucket). We also return info level annotations if we see any NaN observations. The annotation calls out if we returned NaN or even if we took the virtual +Inf bucket into account. Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com> * fix(promql): histogram_fraction NaN observed in native histogram Fixes: #16580 According to the specification we should not take NaN observations into account when calculating the fraction. This commit fixes that and adds an info level annotation to let the user know about this. Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com> * scrape: set validation and escaping defaults in default config vars (#16751) Fixes https://github.com/prometheus/prometheus/issues/16750 Signed-off-by: Owen Williams <owen.williams@grafana.com> * OTLP receiver: Don't append _total suffix to non-monotonic OTel sums (#16776) * OTLP receiver: Don't append _total suffix to non-monotonic OTel sums Fix the OTLP receiver so the suffix _total isn't appended to metrics converted from non-monotonic OTel sum metrics, if otlp.translation_strategy is UnderscoreEscapingWithSuffixes or NoUTF8EscapingWithSuffixes. Also add translation tests. --------- Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Prepare release 3.4.2 Signed-off-by: Jan-Otto Kröpke <mail@jkroepke.de> * Limit concurrency of scrape pool reloads (#16783) To avoid possible overload. As per https://github.com/prometheus/prometheus/pull/16595#issuecomment-3005027067 this changes scrape pool manager to limit the number of scrape pools that can reload at the same time. Signed-off-by: Lukasz Mierzwa <l.mierzwa@gmail.com> * promql: Add test cases for direct mean calculation These demonstrate that direct mean calculation has some merits after all. Signed-off-by: beorn7 <beorn@grafana.com> * promql: Remove falsified comment from test The test in question actually worked fine even before #16569. The finding reported in the comment has turned out to be caused by something else. Signed-off-by: beorn7 <beorn@grafana.com> * promql: Re-introduce direct mean calculation for better accuracy This commit brings back direct mean calculation (for `avg` and `avg_over_time`) but isn't an outright revert of #16569. It keeps the improved incremental mean calculation and features generally a bit cleaner code than before. Also, this commit... - ...updates the lengthy comment explaining the whole situation and trade-offs. - ...divides the running sum and the Kahan compensation term separately (in direct mean calculation) to avoid the (unlikely) possibility that sum and Kahan compensation together ovorflow float64. - ...uncomments the tests that should now work again on darwin/arm64. - ...uncomments the test that should now reliably yield the (inaccurate) value 0 on all hardware platforms. Also, the test description has been updated accordingly. - ...adds avg_over_time tests for zero and one sample in the range. Signed-off-by: beorn7 <beorn@grafana.com> * Prepare release 3.5.0-rc.0 (#16778) Signed-off-by: Bryan Boreham <bjboreham@gmail.com> Co-authored-by: Julius Volz <julius.volz@gmail.com> * Revert: OTLP Support including scope metadata as metric labels (#16842) Reverts #16730 and #16760 This is being done because we've noticed a problem in the spec that could lead to name collisions if attributes name, version or schema_url are added to the scope. They would collide with the already reserved labels otel_scope_name, otel_scope_version and otel_scope_schema_url. Since this new configuration option never made it into a release, we can safely remove it from the 3.5 release. We'll sort this out for the 3.6 release Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com> * Prepare release 3.5.0-rc.1 (#16845) This RC reverts the feature "OTLP: Support promoting OTel scope attributes". Add the line back into the CHANGELOG for 3.5.0-rc.0, since we are not changing that version. Signed-off-by: Bryan Boreham <bjboreham@gmail.com> * Prepare release 3.5.0 Signed-off-by: Bryan Boreham <bjboreham@gmail.com> * [bot] assets: generate Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> * [create-pull-request] automated change Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> * Update base image version to rhel_9_1.24 * Update promu build command in Containerfile.operator * Update Dockerfile.prow * Modify build command to include static assets directory * Update Containerfile.operator * Update PREBUILT_ASSETS_STATIC_DIR in Dockerfile.prow * Update buf-lint.yml * Modify .gitignore for UI asset tracking Updated .gitignore to track specific UI assets and exclude others. * update assets Signed-off-by: Coleen Iona Quadros <coleen.quadros27@gmail.com> * Update Dockerfile.prow * Refactor Containerfile by removing unused copies --------- Signed-off-by: Jan-Otto Kröpke <mail@jkroepke.de> Signed-off-by: Neeraj Gartia <neerajgartia211002@gmail.com> Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com> Signed-off-by: Michael Hoffmann <mhoffmann@cloudflare.com> Signed-off-by: Michael Hoffmann <mhoffm@posteo.de> Signed-off-by: machine424 <ayoubmrini424@gmail.com> Signed-off-by: Fiona Liao <fiona.liao@grafana.com> Signed-off-by: Zhang Zhanpeng <zhangzhanpeng.zzp@alibaba-inc.com> Signed-off-by: amanycodes <amanycodes@gmail.com> Signed-off-by: SungJin1212 <tjdwls1201@gmail.com> Signed-off-by: Yuchen Wang <yuchen.wang@databricks.com> Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com> Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: Bryan Boreham <bjboreham@gmail.com> Signed-off-by: Jonas Lammler <jonas.lammler@hetzner-cloud.de> Signed-off-by: Nicolas Peugnet <nicolas.peugnet@lip6.fr> Signed-off-by: Graham Reed <greed@hypervolt.co.uk> Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com> Signed-off-by: Charles Korn <charles.korn@grafana.com> Signed-off-by: chardch <otwordsne@gmail.com> Signed-off-by: Dimitar Dimitrov <dimitar.dimitrov@grafana.com> Signed-off-by: Andre Branchizio <andrejbranch@gmail.com> Signed-off-by: dongjiang <dongjiang1989@126.com> Signed-off-by: Lukasz Mierzwa <l.mierzwa@gmail.com> Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com> Signed-off-by: Martin Danko <46035688+zepellin@users.noreply.github.com> Signed-off-by: Ryan Wu <rongjun0821@gmail.com> Signed-off-by: hardlydearly <799511800@qq.com> Signed-off-by: Julius Volz <julius.volz@gmail.com> Signed-off-by: Will Hegedus <whegedus@akamai.com> Signed-off-by: SuperQ <superq@gmail.com> Signed-off-by: marcoderama <marcoderamagit@gmail.com> Signed-off-by: bwplotka <bwplotka@gmail.com> Signed-off-by: jub0bs <jcretel-infosec+github@protonmail.com> Signed-off-by: carrychair <linghuchong404@gmail.com> Signed-off-by: Ayoub Mrini <ayoubmrini424@gmail.com> Signed-off-by: Kaveesh Dubey <kadubey@microsoft.com> Signed-off-by: bragi92 <kadubey@microsoft.com> Signed-off-by: Joe Harvey <51208233+jharvey10@users.noreply.github.com> Signed-off-by: Antonio Jimenez <antonjim@thousandEyes.com> Signed-off-by: Antonio Jimenez <123171955+antonjim-te@users.noreply.github.com> Signed-off-by: subhramit <subhramit.bb@live.in> Signed-off-by: zhengkezhou1 <madzhou1@gmail.com> Signed-off-by: tongjicoder <tongjicoder@icloud.com> Signed-off-by: Anand Rajagopal <anrajag@amazon.com> Signed-off-by: beorn7 <beorn@grafana.com> Signed-off-by: Innokentii Konstantinov <innokenty.konstantinov@grafana.com> Signed-off-by: AxcelXander <tyz666@bu.edu> Signed-off-by: Aman <95525722+amanycodes@users.noreply.github.com> Signed-off-by: Julien Pivotto <291750+roidelapluie@users.noreply.github.com> Signed-off-by: Owen Williams <owen.williams@grafana.com> Signed-off-by: datamuc <m@rbfh.de> Signed-off-by: 🌲 Harry 🌊 John 🏔 <johrry@amazon.com> Signed-off-by: Kapil Lamba <kapillamba4@gmail.com> Signed-off-by: Simon Pasquier <spasquie@redhat.com> Signed-off-by: Carrie Edwards <edwrdscarrie@gmail.com> Signed-off-by: rishi-jat <rishijat098@gmail.com> Signed-off-by: Rishi Jat <rishijat098@gmail.com> Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Signed-off-by: Coleen Iona Quadros <coleen.quadros27@gmail.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Jan-Otto Kröpke <mail@jkroepke.de> Co-authored-by: Neeraj Gartia <80708727+NeerajGartia21@users.noreply.github.com> Co-authored-by: Julien <291750+roidelapluie@users.noreply.github.com> Co-authored-by: George Krajcsovits <krajorama@users.noreply.github.com> Co-authored-by: Björn Rabenstein <beorn@grafana.com> Co-authored-by: Bryan Boreham <bjboreham@gmail.com> Co-authored-by: Arthur Silva Sens <arthursens2005@gmail.com> Co-authored-by: Michael Hoffmann <mhoffm@posteo.de> Co-authored-by: Björn Rabenstein <github@rabenste.in> Co-authored-by: machine424 <ayoubmrini424@gmail.com> Co-authored-by: Fiona Liao <fiona.liao@grafana.com> Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com> Co-authored-by: Zhang Zhanpeng <zhangzhanpeng.zzp@alibaba-inc.com> Co-authored-by: Aman <95525722+amanycodes@users.noreply.github.com> Co-authored-by: amanycodes <amanycodes@gmail.com> Co-authored-by: Julius Volz <julius.volz@gmail.com> Co-authored-by: SungJin1212 <tjdwls1201@gmail.com> Co-authored-by: Yuchen Wang <162491048+yuchen-db@…

Proposal for type and unit metadata labels

25305b0

Signed-off-by: David Ashpole <dashpole@google.com>

dashpole force-pushed the type_and_unit_labels branch from 8aab401 to 25305b0 Compare October 24, 2024 15:21

ArthurSens reviewed Oct 28, 2024

View reviewed changes

address feedback

e56b2be

Signed-off-by: David Ashpole <dashpole@google.com>

dashpole force-pushed the type_and_unit_labels branch from 98a86bd to e56b2be Compare November 1, 2024 15:35

beorn7 reviewed Nov 7, 2024

View reviewed changes

dashpole added 2 commits November 12, 2024 20:49

add goal of warning on inappropriate operations

6cfdbd4

Signed-off-by: David Ashpole <dashpole@google.com>

info annotation, rather than warning

de68600

Signed-off-by: David Ashpole <dashpole@google.com>

remove dmention of unit in text format

e0d62d3

Signed-off-by: David Ashpole <dashpole@google.com>

alexgreenbank mentioned this pull request Nov 19, 2024

Return UNIT information alongside queries prometheus/prometheus#12294

Open

bwplotka reviewed Nov 25, 2024

View reviewed changes

dashpole added 2 commits November 25, 2024 21:43

add native histogram motivation, and omit unit alternative

619537d

Signed-off-by: David Ashpole <dashpole@google.com>

rename feature flag to identifying-type-and-unit

a2e5ed9

Signed-off-by: David Ashpole <dashpole@google.com>

dashpole force-pushed the type_and_unit_labels branch from 15b0c44 to a2e5ed9 Compare November 26, 2024 02:52

dashpole added 4 commits November 26, 2024 03:12

add complex value alternative

75ccddd

Signed-off-by: David Ashpole <dashpole@google.com>

add ability to relabel type and unit

d9e0ad3

Signed-off-by: David Ashpole <dashpole@google.com>

preserve existing behavior for PromQL operations

f1760bd

Signed-off-by: David Ashpole <dashpole@google.com>

add future extension to handle type and unit in promql operations

bce6d9c

Signed-off-by: David Ashpole <dashpole@google.com>

dashpole force-pushed the type_and_unit_labels branch from d4ecde7 to bce6d9c Compare November 26, 2024 03:46

beorn7 previously requested changes Apr 22, 2025

View reviewed changes

bwplotka and others added 2 commits April 23, 2025 11:00

Apply suggestions from code review

e418574

Co-authored-by: Björn Rabenstein <github@rabenste.in> Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

Addressing comments.

057dfb8

Signed-off-by: bwplotka <bwplotka@gmail.com>

bwplotka requested review from ArthurSens, aknuds1, beorn7, carrieedwards and fionaliao April 23, 2025 10:08

ArthurSens approved these changes Apr 23, 2025

View reviewed changes

proposals/2024-09-25_metadata-labels.md Outdated Show resolved Hide resolved

Update proposals/2024-09-25_metadata-labels.md

8b4b61a

Co-authored-by: Arthur Silva Sens <arthursens2005@gmail.com> Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

bwplotka reviewed Apr 24, 2025

View reviewed changes

proposals/2024-09-25_metadata-labels.md Outdated Show resolved Hide resolved

Update proposals/2024-09-25_metadata-labels.md

821ce17

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

bwplotka merged commit 0c31275 into prometheus:main Apr 24, 2025
2 checks passed

ArthurSens mentioned this pull request Apr 28, 2025

Promote prometheusremotewriteexporter to Stable. open-telemetry/opentelemetry-collector-contrib#39706

Open

20 tasks

dashpole deleted the type_and_unit_labels branch May 14, 2025 14:41

ArthurSens mentioned this pull request May 14, 2025

Update Prometheus spec based on new updates to UTF-8 and name translation in upstream Prometheus open-telemetry/opentelemetry-specification#4494

Closed

3 tasks

bwplotka mentioned this pull request May 17, 2025

[meta] PROM-39 type-and-unit-labels stability prometheus/prometheus#16610

Open

8 tasks

ArthurSens mentioned this pull request Jun 9, 2025

Optionally add OM unit prometheus/client_golang#1392

Open

yeya24 mentioned this pull request Aug 7, 2025

Support Prometheus type and unit labels in OTLP receiver cortexproject/cortex#6952

Closed

bboreham mentioned this pull request Nov 18, 2025

Promtool Backfill ignoring TYPE and HELP prometheus/prometheus#17536

Closed

Conversation

dashpole commented Oct 24, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Querying for __unit__

Uh oh!

ArthurSens left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

beorn7 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

beorn7 commented Nov 12, 2024

Uh oh!

bwplotka commented Nov 12, 2024

Uh oh!

bwplotka left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

dashpole commented Nov 26, 2024

Uh oh!

beorn7 commented Nov 26, 2024

Uh oh!

ArthurSens commented Nov 26, 2024

Uh oh!

beorn7 commented Nov 27, 2024

Uh oh!

bwplotka commented Nov 27, 2024

Uh oh!

beorn7 commented Nov 27, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bwplotka commented Nov 27, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

beorn7 commented Nov 27, 2024

Uh oh!

bwplotka commented Nov 28, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

beorn7 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

bwplotka commented Apr 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ArthurSens left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

bwplotka commented Apr 24, 2025

Uh oh!

dashpole commented Oct 24, 2024 •

edited

Loading

Querying for `unit`

beorn7 commented Nov 27, 2024 •

edited

Loading

bwplotka commented Nov 27, 2024 •

edited

Loading

bwplotka commented Nov 28, 2024 •

edited

Loading

bwplotka commented Apr 23, 2025 •

edited

Loading