Skip to content

Fix exemplars being added to gauge metrics in the prometheus exporter#5912

Merged
dmathieu merged 12 commits intoopen-telemetry:mainfrom
trthomps:fix-prometheus-gauge-exemplars
Oct 24, 2024
Merged

Fix exemplars being added to gauge metrics in the prometheus exporter#5912
dmathieu merged 12 commits intoopen-telemetry:mainfrom
trthomps:fix-prometheus-gauge-exemplars

Conversation

@trthomps
Copy link
Copy Markdown
Contributor

Prometheus Gauge metrics don't support exemplars and while addGaugeMetric() doesn't add them, addSumMetric() will if the metric is monotonic. This causes the prometheus client to throw an error:

* error collecting metric Desc{fqName: "http_server_request_body_size_bytes", help: "Measures size of RPC request messages (uncompressed).", constLabels: {}, variableLabels: {net_protocol_name,net_protocol_version,http_method,http_route,http_scheme,net_host_name,net_host_port,otel_scope_name,otel_scope_version}}: cannot inject exemplar into Gauge, Summary or Untyped

@linux-foundation-easycla
Copy link
Copy Markdown

linux-foundation-easycla Bot commented Oct 22, 2024

CLA Signed

The committers listed above are authorized under a signed CLA.

@trthomps trthomps marked this pull request as draft October 22, 2024 17:34
@trthomps trthomps force-pushed the fix-prometheus-gauge-exemplars branch from e713527 to 7f366df Compare October 22, 2024 17:38
@dashpole
Copy link
Copy Markdown
Contributor

Would be good to have a test

@codecov
Copy link
Copy Markdown

codecov Bot commented Oct 22, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 84.6%. Comparing base (30c4a9a) to head (3700b79).
Report is 1 commits behind head on main.

Additional details and impacted files

Impacted file tree graph

@@          Coverage Diff          @@
##            main   #5912   +/-   ##
=====================================
  Coverage   84.6%   84.6%           
=====================================
  Files        272     272           
  Lines      22837   22839    +2     
=====================================
+ Hits       19322   19325    +3     
+ Misses      3171    3170    -1     
  Partials     344     344           

see 2 files with indirect coverage changes

@trthomps trthomps force-pushed the fix-prometheus-gauge-exemplars branch from 45c3a10 to 2d16850 Compare October 22, 2024 20:27
@trthomps trthomps marked this pull request as ready for review October 22, 2024 20:27
@trthomps
Copy link
Copy Markdown
Contributor Author

Would be good to have a test

@dashpole Added a test, let me know if you know of a better way to test this, I did step through the debugger and exemplars are getting added to the DataPoint.

Comment thread CHANGELOG.md Outdated
Comment thread exporters/prometheus/exporter.go Outdated
Comment thread exporters/prometheus/exporter_test.go Outdated
Comment thread exporters/prometheus/exporter_test.go Outdated
trthomps and others added 4 commits October 23, 2024 10:54
Co-authored-by: Damien Mathieu <42@dmathieu.com>
Co-authored-by: Damien Mathieu <42@dmathieu.com>
Co-authored-by: David Ashpole <dashpole@google.com>
Co-authored-by: Damien Mathieu <42@dmathieu.com>
@trthomps
Copy link
Copy Markdown
Contributor Author

Thanks so much for working with me on this @dashpole.

Comment thread exporters/prometheus/exporter_test.go
Comment thread exporters/prometheus/exporter_test.go Outdated
Comment thread exporters/prometheus/exporter_test.go Outdated
@dmathieu dmathieu merged commit 664a075 into open-telemetry:main Oct 24, 2024
@trthomps trthomps deleted the fix-prometheus-gauge-exemplars branch October 24, 2024 16:30
@MrAlias MrAlias added this to the v1.32.0 milestone Nov 7, 2024
pellared added a commit that referenced this pull request Nov 8, 2024
### Added

- Add `go.opentelemetry.io/otel/sdk/metric/exemplar.AlwaysOffFilter`,
which can be used to disable exemplar recording. (#5850)
- Add `go.opentelemetry.io/otel/sdk/metric.WithExemplarFilter`, which
can be used to configure the exemplar filter used by the metrics SDK.
(#5850)
- Add `ExemplarReservoirProviderSelector` and
`DefaultExemplarReservoirProviderSelector` to
`go.opentelemetry.io/otel/sdk/metric`, which defines the exemplar
reservoir to use based on the aggregation of the metric. (#5861)
- Add `ExemplarReservoirProviderSelector` to
`go.opentelemetry.io/otel/sdk/metric.Stream` to allow using views to
configure the exemplar reservoir to use for a metric. (#5861)
- Add `ReservoirProvider`, `HistogramReservoirProvider` and
`FixedSizeReservoirProvider` to
`go.opentelemetry.io/otel/sdk/metric/exemplar` to make it convenient to
use providers of Reservoirs. (#5861)
- The `go.opentelemetry.io/otel/semconv/v1.27.0` package.
The package contains semantic conventions from the `v1.27.0` version of
the OpenTelemetry Semantic Conventions. (#5894)
- Add `Attributes attribute.Set` field to `Scope` in
`go.opentelemetry.io/otel/sdk/instrumentation`. (#5903)
- Add `Attributes attribute.Set` field to `ScopeRecords` in
`go.opentelemetry.io/otel/log/logtest`. (#5927)
- `go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc` adds
instrumentation scope attributes. (#5934)
- `go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp` adds
instrumentation scope attributes. (#5934)
- `go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetricgrpc`
adds instrumentation scope attributes. (#5935)
- `go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetrichttp`
adds instrumentation scope attributes. (#5935)
- `go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploggrpc` adds
instrumentation scope attributes. (#5933)
- `go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploghttp` adds
instrumentation scope attributes. (#5933)
- `go.opentelemetry.io/otel/exporters/prometheus` adds instrumentation
scope attributes in `otel_scope_info` metric as labels. (#5932)

### Changed

- Support scope attributes and make them as identifying for `Tracer` in
`go.opentelemetry.io/otel` and `go.opentelemetry.io/otel/sdk/trace`.
(#5924)
- Support scope attributes and make them as identifying for `Meter` in
`go.opentelemetry.io/otel` and `go.opentelemetry.io/otel/sdk/metric`.
(#5926)
- Support scope attributes and make them as identifying for `Logger` in
`go.opentelemetry.io/otel` and `go.opentelemetry.io/otel/sdk/log`.
(#5925)
- Make schema URL and scope attributes as identifying for `Tracer` in
`go.opentelemetry.io/otel/bridge/opentracing`. (#5931)
- Clear unneeded slice elements to allow GC to collect the objects in
`go.opentelemetry.io/otel/sdk/metric` and
`go.opentelemetry.io/otel/sdk/trace`. (#5804)

### Fixed

- Global MeterProvider registration unwraps global instrument Observers,
the undocumented Unwrap() methods are now private. (#5881)
- `go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetricgrpc`
now keeps the metadata already present in the context when `WithHeaders`
is used. (#5892)
- `go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploggrpc` now
keeps the metadata already present in the context when `WithHeaders` is
used. (#5911)
- `go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc` now
keeps the metadata already present in the context when `WithHeaders` is
used. (#5915)
- Fix `go.opentelemetry.io/otel/exporters/prometheus` trying to add
exemplars to Gauge metrics, which is unsupported. (#5912)
- Fix `WithEndpointURL` to always use a secure connection when an https
URL is passed in
`go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetricgrpc`.
(#5944)
- Fix `WithEndpointURL` to always use a secure connection when an https
URL is passed in
`go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetrichttp`.
(#5944)
- Fix `WithEndpointURL` to always use a secure connection when an https
URL is passed in
`go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc`.
(#5944)
- Fix `WithEndpointURL` to always use a secure connection when an https
URL is passed in
`go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp`.
(#5944)
- Fix incorrect metrics generated from callbacks when multiple readers
are used in `go.opentelemetry.io/otel/sdk/metric`. (#5900)

### Removed

- Remove all examples under `go.opentelemetry.io/otel/example` as they
are moved to [Contrib
repository](https://github.com/open-telemetry/opentelemetry-go-contrib/tree/main/examples).
(#5930)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants