Skip to content

Prometheus translation modes#4533

Merged
jsuereth merged 14 commits intoopen-telemetry:mainfrom
ArthurSens:prometheus-translation-modes
Jul 24, 2025
Merged

Prometheus translation modes#4533
jsuereth merged 14 commits intoopen-telemetry:mainfrom
ArthurSens:prometheus-translation-modes

Conversation

@ArthurSens
Copy link
Member

@ArthurSens ArthurSens commented Jun 2, 2025

Related to #4494

Changes

This change focuses on three things:

  • Flexibilization of underscore scaping
  • Flexibilization of unit and type suffix additions

The goal is to mimic Prometheus' existing configuration that controls the translation of metric names when receiving OTLP metrics via its /otlp endpoint. Configuration alignment will help users migrate in both directions (Prometheus/Collector) when switching their metrics collection systems.

Additional Information

Some further reading might be helpful for reviewers:

  • Our guide for receiving metrics via OTLP, specifically the [UTF-8 section]

@ArthurSens ArthurSens marked this pull request as ready for review June 11, 2025 13:03
@ArthurSens ArthurSens requested review from a team June 11, 2025 13:03
@ArthurSens
Copy link
Member Author

Folks, I'm out until June 25th. @ywwg is picking this up while I'm gone :)

I've removed the draft mode to help Owen with his notification settings

@github-actions
Copy link

This PR was marked stale due to lack of activity. It will be closed in 7 days.

@github-actions github-actions bot added the Stale label Jun 19, 2025
@ywwg
Copy link
Contributor

ywwg commented Jun 20, 2025

not stale

@ywwg
Copy link
Contributor

ywwg commented Jun 20, 2025

(it won't let me remove the stale label)

@github-actions github-actions bot removed the Stale label Jun 21, 2025
github-merge-queue bot pushed a commit that referenced this pull request Jun 24, 2025
## Changes

Addressing feedback received during the last Spec SIG meeting, I'm
splitting my previous
PR(#4533)
into two to ensure we keep the scope of the original one for the
configuration changes.

If I understand correctly, almost all SDKs (if not all) are already
compliant since they are wrappers around Prometheus SDKs, and Prometheus
SDKs already do content negotiation properly. Then the PoCs are already
done :)

* [X] Related issues #4494 
* [ ] Links to the prototypes (when adding or changing features)
* [x]
[`CHANGELOG.md`](https://github.com/open-telemetry/opentelemetry-specification/blob/main/CHANGELOG.md)
file updated for non-trivial changes
* [ ]
[`spec-compliance-matrix.md`](https://github.com/open-telemetry/opentelemetry-specification/blob/main/spec-compliance-matrix.md)
updated if necessary

---------

Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com>
Co-authored-by: Reiley Yang <reyang@microsoft.com>
Co-authored-by: David Ashpole <dashpole@google.com>
@ArthurSens ArthurSens force-pushed the prometheus-translation-modes branch from 591044a to e8ee790 Compare June 27, 2025 13:41
@ArthurSens ArthurSens changed the title Prometheus translation modes and content negotiation Prometheus translation modes Jun 27, 2025
@ArthurSens
Copy link
Member Author

The CI failure seems unrelated. Could someone re-trigger it? 😇

Copy link
Contributor

@ywwg ywwg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

gahhh I didn't post my comments before PTO, sorry

@reyang
Copy link
Member

reyang commented Jul 10, 2025

@ArthurSens could you update the changelog?

…tion

Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com>
Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com>
Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com>
Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com>
Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com>
@ArthurSens ArthurSens force-pushed the prometheus-translation-modes branch from bb0d980 to f726025 Compare July 10, 2025 18:04
Co-authored-by: Robert Pająk <pellared@hotmail.com>
@dashpole
Copy link
Contributor

I think this is ready to be merged

dashpole added a commit to open-telemetry/opentelemetry-go that referenced this pull request Jul 21, 2025
Fixes #7039
Fixes #6704

This uses the common prometheus/otlptranslator library to handle name
conversion. It was a little tricky to work around the fact that the
library only lets us configure whether all suffixes are added or not.
But we want to keep supporting WithoutUnit and WithoutCounterSuffixes
for a while longer. Those will eventually be deprecated and replaced
after
open-telemetry/opentelemetry-specification#4533
is released.

We decided to go ahead with the changes despite it being a small
behavioral change when UTF8 is enabled. See:
prometheus/otlptranslator#44 for the
rationale.

This adds a unit test to verify that it properly handles bracketed
units. The test fails on main with:

```
--- FAIL: TestPrometheusExporter (0.01s)
    --- FAIL: TestPrometheusExporter/counter_with_bracketed_unit (0.00s)
        exporter_test.go:646:
            	Error Trace:	/usr/local/google/home/dashpole/go/src/go.opentelemetry.io/opentelemetry-go/exporters/prometheus/exporter_test.go:646
            	Error:      	Received unexpected error:
            	            	-# HELP "foo_{spans}_total" a simple counter
            	            	-# TYPE "foo_{spans}_total" counter
            	            	-{"foo_{spans}_total",A="B",C="D",E="true",F="42",otel_scope_fizz="buzz",otel_scope_name="testmeter",otel_scope_schema_url="",otel_scope_version="v0.1.0"} 24.3
            	            	-{"foo_{spans}_total",A="D",C="B",E="true",F="42",otel_scope_fizz="buzz",otel_scope_name="testmeter",otel_scope_schema_url="",otel_scope_version="v0.1.0"} 5
            	            	+# HELP foo_total a simple counter
            	            	+# TYPE foo_total counter
            	            	+foo_total{A="B",C="D",E="true",F="42",otel_scope_fizz="buzz",otel_scope_name="testmeter",otel_scope_schema_url="",otel_scope_version="v0.1.0"} 24.3
            	            	+foo_total{A="D",C="B",E="true",F="42",otel_scope_fizz="buzz",otel_scope_name="testmeter",otel_scope_schema_url="",otel_scope_version="v0.1.0"} 5
            	            	 # HELP target_info Target metadata
            	            	 # TYPE target_info gauge
            	            	 target_info{"service.name"="prometheus_test","telemetry.sdk.language"="go","telemetry.sdk.name"="opentelemetry","telemetry.sdk.version"="latest"} 1

            	Test:       	TestPrometheusExporter/counter_with_bracketed_unit
2025/07/18 15:07:47 internal_logging.go:50: "msg"="Using existing type definition." "error"="instrument type conflict" "instrument"="foo_total" "existing"="COUNTER" "dropped"="GAUGE"
2025/07/18 15:07:47 internal_logging.go:50: "msg"="Using existing type definition." "error"="instrument type conflict" "instrument"="foo_bytes" "existing"="GAUGE" "dropped"="HISTOGRAM"
FAIL
FAIL	go.opentelemetry.io/otel/exporters/prometheus	0.054s
FAIL
```

cc @TylerHelmuth @ywwg @ArthurSens

---------

Co-authored-by: Tyler Yahn <MrAlias@users.noreply.github.com>
Co-authored-by: Robert Pająk <pellared@hotmail.com>
@jsuereth jsuereth added this pull request to the merge queue Jul 24, 2025
Merged via the queue into open-telemetry:main with commit f0c0cf2 Jul 24, 2025
6 checks passed
@ArthurSens ArthurSens deleted the prometheus-translation-modes branch July 24, 2025 14:56
songy23 pushed a commit to open-telemetry/opentelemetry-collector-contrib that referenced this pull request Aug 1, 2025
#### Description
Implements `translation_strategy` as described in
open-telemetry/opentelemetry-specification#4533

#### Link to tracking issue
Fixes #35459

<!--Describe what testing was performed and which tests were added.-->
#### Testing

<!--Describe the documentation added.-->
#### Documentation

<!--Please delete paragraphs that you did not use before submitting.-->

---------

Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com>
Dylan-M pushed a commit to Dylan-M/opentelemetry-collector-contrib that referenced this pull request Aug 5, 2025
…#41530)

#### Description
Implements `translation_strategy` as described in
open-telemetry/opentelemetry-specification#4533

#### Link to tracking issue
Fixes open-telemetry#35459

<!--Describe what testing was performed and which tests were added.-->
#### Testing

<!--Describe the documentation added.-->
#### Documentation

<!--Please delete paragraphs that you did not use before submitting.-->

---------

Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com>
@carlosalberto carlosalberto mentioned this pull request Aug 11, 2025
github-merge-queue bot pushed a commit that referenced this pull request Aug 13, 2025
August 2025 release.

### Logs

- Improve concurrency safety description of `LogRecordProcessor.OnEmit`.

([#4578](#4578))
- Clarify that `SeverityNumber` values are used when comparing
severities.

([#4552](#4552))

### Entities

- Mention entity references in the stability guarantees.

([#4593](#4593))

### OpenTelemetry Protocol

- Clarify protocol defaults on specification.

([#4585](#4585))

### Compatibility

- Flexibilie escaping of characters that are discouraged by Prometheus
Conventions
  in Prometheus exporters.

([#4533](#4533))
- Flexibilize addition of unit/type related suffixes in Prometheus
exporters.

([#4533](#4533))
- Define the configuration option "Translation Strategies" for
Prometheus exporters.

([#4533](#4533))
- Define conversion of Prometheus native histograms to OpenTelemetry
exponential histograms.

([#4561](#4561))
- Clarify what to do when scope attribute conflicts with name, version
and schema URL.

([#4599](#4599))

### SDK Configuration

- Enum values provided via environment variables SHOULD be interpreted
in a case-insensitive manner.

([#4576](#4576))

Co-authored-by: Armin Ruech <7052238+arminru@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants