[serving] OTel Metrics Documentation #6352

dprotaso · 2025-08-28T17:01:01Z

Updating collecting metrics steps
remove collector installation steps and separate out shared metrics into a snippet

…nto a snippet

netlify · 2025-08-28T17:02:05Z

✅ Deploy Preview for knative ready!

Name	Link
🔨 Latest commit	`dce1aea`
🔍 Latest deploy log	https://app.netlify.com/projects/knative/deploys/68b85ec1bc5155000856b4f8
😎 Deploy Preview	https://deploy-preview-6352--knative.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

dprotaso · 2025-09-01T20:25:18Z

/assign @Cali0707 @evankanderson

Note for some of these docs to work now we need the dashboards PR to be merged - knative-extensions/monitoring#27

dprotaso · 2025-09-01T20:27:04Z

/cherry-pick release-1.19

knative-prow-robot · 2025-09-01T20:27:07Z

@dprotaso: once the present PR merges, I will cherry-pick it on top of release-1.19 in a new PR and assign it to you.

Details

In response to this:

/cherry-pick release-1.19

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

docs/eventing/observability/metrics/collecting-metrics.md

Cali0707 · 2025-09-03T15:07:07Z

docs/serving/observability/metrics/serving-metrics.md

+
+**Instrument Type:** Int64Gauge
+
+**Unit (UCUM):** {item}


What does UCUM stand for?

Some unit standard that OTel uses

{item} sorta means custom unit

Unified Code for Units of Measure

I do wonder if our unit here should be {request}, since that's what the queue is full of.

docs/snippets/collecting-metrics.md

Co-authored-by: Calum Murray <cmurray@redhat.com>

Cali0707

/lgtm

knative-prow · 2025-09-03T18:48:37Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: Cali0707, dprotaso

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

~~OWNERS~~ [Cali0707,dprotaso]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

knative-prow-robot · 2025-09-03T18:49:33Z

@dprotaso: #6352 failed to apply on top of branch "release-1.19":

Applying: Updating collecting metrics steps
.git/rebase-apply/patch:126: trailing whitespace.
    By default metrics are exporting is off. 
warning: 1 line adds whitespace errors.
Using index info to reconstruct a base tree...
M	docs/eventing/observability/metrics/collecting-metrics.md
M	docs/serving/observability/metrics/collecting-metrics.md
Falling back to patching base and 3-way merge...
Auto-merging docs/serving/observability/metrics/collecting-metrics.md
Auto-merging docs/eventing/observability/metrics/collecting-metrics.md
Applying: remove collector installation steps and separate out shared metrics into a snippet
.git/rebase-apply/patch:477: trailing whitespace.
Knative implements the [semantic conventions for Go runtime metrics](https://opentelemetry.io/docs/specs/semconv/runtime/go-metrics/) using the OpenTelemetry [otel-go/instrumentation/runtime](https://pkg.go.dev/go.opentelemetry.io/contrib/instrumentation/runtime) package. 
warning: 1 line adds whitespace errors.
Using index info to reconstruct a base tree...
M	docs/eventing/observability/metrics/eventing-metrics.md
M	docs/serving/observability/metrics/serving-metrics.md
Falling back to patching base and 3-way merge...
Removing docs/serving/observability/metrics/system-diagram.svg
Auto-merging docs/serving/observability/metrics/serving-metrics.md
CONFLICT (content): Merge conflict in docs/serving/observability/metrics/serving-metrics.md
Removing docs/serving/observability/metrics/collector.yaml
Removing docs/eventing/observability/metrics/system-diagram.svg
Auto-merging docs/eventing/observability/metrics/eventing-metrics.md
CONFLICT (content): Merge conflict in docs/eventing/observability/metrics/eventing-metrics.md
Removing docs/eventing/observability/metrics/collector.yaml
error: Failed to merge in the changes.
hint: Use 'git am --show-current-patch=diff' to see the failed patch
hint: When you have resolved this problem, run "git am --continue".
hint: If you prefer to skip this patch, run "git am --skip" instead.
hint: To restore the original branch and stop patching, run "git am --abort".
hint: Disable this message with "git config advice.mergeConflict false"
Patch failed at 0002 remove collector installation steps and separate out shared metrics into a snippet

Details

In response to this:

/cherry-pick release-1.19

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

evankanderson

A few small comments, but this seems to fill in the gaps for the 1.19 release, thanks!

evankanderson · 2025-09-03T20:00:27Z

docs/serving/observability/metrics/serving-metrics.md

+
+**Instrument Type:** Int64Gauge
+
+**Unit (UCUM):** {item}


Unified Code for Units of Measure

evankanderson · 2025-09-03T20:01:47Z

docs/serving/observability/metrics/serving-metrics.md

+
+**Instrument Type:** Int64Gauge
+
+**Unit (UCUM):** {item}


I do wonder if our unit here should be {request}, since that's what the queue is full of.

evankanderson · 2025-09-03T20:04:25Z

docs/serving/observability/metrics/serving-metrics.md

+### `kn.revision.request.concurrency`
+
+**Instrument Type:** Float64Gauge
+
+**Unit (UCUM):** {request}
+
+**Description:** Concurrent requests that are routed to the Activator
+
+The following attributes are included with the metrics below
+
+Name | Type | Description
+-|-|-
+`k8s.namespace.name` | string | Namespace of the Revision
+`kn.service.name` | string | Knative Service name associated with this Revision
+`kn.configuration.name` | string | Knative Configuration name associated with this Revision
+`kn.revision.name` | string | The name of the Revision


I wasn't certain about this format vs the previous tabular format, but reviewing it on the site preview, it seems pretty reasonable. I sort of want a database of these, but I don't think I want to invest in that enough to make it happen.

I'm finding the navigation bar on the left and the sidebar on the right make it harder to read wide tables due to word wrapping.

I don't think I want to invest in that enough to make it happen.

AI would make this work easier haha

evankanderson · 2025-09-03T20:06:21Z

docs/serving/observability/metrics/serving-metrics.md

+
+**Description:** Concurrent requests that are routed to the Activator
+
+The following attributes are included with the metrics below


If this applies to all the autoscaler metrics, it feels like it should be after the ## Autoscaler header.

It doesn't let me update

evankanderson · 2025-09-03T20:06:47Z

docs/serving/observability/metrics/serving-metrics.md

+
+**Instrument Type:** Int64Gauge
+
+**Unit (UCUM):** {item}


Should this unit be {pod}?

(and then document next to the other kn.revision.pods.* metrics)

evankanderson · 2025-09-03T20:09:01Z

docs/snippets/collecting-metrics.md


    ```bash
-    kubectl port-forward -n default svc/prometheus-grafana 3000:80
+    kubectl port-forward -n observability svc/knative-grafana 3000:80


I'm assuming the "load grafana dashboards" are currently an exercise for the user?

It's in the serving/../collecting-metrics.md and It's in the eventing/../collecting-metrics.md

evankanderson · 2025-09-03T20:10:09Z

docs/snippets/observability-shared-metrics.md

+
+Webhook metrics report useful info about operations. For example, if a large number of operations fail, this could indicate an issue with a user-created resource.
+
+### `http.server.request.duration`


Deliberate not to have ``Instrument Type:andUnit (UCUM):`?

Since this is an OTel metric I link out to it which includes the details.

* Cherry-pick (#6352) * OTel docs follow up (#6360) * include UCUM link * address Evan's OTel Docs feedback * include autoscaler attributes

dprotaso added 2 commits August 26, 2025 11:24

Updating collecting metrics steps

56c7d6b

remove collector installation steps and separate out shared metrics i…

77e6689

…nto a snippet

knative-prow bot added do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels Aug 28, 2025

knative-prow bot requested review from creydr and skonto August 28, 2025 17:01

knative-prow bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Aug 28, 2025

dprotaso added 2 commits August 28, 2025 13:11

combine notes

885d93c

update nav to remove stutter

4b226de

dprotaso force-pushed the otel-metrics branch 8 times, most recently from 2761f10 to 0690f72 Compare August 28, 2025 18:50

include webhook metrics

64d0195

dprotaso force-pushed the otel-metrics branch from 0690f72 to 64d0195 Compare August 28, 2025 18:54

dprotaso added 2 commits September 1, 2025 16:06

update common metrics with attributes and include serving metrics

90416c4

update notice banners for eventing and serving

e13c6e6

dprotaso changed the title ~~[wip] OTel Metrics Documentation~~ OTel Metrics Documentation Sep 1, 2025

knative-prow bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Sep 1, 2025

dprotaso changed the title ~~OTel Metrics Documentation~~ [serving] OTel Metrics Documentation Sep 1, 2025

fix typos

8cbd5e7

knative-prow bot assigned Cali0707 and evankanderson Sep 1, 2025

Cali0707 reviewed Sep 3, 2025

View reviewed changes

Update docs/eventing/observability/metrics/collecting-metrics.md

dce1aea

Co-authored-by: Calum Murray <cmurray@redhat.com>

Cali0707 approved these changes Sep 3, 2025

View reviewed changes

knative-prow bot added the lgtm Indicates that a PR is ready to be merged. label Sep 3, 2025

knative-prow bot merged commit b936c72 into knative:main Sep 3, 2025
19 checks passed

evankanderson reviewed Sep 3, 2025

View reviewed changes

dprotaso deleted the otel-metrics branch September 3, 2025 20:25

dprotaso mentioned this pull request Sep 3, 2025

Serving Metric Tweaks knative/serving#16062

Merged

dprotaso added a commit to dprotaso/knative-docs that referenced this pull request Sep 3, 2025

Cherry-pick (knative#6352)

18a7930

dprotaso mentioned this pull request Sep 3, 2025

[release-1.19] Cherry pick OTel docs changes #6361

Merged

knative-prow bot pushed a commit that referenced this pull request Sep 3, 2025

[release-1.19] Cherry pick OTel docs changes (#6361)

01ca6d0

* Cherry-pick (#6352) * OTel docs follow up (#6360) * include UCUM link * address Evan's OTel Docs feedback * include autoscaler attributes


		Description: Concurrent requests that are routed to the Activator

		The following attributes are included with the metrics below


		Webhook metrics report useful info about operations. For example, if a large number of operations fail, this could indicate an issue with a user-created resource.

		### `http.server.request.duration`

[serving] OTel Metrics Documentation #6352

[serving] OTel Metrics Documentation #6352

Uh oh!

Conversation

dprotaso commented Aug 28, 2025

Uh oh!

netlify bot commented Aug 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ Deploy Preview for knative ready!

Uh oh!

dprotaso commented Sep 1, 2025

Uh oh!

dprotaso commented Sep 1, 2025

Uh oh!

knative-prow-robot commented Sep 1, 2025

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Cali0707 left a comment

Choose a reason for hiding this comment

Uh oh!

knative-prow bot commented Sep 3, 2025

Uh oh!

Uh oh!

knative-prow-robot commented Sep 3, 2025

Uh oh!

evankanderson left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

netlify bot commented Aug 28, 2025 •

edited

Loading