fix: Correct project id on client side metrics by avoiding getProjectId calls with the metric service client by danieljbruce · Pull Request #1757 · googleapis/nodejs-bigtable

danieljbruce · 2026-01-13T16:07:31Z

Description

The project id being recorded for client side metrics is incorrect. The client side metrics monitoring tests are showing that the project id the metrics get recorded for does not actually match the project id the client makes a grpc call for. Specifically, values recorded for bigtable.googleapis.com/frontend_server/handler_latencies don't match values recorded for bigtable.googleapis.com/internal/client/attempt_latencies, but they should match.

There is some speculation that since getProjectId calls on the MetricServiceClient determine the projectId recorded for client side metrics that the incorrect projectId is due to the differences between this getProjectId call on the MetricServiceClient and the way the project id is fetched on the Bigtable client. Therefore, we introduce this PR to record a projectId that is certain to be an exact match of the project id on the Bigtable client.

Impact

This has a high probability of correcting the project id for the client side metrics

Testing

The fixtures have changed to accommodate for the fact that we now pass the projectId through the OTEL stack.

modifications

mutianf · 2026-01-13T18:20:12Z

src/client-side-metrics/metrics-handler.ts

  client_name: string;
  streaming: StreamingState;
  status: string;
+  projectId: string;


I think projectId should be part of IMetricsCollectorData

I made this change

mutianf · 2026-01-13T18:27:45Z

src/client-side-metrics/operation-metrics-collector.ts

   * @param {grpc.status} attemptStatus The grpc status for the attempt.
   */
-  onAttemptComplete(attemptStatus: grpc.status) {
+  onAttemptComplete(projectId: string, attemptStatus: grpc.status) {


projectId is static and won't change per request. Why not add it to line 228 this.getMetricsCollectorData()?

Sure. Then we can get it from this.tabularApiSurface.

mutianf · 2026-01-13T18:28:51Z

src/client-side-metrics/gcp-metrics-handler.ts

 * @param exporter The exporter the metrics will be sent to.
 */
-function createInstruments(exporter: PushMetricExporter): MetricsInstruments {
+function createInstruments(


I don't think you need to update this, instead you can get it from the attirbutes in the data point:
https://github.com/googleapis/nodejs-bigtable/blob/3e532ab657409c7acd09008c143c41d36d9efd41/src/client-side-metrics/exporter.ts#L134C15-L135, similar to how you get instance id.

We can definitely do it that way, but then what do we do here? (

nodejs-bigtable/src/client-side-metrics/exporter.ts

Line 245 in 3e532ab

name: `projects/${projectId}`,

)

Should we just use the projectId from the first data point for name?

I don't think you need to update this, instead you can get it from the attirbutes in the data point

There is nothing wrong with doing it this way and I'll do this change, but is there any reason it is preferred?

Second thought, I don't think we should do it this way.

instead you can get it from the attributes in the data point

This is only true if I start sending the projectId through the OTEL stack which I am not doing right now. And I don't recommend doing that because it increases the data load on the OTEL stack with a bunch of projectId values across all the metrics that we know are all the same.

Let me know what you think.

Should we just use the projectId from the first data point for name?

Yes.

but is there any reason it is preferred?

OTEL attributes provide context of a metrics. If someone attaches a different exporter on their OTEL instance, without the project id as part of the attributes, this context would be missing and they won't be able to group or filter metrics on project id. It also feels weird to have the project id on Resource because Resource describes where the metric is coming from (in this case it's just bigtable). The extra memory usage from an extra project id field is minimal.

I made the change you requested.

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

…ub.com/googleapis/nodejs-bigtable into 467064670-pass-project-id-through-otel

mutianf · 2026-01-14T00:18:50Z

src/client-side-metrics/metric-interceptor.ts


 // Helper to create interceptor provider for OperationMetricsCollector
 function createMetricsInterceptorProvider(
+  table: TabularApiSurface,


why do we need to pass this in?

We don't. I removed this. We needed it before because onAttemptComplete needed the projectId, but now it doesn't need the project id.

mutianf · 2026-01-14T00:31:25Z

src/client-side-metrics/gcp-metrics-handler.ts

 * @param exporter The exporter the metrics will be sent to.
 */
-function createInstruments(exporter: PushMetricExporter): MetricsInstruments {
+function createInstruments(


Should we just use the projectId from the first data point for name?

Yes.

but is there any reason it is preferred?

OTEL attributes provide context of a metrics. If someone attaches a different exporter on their OTEL instance, without the project id as part of the attributes, this context would be missing and they won't be able to group or filter metrics on project id. It also feels weird to have the project id on Resource because Resource describes where the metric is coming from (in this case it's just bigtable). The extra memory usage from an extra project id field is minimal.

mutianf · 2026-01-14T18:11:44Z

src/client-side-metrics/gcp-metrics-handler.ts

      table: data.metricsCollectorData.table,
      cluster: data.metricsCollectorData.cluster,
      zone: data.metricsCollectorData.zone,
+      projectId: data.metricsCollectorData.projectId,


nit: can we move this above instanceId?

mutianf · 2026-01-14T18:11:51Z

src/client-side-metrics/gcp-metrics-handler.ts

      table: data.metricsCollectorData.table,
      cluster: data.metricsCollectorData.cluster,
      zone: data.metricsCollectorData.zone,
+      projectId: data.metricsCollectorData.projectId,


mutianf · 2026-01-14T18:12:10Z

src/client-side-metrics/metrics-handler.ts

  zone?: string;
  app_profile?: string;
  method: MethodName;
+  projectId: string;


nit: can we move this on top of instance id?

danieljbruce added 3 commits January 12, 2026 17:21

Make upstream and downstream changes for projectId

795ef62

modifications

skips the timed stream tests for now

e5e2f9c

Address broken unit test

54d948c

danieljbruce requested a review from a team January 13, 2026 16:07

danieljbruce requested a review from a team as a code owner January 13, 2026 16:07

product-auto-label bot added size: m Pull request size is medium. api: bigtable Issues related to the googleapis/nodejs-bigtable API. labels Jan 13, 2026

Don’t skip the unit tests

7dfd1e1

danieljbruce requested a review from mutianf January 13, 2026 16:21

danieljbruce added the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Jan 13, 2026

yoshi-kokoro removed the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Jan 13, 2026

mutianf requested changes Jan 13, 2026

View reviewed changes

danieljbruce and others added 6 commits January 13, 2026 16:27

Move the projectId to IMetricsCollectorData

6ca759f

Remove the only blocks

866fa4b

Eliminate the projectId parameter

110eb91

🦉 Updates from OwlBot post-processor

2311f9e

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

skip the timed stream tests

a2b5cbc

Merge branch '467064670-pass-project-id-through-otel' of https://gith…

6aaa69a

…ub.com/googleapis/nodejs-bigtable into 467064670-pass-project-id-through-otel

mutianf requested changes Jan 14, 2026

View reviewed changes

danieljbruce added 9 commits January 14, 2026 10:15

Move project id to the metrics collector data

766933e

pass project id through otel stack

de74a33

Modify fixtures to include projectId in OTEL stack

b4c1331

Do not pull the projectId from the metric service

5ce46cc

remove the onlys

fd864a1

Ensure metrics are written for correct projectId

f2c9985

Don’t pass projectId into the interceptors

af1c50d

Remove unused import

9590486

undo unnecessary changes

71e6abc

danieljbruce requested a review from mutianf January 14, 2026 17:56

mutianf reviewed Jan 14, 2026

View reviewed changes

Change project id location

7709d87

mutianf approved these changes Jan 14, 2026

View reviewed changes

danieljbruce added the automerge Merge the pull request once unit tests and other checks pass. label Jan 14, 2026

gcf-merge-on-green bot merged commit a719d9f into main Jan 14, 2026
24 checks passed

gcf-merge-on-green bot removed the automerge Merge the pull request once unit tests and other checks pass. label Jan 14, 2026

gcf-merge-on-green bot deleted the 467064670-pass-project-id-through-otel branch January 14, 2026 19:42

release-please bot mentioned this pull request Jan 9, 2026

chore(main): release 6.5.0 #1733

Merged

Conversation

danieljbruce commented Jan 13, 2026

Description

Impact

Testing

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants