Skip to content

bridge/opencensus: Causing infinite trace exporting as exporter traces itself #1928

@ahmetb

Description

@ahmetb

Description

When I use bridge/opencensus along with trace.WithBatcher and GCP's span exporter (github.com/GoogleCloudPlatform/opentelemetry-operations-go/exporter/trace), I get an infinite loop of Batch upload RPCs' own traces triggering more Batch uploads.

My motive of using opencensus bridge is to tap into builtin OpenCensus Tracing capabilities in many GCP Client Libraries that are already in place (but not captured in OpenTelemetry).

Environment

  • OS: linux
  • Architecture: x86_64
  • Go Version: 1.16
  • opentelemetry-go version: v0.20.0

Steps To Reproduce

gcp, err := texporter.NewExporter()
// handle err

tracer := trace.NewTracerProvider(trace.WithBatcher(gcp,
		trace.WithBatchTimeout(time.Millisecond*200),
		trace.WithMaxExportBatchSize(10)),
		trace.WithSampler(trace.AlwaysSample()))

otel.SetTracerProvider(tracer)
tp := otel.GetTracerProvider().Tracer("my-server")

// register opencensus bridge
octrace.DefaultTracer = opencensus.NewTracer(tp)

Once I do this:

  1. Good: It successfully exports OpenCensus spans (from GCP client libraries) to OpenTelemetry and they show up on Google Cloud Trace.

  2. Bad: My theory is that the batch uploader gets caught up in a loop. As soon as it starts pushing traces out, the "google.devtools.cloudtrace.v2.TraceService.BatchWriteSpans" span is also written into the exporter, so there's always something to export and I get a flood of spans that are just completely useless.

image

Span details of these spam traces from the tracer's own upload RPC:

image

Expected behavior

No infinite loops and trace span list being polluted.

cc: @dashpole @ymotongpoo

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

No type

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions