logging: OpenTelemetry spans are ignored as parents for LogRecords #9302
Description
Client
logging: https://github.com/googleapis/google-cloud-go/tree/main/logging
Current behavior
The current implementation of populateTraceInfo always uses a remote span context as the parent
google-cloud-go/logging/logging.go
Lines 807 to 838 in ac16144
By "Remote span context", I mean a trace ID and span ID that originate from outside the application. The code above always uses a remote span context because it explicitly parses the trace id and span id from HTTP headers.
When a user has instrumented their application with OpenTelemetry, the trace ID and span ID have already been parsed from the HTTP headers, and used to start a span. If they are using the otelhttp library, it:
- extracts the context from req.Header into a golang context.Context
- uses the context to start a span
- updates the context.Context on the req using req.WithContext
Expected behavior
If the user is using opentelemetry instrumentation, we want to use the traceID and spanID of the span that OpenTelemetry created. This way, when a user looks for logs associated with that span, the logs produced by this logging library will appear. The traceID and spanID are easy to get since otelhttp stores them in req.Context().
We can do something close to this:
func populateTraceInfo(e *Entry, req *http.Request) bool {
if req == nil {
if e.HTTPRequest != nil && e.HTTPRequest.Request != nil {
req = e.HTTPRequest.Request
} else {
return false
}
}
// New handing code
otelSpanContext := trace.SpanContextFromContext(req.Context())
if otelSpanContext.IsValid() {
e.Trace := otelSpanContext.TraceID()
e.SpanID := otelSpanContext.SpanID()
e.TraceSampled = e.TraceSampled | otelSpanContext.IsSampled()
return true
}
// end of new code
header := req.Header.Get("Traceparent")
if header != "" {
// do not use traceSampled flag defined by traceparent because
// flag's definition differs from expected by Cloud Tracing
traceID, spanID, _ := deconstructTraceParent(header)
if traceID != "" {
e.Trace = traceID
e.SpanID = spanID
return true
}
}
header = req.Header.Get("X-Cloud-Trace-Context")
if header != "" {
traceID, spanID, traceSampled := deconstructXCloudTraceContext(header)
if traceID != "" {
e.Trace = traceID
e.SpanID = spanID
// enforce sampling if required
e.TraceSampled = e.TraceSampled || traceSampled
return true
}
}
return false
} To test this, we should make sure an http request with a context from a span creation ends up in the LogEntry:
req := &http.Request{}
ctx := context.Background()
ctx, _ = trace.NewNoopTracerProvider().Tracer("test instrumentation").Start(ctx, "span name")
req = req.WithContext(ctx)
e := &Entry{}
got := populateTraceInfo(e, req)
assert.True(t, got)
assert.Len(t, e.Trace, 32)
assert.Len(t, e.SpanID, 16)
// TODO test sampled somehow