-
Notifications
You must be signed in to change notification settings - Fork 956
Closed
Labels
Feature RequestSuggest an idea for this projectSuggest an idea for this project
Description
Suppose you have a span hierarchy of ({tracerName}:{spanKind}:{spanName}):
tracerA:server:parent -> tracerB:internal:child -> tracerC:client:grandchild
And you decide that internal spans from tracerB are too noisy. You want to turn them off while:
- Not losing downstream spans. You want to keep
tracerC:client:grandchildbecause it contains important information about calls to downstream systems. - Not producing broken traces.
tracerC:client:grandchild's parent should betracerA:server:parent, nottracerB:internal:child.
How can you accomplish this? Well, you pretty much can't. A couple of problems prevent it:
- Samplers dont have access to scope, so there's no SDK configuration mechanism which can do this.
- Even if Samplers could make decisions based on scope, the sampling decision options don't make the desired behavior easy.
- If a sampler returns
DROPfortracerBand delegates toParentBased{root=AlwaysOn}for other spans: Then all descendants oftracerB:internal:childwill be dropped as well. Not what we want. - If a sampler returns
DROPfortracerBand delegates toParentBased{root=AlwaysOn,localParentNotSampled=AlwaysOn}for other spans: Then we end up with a broken trace sincetracerC:client:grandchildwill assign its parent to betracerB:internal:child, which is dropped. - If a sampler returns
DROPfortracerBand delegates toParentBased{root=AlwaysOn,localParentNotSampled=AlwaysOn}and instrumentation is aware and only sets spans in context ifSpan.isRecording()=true: Then we achieve the desired affect.tracerB:internal:childis dropped andtracerC:client:grandchildwill assign its parent to betracerA:server:parent, so we have a complete trace.
- If a sampler returns
- Even if Samplers could make decisions based on scope, its cumbersome to configure a sampler to have special rules for a scope. It's simpler to think of a sampling policy as orthogonal to the decision of which scopes / instrumentation libraries should be enabled.
So there is no way to get the behavior we want today. And even if samplers were given access to scope, we'd still need: a complex / non-standard sampling policy and coordination with instrumentation (i.e. checking Span.isRecording()==true). Ouch.
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
Feature RequestSuggest an idea for this projectSuggest an idea for this project