Problem statement
In the W3C TraceContext specification it's possible that a system is monitored by multiple tracing tools.
Example: A -> B -> C (Service A calls Service B which calls Service C).
Let's say in this case B is monitored by an Elastic APM Agent, but A isn't - let's say A is monitored by another tool which also implements TraceContext. In this case the other tool will create a traceparent header with a traceid which is outside Elastic and a parentid which is from a non-Elastic tool. In B the Elastic APM agent will read this, create a transaction, set the traceid and the parentid based on the traceparent header. Problem is, in Kibana, under traces, only traces are listed where the 1. transaction has a parentid which is null (feel free to correct me, but from what I see that's how it works now), therefore in such a scenario those traces won't show up under traces in Kibana.
Related, recent issue from .NET
Reason I open this issue is that in .NET the situation is more-common, you don't even need another tool for this to happen: In ASP.NET Core, since version 3, there is built-in trace propagation support (details here). In short: the framework itself creates a traceid and it propagates it through HTTP calls. By default it uses a header called Request-Id, which is ok, but it has opt-in support for TraceContext and users can turn on context propagation through the traceparent header. That way in our previous example if A is not monitored at all, but opted-in and B is monitored by Elastic APM, we have the same problem.
Questions
- Do other agents potentially have an issue with this? From my understanding there is no way to know who set's the traceparent header, it could be easily another tool (that's in fact one of the motivation of the standard), or in case of .NET, the framework itself.
- I feel the key problem here is that the definition of a trace in Kibana is that it starts with a transaction where the parentid is null. Do others also feel, this is something which is not compatible with the TraceContext standard and we maybe should reiterate this?
- Related problem is when let's say B isn't monitored by Elastic APM, but A and C are - the parent id in C will point to something which is outside Elastic. Is this an issue for our UI?
Problem statement
In the W3C TraceContext specification it's possible that a system is monitored by multiple tracing tools.
Example: A -> B -> C (Service A calls Service B which calls Service C).
Let's say in this case B is monitored by an Elastic APM Agent, but A isn't - let's say A is monitored by another tool which also implements TraceContext. In this case the other tool will create a traceparent header with a traceid which is outside Elastic and a parentid which is from a non-Elastic tool. In B the Elastic APM agent will read this, create a transaction, set the traceid and the parentid based on the traceparent header. Problem is, in Kibana, under traces, only traces are listed where the 1. transaction has a parentid which is
null(feel free to correct me, but from what I see that's how it works now), therefore in such a scenario those traces won't show up under traces in Kibana.Related, recent issue from .NET
Reason I open this issue is that in .NET the situation is more-common, you don't even need another tool for this to happen: In ASP.NET Core, since version 3, there is built-in trace propagation support (details here). In short: the framework itself creates a traceid and it propagates it through HTTP calls. By default it uses a header called
Request-Id, which is ok, but it has opt-in support for TraceContext and users can turn on context propagation through thetraceparentheader. That way in our previous example if A is not monitored at all, but opted-in and B is monitored by Elastic APM, we have the same problem.Questions