Describe the feature request

Recently we contributed new features on Istio to improve the support for OpenTelemetry tracing in Istio. OTLP/HTTP Exporter and Resource Detectors.

We want to add the last piece of our contributions from Envoy to istio: The ability to configure samplers to the OpenTelemetry tracing provider.

The OpenTelemetry project defines a Sampler API and this API is available now in Envoy. With the next Envoy release, it will also come with a Dynatrace Sampler. To make it clear, the API can be implemented by anyone, so it is expected that more samplers are added in Envoy.

I had a discussion with @howardjohn in one of the PRs on how the sampler would interact with the existing ways of configuring sampling in Istio, and the conclusion we arrived was:

provider.sampler > telemetry.RandomSamplingPercentage > defaultConfig.tracing.sampling > PILOT_TRACE_SAMPLING

Which means:

• If OpenTelemetryTracingProvider has a custom sampler configured:

  • Set telemetry.RandomSamplingPercentage to 100 as all spans should arrive on the sampler, so it can perform its algorithm. This is hardcoded and the user cannot change (doesn't make sense to). Without all spans a sampler can't make consistent sampling decisions.

• Else continue with the same precedence as today, essentially nothing changes when users don't use a Sampler:

 telemetry.RandomSamplingPercentage > defaultConfig.tracing.sampling > PILOT_TRACE_SAMPLING

Affected product area (please put an X in all that apply)

[ ] Ambient
[ ] Docs
[ ] Dual Stack
[ ] Installation
[ ] Networking
[ ] Performance and Scalability
[x] Extensions and Telemetry
[ ] Security
[ ] Test and Release
[ ] User Experience
[ ] Developer Infrastructure

Affected features (please put an X in all that apply)

[ ] Multi Cluster
[ ] Virtual Machine
[ ] Multi Control Plane

Additional context

CC @zirain @howardjohn