Please answer these questions before submitting a bug report.
What version of OpenCensus are you using?
0.23.0
What version of Go are you using?
1.13
What did you do?
Created a custom exporter that handles views of type Distribution; when creating the distribution I included 0 as a bound.
What did you expect to see?
The number of buckets in the view and the number of buckets in the row of actual data match up
What did you see instead?
The view and the row data do not match up:
len(vd.View.Aggregation.Buckets) = 10
and len(vd.Rows[0].Data.CountPerBucket) = 12
Additional context
Wall of text for you!
My team is working on a custom exporter for metrics. I'm encountering an issue specifically with Views of type Distribution. What I am encountering is that if I create a view of type Distribution where the bounds includes 0, when that view gets registered the 0 bound is silently dropped. However, when recording the data, that bucket is NOT dropped, leading to a mis-match in the view vs row that arrives in the view.Data that is passed to my ExportView function.
I'll post all of the details of my exploration below, but the main questions I have are:
- is this a bug? or
- if this is expected behavior, can it be clearly documented that you should NOT use 0 bounds for Distributions?
- The example here shows Distributions including a 0 bound:
Aggregation: view.Distribution(0, 25, 50, 75, 100, 200, 400, 600, 800, 1000, 2000, 4000, 6000),
Here are the detailed steps of what I am finding.
My main program is creating Distribution View:
var (
values = stats.Int64("balls", "measure of balls", "balls")
distributionView = &view.View{
Measure: values,
Name: "distribution",
Description: "a distribution of the values",
Aggregation: view.Distribution(0, 10, 20, 30, 40, 50, 60, 70, 80, 90, 100),
}
)
When view.Distribution(...) is called, it is ending up in the code here: https://github.com/census-instrumentation/opencensus-go/blob/master/stats/view/aggregation.go#L106
return &Aggregation{
Type: AggTypeDistribution,
Buckets: bounds,
newData: func() AggregationData {
return newDistributionData(bounds)
},
}
When that newData function is assinged, the value of len(bounds) is 11, as one would expect.
Next in my main program, I register the view:
if err := view.Register(distributionView); err != nil {
panic(err)
}
In this step, when view.Register(...) is called, a registerViewReq is created. When it is handled, the view has canonicalize() called on it, which eventually gets to this line: https://github.com/census-instrumentation/opencensus-go/blob/master/stats/view/view.go#L104
// drop 0 bucket silently.
v.Aggregation.Buckets = dropZeroBounds(v.Aggregation.Buckets...)
After this code executes, the length of v.Aggregation.Buckets is now 10, instead of 11.
Okay, now everything is registered and ready to go; next we record our metric:
stats.Record(context.Background(), values.M(rand.Int63n(100)))
Following this down the rabbit hole, we end up in collector.go, specifically here: https://github.com/census-instrumentation/opencensus-go/blob/master/stats/view/collector.go#L38
aggregator, ok := c.signatures[s]
if !ok {
aggregator = c.a.newData()
c.signatures[s] = aggregator
}
The first time through, c.a.newData() gets called, which returns the function that was defined up above - when len(bounds) was equal to 11.
Stepping into newDistributionData() https://github.com/census-instrumentation/opencensus-go/blob/master/stats/view/aggregation_data.go#L132
func newDistributionData(bounds []float64) *DistributionData {
bucketCount := len(bounds) + 1
return &DistributionData{
...
Here, the bucket count gets set to 12 (11+1)
Now, finally, we make it to our ExportView function that we wrote:
func (e *Exporter) ExportView(vd *view.Data) {
for _, row := range vd.Rows {
e.processRow(vd, row)
}
}
At this point,
len(vd.View.Aggregation.Buckets) = 10 (11 - 1 because it dropped the 0 bounds)
len(row.Data.CountPerBucket) = 12 (11 + 1 as defined in newDistributionData)
Note: as the two things I am comparing are both referring to the "buckets", I would expect them to be equal. I can see how, since vd.View.Aggregation.Buckets is actually referring to the bounds, not the buckets, that in that case it would be expected that it be one less than the number of buckets in row.Data.CountPerBucket, as described here:
For a distribution with N bounds, the associated DistributionData will have N+1 buckets.
But in that case, I feel like that should be referred to as vd.View.Aggregation.Bounds instead of Buckets.
At any rate, even taking into account that this is comparing bounds to buckets, we still have a mismatch - it should be either 11 and 12, if the 0 bound is not dropped for either, or 10 and 11, if the 0 bound is dropped for both.
Please answer these questions before submitting a bug report.
What version of OpenCensus are you using?
0.23.0
What version of Go are you using?
1.13
What did you do?
Created a custom exporter that handles views of type Distribution; when creating the distribution I included 0 as a bound.
What did you expect to see?
The number of buckets in the view and the number of buckets in the row of actual data match up
What did you see instead?
The view and the row data do not match up:
len(vd.View.Aggregation.Buckets)= 10and
len(vd.Rows[0].Data.CountPerBucket)= 12Additional context
Wall of text for you!
My team is working on a custom exporter for metrics. I'm encountering an issue specifically with
Viewsof typeDistribution. What I am encountering is that if I create a view of type Distribution where the bounds includes 0, when that view gets registered the 0 bound is silently dropped. However, when recording the data, that bucket is NOT dropped, leading to a mis-match in the view vs row that arrives in theview.Datathat is passed to myExportViewfunction.I'll post all of the details of my exploration below, but the main questions I have are:
Here are the detailed steps of what I am finding.
My main program is creating Distribution View:
When
view.Distribution(...)is called, it is ending up in the code here: https://github.com/census-instrumentation/opencensus-go/blob/master/stats/view/aggregation.go#L106When that
newDatafunction is assinged, the value oflen(bounds)is 11, as one would expect.Next in my main program, I register the view:
In this step, when
view.Register(...)is called, aregisterViewReqis created. When it is handled, the view hascanonicalize()called on it, which eventually gets to this line: https://github.com/census-instrumentation/opencensus-go/blob/master/stats/view/view.go#L104After this code executes, the length of v.Aggregation.Buckets is now 10, instead of 11.
Okay, now everything is registered and ready to go; next we record our metric:
Following this down the rabbit hole, we end up in
collector.go, specifically here: https://github.com/census-instrumentation/opencensus-go/blob/master/stats/view/collector.go#L38The first time through,
c.a.newData()gets called, which returns the function that was defined up above - whenlen(bounds)was equal to 11.Stepping into
newDistributionData()https://github.com/census-instrumentation/opencensus-go/blob/master/stats/view/aggregation_data.go#L132Here, the bucket count gets set to 12 (11+1)
Now, finally, we make it to our ExportView function that we wrote:
At this point,
len(vd.View.Aggregation.Buckets)= 10 (11 - 1 because it dropped the 0 bounds)len(row.Data.CountPerBucket)= 12 (11 + 1 as defined innewDistributionData)Note: as the two things I am comparing are both referring to the "buckets", I would expect them to be equal. I can see how, since
vd.View.Aggregation.Bucketsis actually referring to the bounds, not the buckets, that in that case it would be expected that it be one less than the number of buckets inrow.Data.CountPerBucket, as described here:But in that case, I feel like that should be referred to as
vd.View.Aggregation.Boundsinstead ofBuckets.At any rate, even taking into account that this is comparing bounds to buckets, we still have a mismatch - it should be either 11 and 12, if the 0 bound is not dropped for either, or 10 and 11, if the 0 bound is dropped for both.