Skip to content
This repository was archived by the owner on Jul 31, 2023. It is now read-only.
This repository was archived by the owner on Jul 31, 2023. It is now read-only.

Including 0 bound when creating distribution leads to mismatch in bucket count between view and rows of actual data. #1169

@newrelic-eheinlein

Description

@newrelic-eheinlein

Please answer these questions before submitting a bug report.

What version of OpenCensus are you using?

0.23.0

What version of Go are you using?

1.13

What did you do?

Created a custom exporter that handles views of type Distribution; when creating the distribution I included 0 as a bound.

What did you expect to see?

The number of buckets in the view and the number of buckets in the row of actual data match up

What did you see instead?

The view and the row data do not match up:
len(vd.View.Aggregation.Buckets) = 10
and len(vd.Rows[0].Data.CountPerBucket) = 12

Additional context

Wall of text for you!

My team is working on a custom exporter for metrics. I'm encountering an issue specifically with Views of type Distribution. What I am encountering is that if I create a view of type Distribution where the bounds includes 0, when that view gets registered the 0 bound is silently dropped. However, when recording the data, that bucket is NOT dropped, leading to a mis-match in the view vs row that arrives in the view.Data that is passed to my ExportView function.

I'll post all of the details of my exploration below, but the main questions I have are:

  1. is this a bug? or
  2. if this is expected behavior, can it be clearly documented that you should NOT use 0 bounds for Distributions?
  • The example here shows Distributions including a 0 bound:
Aggregation: view.Distribution(0, 25, 50, 75, 100, 200, 400, 600, 800, 1000, 2000, 4000, 6000),

Here are the detailed steps of what I am finding.

My main program is creating Distribution View:

var (
	values = stats.Int64("balls", "measure of balls", "balls")

	distributionView = &view.View{
		Measure:     values,
		Name:        "distribution",
		Description: "a distribution of the values",
		Aggregation: view.Distribution(0, 10, 20, 30, 40, 50, 60, 70, 80, 90, 100),
	}
)

When view.Distribution(...) is called, it is ending up in the code here: https://github.com/census-instrumentation/opencensus-go/blob/master/stats/view/aggregation.go#L106

	return &Aggregation{
		Type:    AggTypeDistribution,
		Buckets: bounds,
		newData: func() AggregationData {
			return newDistributionData(bounds)
		},
	}

When that newData function is assinged, the value of len(bounds) is 11, as one would expect.

Next in my main program, I register the view:

	if err := view.Register(distributionView); err != nil {
		panic(err)
	}

In this step, when view.Register(...) is called, a registerViewReq is created. When it is handled, the view has canonicalize() called on it, which eventually gets to this line: https://github.com/census-instrumentation/opencensus-go/blob/master/stats/view/view.go#L104

	// drop 0 bucket silently.
	v.Aggregation.Buckets = dropZeroBounds(v.Aggregation.Buckets...)

After this code executes, the length of v.Aggregation.Buckets is now 10, instead of 11.

Okay, now everything is registered and ready to go; next we record our metric:

	stats.Record(context.Background(), values.M(rand.Int63n(100)))

Following this down the rabbit hole, we end up in collector.go, specifically here: https://github.com/census-instrumentation/opencensus-go/blob/master/stats/view/collector.go#L38

	aggregator, ok := c.signatures[s]
	if !ok {
		aggregator = c.a.newData()
		c.signatures[s] = aggregator
	}

The first time through, c.a.newData() gets called, which returns the function that was defined up above - when len(bounds) was equal to 11.

Stepping into newDistributionData() https://github.com/census-instrumentation/opencensus-go/blob/master/stats/view/aggregation_data.go#L132

func newDistributionData(bounds []float64) *DistributionData {
	bucketCount := len(bounds) + 1
	return &DistributionData{
	...

Here, the bucket count gets set to 12 (11+1)

Now, finally, we make it to our ExportView function that we wrote:

func (e *Exporter) ExportView(vd *view.Data) {
	for _, row := range vd.Rows {
		e.processRow(vd, row)
	}
}

At this point,

  • len(vd.View.Aggregation.Buckets) = 10 (11 - 1 because it dropped the 0 bounds)
  • len(row.Data.CountPerBucket) = 12 (11 + 1 as defined in newDistributionData)

Note: as the two things I am comparing are both referring to the "buckets", I would expect them to be equal. I can see how, since vd.View.Aggregation.Buckets is actually referring to the bounds, not the buckets, that in that case it would be expected that it be one less than the number of buckets in row.Data.CountPerBucket, as described here:

For a distribution with N bounds, the associated DistributionData will have N+1 buckets.

But in that case, I feel like that should be referred to as vd.View.Aggregation.Bounds instead of Buckets.

At any rate, even taking into account that this is comparing bounds to buckets, we still have a mismatch - it should be either 11 and 12, if the 0 bound is not dropped for either, or 10 and 11, if the 0 bound is dropped for both.

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions