attribute: add SLICE type support#8166
Conversation
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #8166 +/- ##
=======================================
+ Coverage 82.3% 82.5% +0.1%
=======================================
Files 310 310
Lines 24267 24555 +288
=======================================
+ Hits 19990 20261 +271
- Misses 3901 3916 +15
- Partials 376 378 +2
🚀 New features to boost your workflow:
|
MrAlias
left a comment
There was a problem hiding this comment.
Overall, looks good.
I assume we're going to address the attribute limits and exporter support in a follow-up, right?
Correct: |
Replace reflect.ValueOf + reflection-based iteration with direct typed slice access (asBoolSlice, asInt64Slice, asFloat64Slice, asStringSlice) for BOOLSLICE, INT64SLICE, FLOAT64SLICE, and STRINGSLICE in hashKV. This avoids the per-element reflection overhead — the same improvement applied to the SLICE type in open-telemetry#8166. Fixes open-telemetry#8200 Signed-off-by: alliasgher <alliasgher123@gmail.com>
|
I am going to create a separate PR for improving the speed execution of EDIT: My idea is to change the test so that it only checks if |
Unblocks #8166 Per #8166 (comment) ## Why This updates `TestHashKVsEquality` to avoid the quadratic pairwise comparison that was causing slow runs and timeouts in CI. Below are the results from my machine. Old: ``` $ go test -run=TestHashKVs -count=1 PASS ok go.opentelemetry.io/otel/attribute 2.063s ``` New: ``` $ go test -run=TestHashKVs -count=1 PASS ok go.opentelemetry.io/otel/attribute 0.024s ``` Instead of collecting every generated testcase and comparing each hash against every other hash, the test now checks uniqueness as each case is generated by storing previously seen hashes in a map. Note that it also does not use the equality operator of `KeyValue` to determine if the hash should be equal or different. ## What - Rename the test from `TestHashKVsEquality` to `TestHashKVs` - Replace the O(n^2) post-processing loop with an O(n) streaming uniqueness check - Simplify failure reporting by removing the dedicated `msg` helper Side note: for me it also makes the test more readable (however, this is opinionated).
There was a problem hiding this comment.
Pull request overview
Adds first-class support for heterogeneous SLICE attribute values (arrays of attribute.Value) across the attribute package and downstream conversions, aligning with the OTel AnyValue array concept.
Changes:
- Introduce
attribute.SLICEplusSlice/SliceValueconstructors andValue.AsSlice()/string formatting support. - Propagate
SLICEhandling into trace auto-conversion, log attribute conversion, hashing, and metric test comparisons. - Add/extend unit tests, fuzz/bench coverage, and update changelog.
Reviewed changes
Copilot reviewed 16 out of 16 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| trace/auto.go | Convert attribute.SLICE into telemetry nested slice values. |
| sdk/metric/metricdata/metricdatatest/comparisons.go | Extend attribute equality helper to handle SLICE. |
| log/keyvalue.go | Convert attribute.SLICE into log SliceValue recursively. |
| log/keyvalue_test.go | Add test coverage for SLICE conversion to log values. |
| log/keyvalue_bench_test.go | Add benchmark case for SLICE conversion. |
| attribute/value.go | Add SLICE type, constructors/accessors, formatting, and base64 writer helper. |
| attribute/value_test.go | Add tests for SLICE value behavior, AsSlice, and string formatting. |
| attribute/type_string.go | Regenerate Type.String() backing tables to include SLICE. |
| attribute/kv.go | Add attribute.Slice convenience constructor. |
| attribute/key.go | Add Key.Slice constructor. |
| attribute/kv_test.go | Add constructor/validity/incorrect-cast tests for SLICE. |
| attribute/key_test.go | Add Emit/String tests for SLICE formatting behavior. |
| attribute/hash.go | Add hashing support for SLICE values (including nested). |
| attribute/hash_test.go | Extend hash test generators + add SLICE fuzz/bench coverage. |
| attribute/benchmark_test.go | Add slice benchmarks for construction/access/string/emit. |
| CHANGELOG.md | Document new SLICE attribute type API additions. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Unblocks open-telemetry#8166 Per open-telemetry#8166 (comment) ## Why This updates `TestHashKVsEquality` to avoid the quadratic pairwise comparison that was causing slow runs and timeouts in CI. Below are the results from my machine. Old: ``` $ go test -run=TestHashKVs -count=1 PASS ok go.opentelemetry.io/otel/attribute 2.063s ``` New: ``` $ go test -run=TestHashKVs -count=1 PASS ok go.opentelemetry.io/otel/attribute 0.024s ``` Instead of collecting every generated testcase and comparing each hash against every other hash, the test now checks uniqueness as each case is generated by storing previously seen hashes in a map. Note that it also does not use the equality operator of `KeyValue` to determine if the hash should be equal or different. ## What - Rename the test from `TestHashKVsEquality` to `TestHashKVs` - Replace the O(n^2) post-processing loop with an O(n) streaming uniqueness check - Simplify failure reporting by removing the dedicated `msg` helper Side note: for me it also makes the test more readable (however, this is opinionated).
Fixes open-telemetry#7934 ``` $ go test -run=^$ -bench=BenchmarkSlice goos: linux goarch: amd64 pkg: go.opentelemetry.io/otel/attribute cpu: 13th Gen Intel(R) Core(TM) i7-13800H BenchmarkSlice/Len3/Value-20 25297926 52.56 ns/op 144 B/op 1 allocs/op BenchmarkSlice/Len3/KeyValue-20 21315132 55.97 ns/op 144 B/op 1 allocs/op BenchmarkSlice/Len3/AsSlice-20 24214248 50.03 ns/op 144 B/op 1 allocs/op BenchmarkSlice/Len3/String-20 14148270 86.48 ns/op 48 B/op 1 allocs/op BenchmarkSlice/Len3/Emit-20 13605388 85.18 ns/op 48 B/op 1 allocs/op BenchmarkSlice/Len5Nested/Value-20 16086171 71.30 ns/op 240 B/op 1 allocs/op BenchmarkSlice/Len5Nested/KeyValue-20 15547844 75.81 ns/op 240 B/op 1 allocs/op BenchmarkSlice/Len5Nested/AsSlice-20 17806996 66.16 ns/op 240 B/op 1 allocs/op BenchmarkSlice/Len5Nested/String-20 7409064 165.2 ns/op 64 B/op 1 allocs/op BenchmarkSlice/Len5Nested/Emit-20 7666302 161.0 ns/op 64 B/op 1 allocs/op PASS ok go.opentelemetry.io/otel/attribute 12.980s ``` ``` $ go test -run=^$ -bench=BenchmarkHashKVs goos: linux goarch: amd64 pkg: go.opentelemetry.io/otel/attribute cpu: 13th Gen Intel(R) Core(TM) i7-13800H BenchmarkHashKVs-20 1268742 940.5 ns/op 0 B/op 0 allocs/op PASS ok go.opentelemetry.io/otel/attribute 1.198s ```
### Added - Add `ByteSlice` and `ByteSliceValue` functions for new `BYTESLICE` attribute type in `go.opentelemetry.io/otel/attribute`. (#7948) - Apply attribute value limit to the `KindBytes` attribute type in `go.opentelemetry.io/otel/sdk/log`. (#7990) - Apply attribute value limit to the `BYTESLICE` attribute type in `go.opentelemetry.io/otel/sdk/trace`. (#7990) - Support `BYTESLICE` attributes in `go.opentelemetry.io/otel/trace`. (#8153) - Support `BYTESLICE` attributes in `go.opentelemetry.io/otel/exporters/otlp/otlptrace`. (#8153) - Support `BYTESLICE` attributes in `go.opentelemetry.io/otel/exporters/otlp/otlplog`. (#8153) - Support `BYTESLICE` attributes in `go.opentelemetry.io/otel/exporters/otlp/otlpmetric`. (#8153) - Support `BYTESLICE` attributes in `go.opentelemetry.io/otel/exporters/zipkin`. (#8153) - Add `String` method for `Value` type in `go.opentelemetry.io/otel/attribute`. (#8142) - Add `Slice` and `SliceValue` functions for new `SLICE` attribute type in `go.opentelemetry.io/otel/attribute`. (#8166) - Support `SLICE` attributes in `go.opentelemetry.io/otel/exporters/otlp/otlptrace`. (#8216) - Support `SLICE` attributes in `go.opentelemetry.io/otel/exporters/otlp/otlplog`. (#8216) - Support `SLICE` attributes in `go.opentelemetry.io/otel/exporters/otlp/otlpmetric`. (#8216) - Support `SLICE` attributes in `go.opentelemetry.io/otel/exporters/zipkin`. (#8216) - Apply `AttributeValueLengthLimit` to `attribute.SLICE` type attribute values in `go.opentelemetry.io/otel/sdk/trace`, recursively truncating contained string values. (#8217) - Add `Error` field on `Record` type in `go.opentelemetry.io/otel/log/logtest`. (#8148) - Add `WithMaxRequestSize` option in `go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc`. (#8157) - Add `WithMaxRequestSize` option in `go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp`. (#8157) - Add `WithMaxRequestSize` option in `go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetricgrpc`. (#8157) - Add `WithMaxRequestSize` option in `go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetrichttp`. (#8157) - Add `WithMaxRequestSize` option in `go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploggrpc`. (#8157) - Add `WithMaxRequestSize` option in `go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploghttp`. (#8157) - Add `Settable` to `go.opentelemetry.io/otel/metric/x` to allow reusing attribute options. (#8178) - Add experimental support for splitting metric data across multiple batches in `go.opentelemetry.io/otel/sdk/metric`. Set `OTEL_GO_X_METRIC_EXPORT_BATCH_SIZE=<max_size>` to enable for all periodic readers. See `go.opentelemetry.io/otel/sdk/metric/internal/x` for feature documentation. (#8071) - Add experimental self-observability metrics in `go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetricgrpc`. Enable with `OTEL_GO_X_SELF_OBSERVABILITY=true` environment variable. See `go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetricgrpc/internal/x` for feature documentation. (#8192) - Add experimental self-observability metrics in `go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetrichttp`. Enable with `OTEL_GO_X_SELF_OBSERVABILITY=true` environment variable. See `go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetrichttp/internal/x` for feature documentation. (#8194) - Add experimental self-observability metrics in `go.opentelemetry.io/otel/exporters/stdout/stdoutlog`. Enable with `OTEL_GO_X_SELF_OBSERVABILITY=true` environment variable. See `go.opentelemetry.io/otel/stdout/stdoutlog/internal/x` for feature documentation. (#8263) - Add `WithDefaultAttributes` to `go.opentelemetry.io/otel/metric/x` to support setting default attributes on instruments. (#8135) - Add `go.opentelemetry.io/otel/semconv/v1.41.0` package. The package contains semantic conventions from the `v1.41.0` version of the OpenTelemetry Semantic Conventions. See the [migration documentation](./semconv/v1.41.0/MIGRATION.md) for information on how to upgrade from `go.opentelemetry.io/otel/semconv/v1.40.0`. (#8324) - Add Observable variants of instruments to `go.opentelemetry.io/otel/semconv/v1.41.0` package. (#8350) - Generate explicit histogram bucket boundaries from weaver configuration for HTTP and RPC duration instruments in `go.opentelemetry.io/otel/semconv/v1.41.0`. (#8002) ### Changed -⚠️ **Breaking Change:** `go.opentelemetry.io/otel/sdk/metric` now applies a default cardinality limit of 2000 to comply with the Metrics SDK specification recommendation. New attribute sets are dropped when the cardinality limit is reached. The measurement of these sets are aggregated into a special attribute set containing `attribute.Bool("otel.metric.overflow", true)`. This can break users who relied on the previous unlimited default. Set `WithCardinalityLimit(0)` or the deprecated `OTEL_GO_X_CARDINALITY_LIMIT=0` environment variable to preserve unlimited cardinality. Note that support for `OTEL_GO_X_CARDINALITY_LIMIT` may be removed in a future release. (#8247) - `ErrorType` in `go.opentelemetry.io/otel/semconv` now unwraps errors created with `fmt.Errorf` when deriving the `error.type` attribute. (#8133) - `go.opentelemetry.io/otel/sdk/log` now unwraps error chains created with `fmt.Errorf` when deriving the `error.type` attribute from errors on log records. (#8133) - `Set.MarshalLog` method in `go.opentelemetry.io/otel/attribute` now uses `Value.String` formatting following the [OpenTelemetry AnyValue representation for non-OTLP protocols](https://opentelemetry.io/docs/specs/otel/common/#anyvalue). (#8169) - Optimize `go.opentelemetry.io/otel/sdk/metric` to return a drop reservoir and short-circuit `Offer` calls to the exemplar reservoir when `exemplar.AlwaysOffFilter` is configured. (#8211) (#8267) - Optimize `go.opentelemetry.io/otel/sdk/metric` to return a drop reservoir for asynchronous instruments when `exemplar.TraceBasedFilter` is configured. (#8286) ### Deprecated - Deprecate `Value.Emit` method in `go.opentelemetry.io/otel/attribute`. Use `Value.String` instead. (#8176) ### Fixed - Limit OTLP request size to 64 MiB by default in `go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc`. The limit applies before compression, oversized requests are treated as non-retryable errors, and the limit can be configured with the new `WithMaxRequestSize` option. (#8157, #8365) - Limit OTLP request size to 64 MiB by default in `go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp`. The limit applies before compression, oversized requests are treated as non-retryable errors, and the limit can be configured with the new `WithMaxRequestSize` option. (#8157, #8365) - Limit OTLP request size to 64 MiB by default in `go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetricgrpc`. The limit applies before compression, oversized requests are treated as non-retryable errors, and the limit can be configured with the new `WithMaxRequestSize` option. (#8157, #8365) - Limit OTLP request size to 64 MiB by default in `go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetrichttp`. The limit applies before compression, oversized requests are treated as non-retryable errors, and the limit can be configured with the new `WithMaxRequestSize` option. (#8157, #8365) - Limit OTLP request size to 64 MiB by default in `go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploggrpc`. The limit applies before compression, oversized requests are treated as non-retryable errors, and the limit can be configured with the new `WithMaxRequestSize` option. (#8157, #8365) - Limit OTLP request size to 64 MiB by default in `go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploghttp`. The limit applies before compression, oversized requests are treated as non-retryable errors, and the limit can be configured with the new `WithMaxRequestSize` option. (#8157, #8365) - Fix gzipped request body replay on redirect in `go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetrichttp`. (#8135) - Fix gzipped request body replay on redirect in `go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploghttp`. (#8152) - `go.opentelemetry.io/otel/exporters/prometheus` now uses `Value.String` formatting for label values following the [OpenTelemetry AnyValue representation for non-OTLP protocols](https://opentelemetry.io/docs/specs/otel/common/#anyvalue). (#8170) - Propagate errors from the exporter when calling `Shutdown` on `BatchSpanProcessor` in `go.opentelemetry.io/otel/sdk/trace`. (#8197) - Fix stale status code reporting on self-observability metrics in `go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp` and `go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploghttp`. (#8226) - Fix a concurrent `Collect` data race and potential panic in `go.opentelemetry.io/otel/exporters/prometheus` when `WithResourceAsConstantLabels` option is used. (#8227) - Fix race condition in `FixedSizeReservoir` in `go.opentelemetry.io/otel/sdk/metric/exemplar` by reverting #7447. (#8249) - Fix `FixedSizeReservoir` in `go.opentelemetry.io/otel/sdk/metric/exemplar` to safely handle zero size. A capacity check in the constructor initializes the reservoir safely and skips initialization for zero-cap; early returns in `Offer()` and `Collect()` ensure no-op behavior. (#8295) - Fix counting of spans and logs in self-observability metrics in `go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc`, `go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp`, `go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploggrpc`, and `go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploghttp`. (#8254) - Drop conflicting scope attributes named `name`, `version`, or `schema_url` from metric labels in `go.opentelemetry.io/otel/exporters/prometheus`, preserving the dedicated `otel_scope_name`, `otel_scope_version`, and `otel_scope_schema_url` labels. (#8264) - Close schema files opened by `ParseFile` in `go.opentelemetry.io/otel/schema/v1.0` and `go.opentelemetry.io/otel/schema/v1.1`. ([GHSA-995v-fvrw-c78m](GHSA-995v-fvrw-c78m)) - Enforce the 8192-byte baggage size limit during extraction/parsing, changing behavior when the limit is exceeded in `go.opentelemetry.io/otel/baggage` and `go.opentelemetry.io/otel/propagation`. (#8222) - Fix `go.opentelemetry.io/otel/semconv/v1.41.0` to include `Attr*` helper methods for required attributes on observable instruments. (#8361) - Limit baggage extraction error reporting in `go.opentelemetry.io/otel/propagation` to prevent malformed or oversized baggage headers from flooding logs. ([GHSA-5wrp-cwcj-q835](GHSA-5wrp-cwcj-q835))
Fixes #7934