Fix resolution for dependencies#1280
Conversation
| /// Attribute definitions available in this registry (including those | ||
| /// from dependencies). Used for cross-registry attribute lookup. | ||
| /// Not serialized — populated only for freshly resolved schemas. | ||
| root_attributes: HashMap<String, (Attribute, String)>, |
There was a problem hiding this comment.
this is the real change for v1, let's populate root_attributes directly instead of trying to use lineage to infer if something in the catalog is original definition.
The rest in this file is cleanup - this struct should not allow to mutate attrs, does not need to be (de)serializable, many of its features were only used in tests
| stability: stable | ||
| requirement_level: recommended | ||
| examples: "Ex" | ||
| - id: db.client.metrics |
There was a problem hiding this comment.
this is repro for the v1 issue
There was a problem hiding this comment.
Ah - I gotcha, we don't use extends here, so all the assumptions fail, and the catalog is ambiguous. I thought it'd be safe to remove the hardcoded registry. paths for V1, but looks like not, thanks for catching this.
| "requirement_level": "recommended", | ||
| "note": "Destination name SHOULD uniquely identify a specific queue, topic or other entity within the broker. If\nthe broker doesn't have such notion, the destination name SHOULD uniquely identify the broker.\n", | ||
| "stability": "stable" | ||
| [ |
There was a problem hiding this comment.
no change here except unwrapping list of attrs - this allows to remove serialization from Catalog
There was a problem hiding this comment.
OOF, yeah you can blame me for that change and I regret doing it.
I wonder if maybe serializing this to YAML or TOML or something would make this more readable / easier to diff in the future. WDYT?
| # TODO: this only works with definition registry, but not with | ||
| # resolved one, because resolved does not know how to get | ||
| # attributes from transitive dependencies. | ||
| registry_path: data/registry-test-v2-dep/consumer_registry |
There was a problem hiding this comment.
fun around attrs from transitive dependencies:
- if you use definition as a dependency, you can use attributes in transitive one
- if you use resolved as dependency, you can't - they are not there
Let's make it consistent (regardless of the approach we take)
There was a problem hiding this comment.
also more fun: since only one dependency is allowed, you can't really make
A -> resolved B -> C work today, unless you can modify B to import things
| }) | ||
| }); | ||
| result | ||
| self.catalog |
There was a problem hiding this comment.
this is part of solution for v1 - use root attributes instead of refs, the logic here didn't work for dependencies
| let fake_group_id = format!("v2_dependency.{}", self.schema_url.name()); | ||
| self.attribute_catalog.iter().find_map(|attr| { | ||
| self.registry.attributes.iter().find_map(|attr_ref| { | ||
| let attr = self.attribute_catalog.get(attr_ref.0 as usize)?; |
There was a problem hiding this comment.
this is solution for v2 - attribute_catalog contains definitions and refinements. given lookup by key, we can only return the original definition from registry.attributes
There was a problem hiding this comment.
Doh! Yeah - I can't believe I missed this the first time.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #1280 +/- ##
=======================================
+ Coverage 81.0% 81.1% +0.1%
=======================================
Files 111 111
Lines 9380 9375 -5
=======================================
+ Hits 7598 7606 +8
+ Misses 1782 1769 -13 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
| /// and signals. | ||
| pub catalog: Catalog, | ||
| /// Resource definition (only for application). | ||
| #[serde(skip_serializing_if = "Option::is_none")] |
There was a problem hiding this comment.
yep, it does not need json schema - we don't (and don't need to) support it for v1. And it's not (de)serializable either
There was a problem hiding this comment.
I'm trying to remove serialization from things that are not intended to be read or written by weaver. This helps to understand what's internal detail and what's a real contract
Both v1 and v2 resolution has a problem when using dependencies - it returns random attribute from attribute catalog instead of definition resulting in wrong briefs/notes, etc.
E.g. in this yaml https://github.com/lmolkova/semconv-scale23x-demo/blob/965f9f0a6094e1d21150d4a0441ede7aafbb4a0b/conventions/src/registry.yaml#L36 (
- ref: server.addressfrom semconv), theserver.addressresolves to "Name of the database host." which comes from DB refinement.https://github.com/lmolkova/semconv-scale23x-demo/blob/2d31c5bf2437f5936d8740a338845bcb06ced456/docs/storage/metrics.md#attributes-1
This PR fixes those. It does not solve multi-dependency/from-transitive-dependency resolution.