Skip to content
This repository was archived by the owner on Oct 3, 2023. It is now read-only.

update Tag definition with TagMetadata.#233

Merged
rghetia merged 9 commits intocensus-instrumentation:masterfrom
rghetia:tag_ttl
Mar 8, 2019
Merged

update Tag definition with TagMetadata.#233
rghetia merged 9 commits intocensus-instrumentation:masterfrom
rghetia:tag_ttl

Conversation

@rghetia
Copy link
Copy Markdown
Contributor

@rghetia rghetia commented Feb 13, 2019

No description provided.

tags/TagMap.md Outdated
- A receiver MUST decrement the value of `TagTTL` by one if it is greater than zero.
- A receiver MUST discard the `Tag` if the `TagTTL` value is zero.
- A receiver MUST not change the value of `TagTLL` if it is -1.
- A sender MUST propagates a tag if its `TagTTL` value is not zero.
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

s:/propagates a tag/propagate the tag/

tags/TagMap.md Outdated

- A receiver MUST decrement the value of `TagTTL` by one if it is greater than zero.
- A receiver MUST discard the `Tag` if the `TagTTL` value is zero.
- A receiver MUST not change the value of `TagTLL` if it is -1.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this is true. Receiver can overwrite any TagTTL value locally and propagation should respect the local value.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The above was meant for when tag is received. Later in conflict resolution it does say local tags can override received tags or other local tags created earlier. I agree that it is misleading.

How about I rephrase the above to

"
Upon receiving tag from remote entity a receiver

  • MUST decrement the value of TagTTL by one if it is greater than zero.
  • MUST discard the Tag if the TagTTL value is zero.
  • MUST not change the value of TagTLL if it is -1.
    "
    Does that sound ok?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about replacing "receiver" with "propagator"?

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"propagator" could be a proxy in the middle, no? Proxy probably should not affect TTL.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm referring to the component that handles the encoding/decoding of TTL info. Agree that "propagator" may not be the best term here. How about "tag serializer"?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@songy23 'propagator' sounds good.

@yurishkuro are you referring to a http-proxy or a proxy that may simply extract tags/tracecontext and inject into same or another format? If it is later I can add something like "If propagator is acting as a proxy then it MUST leave TagTTL unchanged".

See `TagPropagationFilter` in [Tag Propagation](#Tag Propagation). Tag with `TagTTL` value of -1
is used to represent a request, processing of which may span multiple entities.

## Tag Conflict Resolution
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is a little misleading - as we've discussed, the latest tag defined takes precedence, which is consistent whether the tag is received externally or defined internally.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let me rephrase it to
"If a new tag conflicts with an existing tag then the new tag takes precedence. Entire Tag along with TagValue and TagMetadata is overwritten with a newly generated tag."

what about the sequence where I have already created a tag foo and then I receive tag from remote? Does remote tag takes precedence?

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, the rephrasing works better.

Remote vs local doesn't matter - whenever a tag is put into scope, it shadows any previous tag (with the same tag key).

@songy23
Copy link
Copy Markdown
Contributor

songy23 commented Feb 13, 2019

tags/TagMap.md Outdated
generated tag.
If a new tag conflicts with an existing tag then the new tag takes precedence. Entire `Tag` along
with `TagValue` and `TagMetadata` is overwritten with a most recent tag (either it is locally
generated or received fromi a remote peer).
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: s/fromi/from

tags/TagMap.md Outdated

`TagScope` is used to determine the scope of a `Tag`. The values for the `TagScope` are
Local or Request. In future, additional values can be added to address specific situations.
`TagMetadata` contains properties associated with a `Tag`. For now the only property `TagTTL`
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: 'only the property'

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed.

tags/TagMap.md Outdated
`TagTTL` is an integer that represents number of hops a tag can propagate. Anytime a sender serializes a tag
, sends it over the wire and receiver unserializes the tag then the tag is considered to have travelled one hop.
There could be one or more proxy(ies) between sender and receiver. Proxies are treated as transparent
entity and they do not create additional hops.
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: entities

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed.

tags/TagMap.md Outdated
Upon receiving tag from remote entity a tag extractor

- MUST decrement the value of `TagTTL` by one if it is greater than zero.
- MUST discard the `Tag` if the `TagTTL` value is zero.
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this be an error case?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes. I have added a line to say that "this is an error condition".

tags/TagMap.md Outdated
Upon sending tag to remote entity a tag extractor
- MUST send `TagTTL` ONLY if its value is greater than 0.

Absence of TagTTL on the wire is treated as having TagTTL of -1. This is for backward compatibility.
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the backward compatibility comment necessary? We could just as easily say 'this is to optimize the on-the-wire representation of the common case'.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sure.

tags/TagMap.md Outdated

## Tag Conflict Resolution
If a new tag conflicts with an existing tag then the new tag takes precedence. Entire `Tag` along
with `TagValue` and `TagMetadata` is overwritten with a most recent tag (either it is locally
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think 'shadowed by' or similarly is better than 'overwritten with', which implies an update and obscures the fact that the original tag will be visible again once the new tag goes out of scope.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

made it more clearer with an example.

tags/TagMap.md Outdated
If a new tag conflicts with an existing tag then the new tag takes precedence. Entire `Tag` along
with `TagValue` and `TagMetadata` is overwritten with a most recent tag (either it is locally
generated or received fromi a remote peer).
with `TagValue` and `TagMetadata` is replaced by the most recent tag (either it is locally
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: I would replace 'either' with 'whether' or 'regardless of'

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed.

@rghetia
Copy link
Copy Markdown
Contributor Author

rghetia commented Feb 15, 2019

@bogdandrutu PTAL.

tags/TagMap.md Outdated
- MUST not change the value of `TagTLL` if it is -1.

Upon sending tag to remote entity a tag extractor
- MUST send `TagTTL` ONLY if its value is greater than 0.
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This contradicts -1 being unlimited propagation

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if the value of TagTTL is -1 then the tag is sent but TagTTL is not. The absence of TagTTL is treated as -1.
I'll rephrase it to make it clear.

I just noticed that line 46 should be 'tag injector'. I'll fix that as well.

tags/TagMap.md Outdated
If a new tag conflicts with an existing tag then the new tag takes precedence. Entire `Tag` along
with `TagValue` and `TagMetadata` is replaced by the most recent tag (regardless of it is locally
generated or received from a remote peer). Replacement is limited to a scoped span in which the
conflict arises. When the scoped span is closed the orignal value prior to the conflict is restored.
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was under the impression that context propagation was decoupled from tracing spans.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is decoupled but conflict about Tag could arise regardless of whether you propagate or not propagate trace context.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed with Yuri, we need to remove the reference to the Span and just use "Scope"

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

changed it scope.

tags/TagMap.md Outdated
### TagTTL

`TagTTL` is an integer that represents number of hops a tag can propagate. Anytime a sender serializes a tag
, sends it over the wire and receiver unserializes the tag then the tag is considered to have travelled one hop.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should the first comma be tied to tag?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, fixed.

tags/TagMap.md Outdated
## Tag Conflict Resolution
If a new tag conflicts with an existing tag then the new tag takes precedence. Entire `Tag` along
with `TagValue` and `TagMetadata` is replaced by the most recent tag (regardless of it is locally
generated or received from a remote peer). Replacement is limited to a scoped span in which the
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

s/scoped span/scope

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed.

tags/TagMap.md Outdated
If a new tag conflicts with an existing tag then the new tag takes precedence. Entire `Tag` along
with `TagValue` and `TagMetadata` is replaced by the most recent tag (regardless of it is locally
generated or received from a remote peer). Replacement is limited to a scoped span in which the
conflict arises. When the scoped span is closed the orignal value prior to the conflict is restored.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed with Yuri, we need to remove the reference to the Span and just use "Scope"

Copy link
Copy Markdown
Contributor

@bogdandrutu bogdandrutu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@yurishkuro do you have any comments on this?

@rghetia
Copy link
Copy Markdown
Contributor Author

rghetia commented Feb 22, 2019

@yurishkuro PTAL.

@yurishkuro
Copy link
Copy Markdown

it looks ok. My main concern is that TagTTL=0 is meant for in-process-only propagation, but it does not really address all the needs of that use case, specifically propagating arbitrary objects, of which a tag with string value is just a special case. The specification already has a section about Context, which fits the larger use case I am talking about. Given that, is TagTTL=0 actually needed? And if not, what other uses of TagTTL besides unlimited propagation?

@rghetia rghetia added the tags label Feb 24, 2019
@bogdandrutu
Copy link
Copy Markdown
Contributor

@yurishkuro so couple of difference for what others call baggages. Tags for OC are properties associated with the current request (and we try hard to not have any business logic in these), we currently use tags with requests stats (metrics), can be associated with logs, or Spans. One example of a tag that is TTL=0 is rpc_server_method (or http_server_method), this is valid only on the current process and all signals recorded within this request can be attributed to that server method.

For the business logic things like "auth" for example we recommend people to use directly the context.

@yurishkuro
Copy link
Copy Markdown

Yes I know. But taking a position that "tag == dimension on metrics" is dangerous, especially when tags can be sent upstream: my service's metrics should not be getting new dimensions just because someone from upstream added a baggage item. So there needs to be a mechanism for service owner to whitelist the tags that are allowed as metrics labels. If such mechanism exists, then TTL=0 kind of becomes unnecessary, since the whitelist can just as well take the tag value from the context directly.

Basically, I think I really dislike the TTL notion. For TTL=0, one can store the key/value in the context and not bother with "tags". TTL=1 is dubious. In fact, any TTL != infinity is semantically very vague, because it's difficult to define the exact boundary where TTL -= 1 is supposed to happen. For example, if I am using a service mesh for outbound traffic, does the transfer from my application to service mesh count as the serialization barrier (and therefore TTL -= 1)? Note that the mesh still produces exit spans on behalf of my service.

@rghetia
Copy link
Copy Markdown
Contributor Author

rghetia commented Feb 25, 2019

Prior to TTL change there was a concept of Local and Remote scope, essentially TTL=0 and TTL= -1 (infinity). TTL gives ability to drop a tag in a different way than a TagFilter. TTL allows control from the originator as oppose to TagFilter. For example, if service A connects to service B, C and D then it would have to configure exclude TagFilter on B,C, and D on outbound. With TTL, service A can set TTL=1 and it will get dropped on B,C and D. It doesn't need any configuration on B, C and D.
In the absence of TTL one can always use TagFilter to achieve the same result. The question is do we need another way to drop the tag?

@yurishkuro
Copy link
Copy Markdown

In the absence of TTL one can always use TagFilter to achieve the same result. The question is do we need another way to drop the tag?

I'd suggest not adding another way. A filter sounds like a much better approach, explicit and unambiguous, where's TTL is quite vague as to where TTL -= 1 is supposed to happen.

@songy23
Copy link
Copy Markdown
Contributor

songy23 commented Feb 25, 2019

In the absence of TTL one can always use TagFilter to achieve the same result.
I'd suggest not adding another way. A filter sounds like a much better approach, explicit and unambiguous

IMO there are scenarios where Tag TTL is more useful, such as propagating down a certain number of hops. Plus it would require global metadata for each tag for downstream services to do proper filtering, which may be difficult in open source world.

@yurishkuro
Copy link
Copy Markdown

scenarios where Tag TTL is more useful, such as propagating down a certain number of hops.

do you have concrete use cases in mind? It sounds like an anti-pattern as it assumes whoever sets TTL knows about the organization of the lower levels, breaking the abstraction principle.

of which may span multiple entities.
## TagMetadata

`TagMetadata` contains properties associated with a `Tag`. For now only the property `TagTTL`
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we call it simply TTL? it is a key of TagMetadata already.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is very language specific and implementation specific: if you make it a inner class of the TagMetadata then your proposal makes a lot of sense, or in Go where you use objects with namespace like tag.TTL.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps this can be mentioned as a note. On a wire I'd expect it will be passed as simply TTL w3c/baggage#4

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll make a note of it.

@rghetia
Copy link
Copy Markdown
Contributor Author

rghetia commented Feb 28, 2019

How do we move forward with this? Would it help to have a conference call?

@bogdandrutu
Copy link
Copy Markdown
Contributor

Talk a bit with @yurishkuro offline about this:

  1. Need to make sure in specs that for the moment we ONLY support TTL(-1) and TTL(0).
  2. Every propagation plugin should have an option "decrementTTL" (default true) that allows proxy implementations to set that to false.
  3. One possible example for TTL(1) is a "caller" tag, in general on the server side we don't have information who called me (besides ip/port), but in every process we can have a "service_name" tag that is added as a "caller" tag before serialization when we make any RPC/HTTP calls.

Are there any open issues on this?

@rghetia
Copy link
Copy Markdown
Contributor Author

rghetia commented Mar 5, 2019

Talk a bit with @yurishkuro offline about this:

  1. Need to make sure in specs that for the moment we ONLY support TTL(-1) and TTL(0).
  2. Every propagation plugin should have an option "decrementTTL" (default true) that allows proxy implementations to set that to false.
  3. One possible example for TTL(1) is a "caller" tag, in general on the server side we don't have information who called me (besides ip/port), but in every process we can have a "service_name" tag that is added as a "caller" tag before serialization when we make any RPC/HTTP calls.

Are there any open issues on this?

I'll make the change. There are no other open issues.

@tsloughter
Copy link
Copy Markdown
Member

Why leave out TTL(1)? I think it makes as much sense as -1 and 0, I assume confusion starts as you get to TTL(>1), which I too am not sure of the use case for, so I am glad to see it is being restricted for a starting point.

@bogdandrutu
Copy link
Copy Markdown
Contributor

@tsloughter Because that implies we need to modify our current binary/http format to serialize this information. I am not saying is not useful but we would prefer to add that in the next stage. Does it make sense? I think we can do that later, happy to start a discussion about that but we want to make progress for the moment for the local/global scopes.

@tsloughter
Copy link
Copy Markdown
Member

@bogdandrutu ah ok, so in order to keep the same format as the "scope" form? Makes sense.

@bogdandrutu
Copy link
Copy Markdown
Contributor

@tsloughter correct, but we rename that to a more generic name that will allow us in the future to open this for a more generic purpose without having to do deprecation etc.

tags/TagMap.md Outdated
`TagTTL` is an integer that represents number of hops a tag can propagate. Anytime a sender serializes a tag,
sends it over the wire and receiver unserializes the tag then the tag is considered to have travelled one hop.
There could be one or more proxy(ies) between sender and receiver. Proxies are treated as transparent
entities and they do not create additional hops. Every propagation implementation shoud support option
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is a strong statement, we should say that they may not create additional hops.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok.

Note that TagTTL value of 1 is not supported at this time. The example is listed here simply to
show a possible use case for TagTTL > 0.

### Processing at Receiver and Sender
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure I understand this. Do you suggest that we implement the functionality in the library already? Maybe not a bad idea but we commit on maybe too much for the first phase.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did not mean to implement this right now. However wanted to lay out how TagTTL should be processed when implemented. IMO it is necessary to describe this to make the concept of TagTTL complete. I can make it more clearer that implementation of this processing is not required as there will be nothing on the wire.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I take back what I said. Some processing is required on sender and receiver. For example,
On sender,

  • MUST send the tag without 'TagTTL' if its value is -1. Absence of TagTTL on the wire is treated as having TagTTL of -1.
  • MUST not send the tag if the value of TagTTL is 0.

On Receiver,

  • MUST treat the value of TagTTL as -1 if is not present.

The processing has to be part of a tag propagator and not the library. This allows someone not to configure propagator at all if they don't want anything to propagate.

tags/TagMap.md Outdated
Add Tags T3=V3, T2=v4
Current Tags T1=V1, T2=V4, T3=V3 <== Value of T2 is replaced by V4.
Close Scope 2
Current Tags T1=V1, T2=V2 <== T2 is restored.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe point that metadata is also restored.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sure.

tags/TagMap.md Outdated

#### At Sender
Upon preparing to send a tag to a remote entity a tag injector
- MUST send the tag AND include `TagTTL` if its value is greater than 0.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't have defined a tag-metadata in our binary format, should we consider to not send this for the moment, or send it as TTL -1 in this phase?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Again this for future. Today nothing goes on wire. When we implement TagTTL>0 then we would need to define tag-metadata for binary format which will include TagTTL.

@rghetia
Copy link
Copy Markdown
Contributor Author

rghetia commented Mar 8, 2019

@bogdandrutu can I merge this now?

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants