Skip to content

Span service target fields specification#627

Merged
SylvainJuge merged 28 commits intoelastic:mainfrom
SylvainJuge:backend-db-granularity-spec
May 30, 2022
Merged

Span service target fields specification#627
SylvainJuge merged 28 commits intoelastic:mainfrom
SylvainJuge:backend-db-granularity-spec

Conversation

@SylvainJuge
Copy link
Copy Markdown
Member

@SylvainJuge SylvainJuge commented Mar 29, 2022

Implementation checklist

This PR is considered a work-in progress draft until the following items have been addressed:

  • clarification needed: do we need to make the agents compatible with older versions of APM servers ? If yes, then it means that we need to keep agents computing and always sending destination.service.resource.
    UPDATE: we need to keep the resource for older servers + compatibility with UI
  • link from other spec + deprecate it
  • link to APM-Server issues: https://github.com/elastic/apm-dev/issues/690
  • Add OTel compatibility
  • Add Jaeger compatibility Skipped for now, maybe in a follow-up PR.
  • update tracing-instrumentation-db.md
  • update tracing-spans-compress.md
  • update tracing-spans-dropped-stats.md
  • Decide on best strategy to simplify apm-server compatibility mapping: Span service target fields specification #627 (comment)
    UPDATE: only infer on resource field, back-fill new fields on a best-effort.

This is a new spec or a bigger enhancement

  • May the instrumentation collect sensitive information, such as secrets or PII (ex. in headers)?
    • Yes
      • Add a section to the spec how agents should apply sanitization (such as sanitize_field_names)
    • No
      • Why:
    • n/a: only re-organizes existing data into different fields.
  • Link to meta issue: [META 621] Spec: Improve Granularity for SQL Databases #622
  • Create PR as draft
  • Approval by at least one other agent
  • Mark as Ready for Review (automatically requests reviews from all agents and PM via CODEOWNERS)
    • Remove PM from reviewers if impact on product is negligible
    • Remove agents from reviewers if the change is not relevant for them
  • Approved by at least 2 agents + PM (if relevant)
  • Merge after 7 days passed without objections
  • Create implementation issues (ideally add a milestone)
  • Create a status table and add it to the meta issue

Fixes #622


Summary of the impact on product:

  • higher granularity for database backends (and also some messaging and RPC backends), visible in dependencies & map.
  • agents remain compatible with current APM server & UI

@ghost
Copy link
Copy Markdown

ghost commented Mar 29, 2022

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview preview

Expand to view the summary

Build stats

  • Start Time: 2022-05-30T04:13:00.396+0000

  • Duration: 10 min 33 sec

@basepi basepi changed the title WIP first draft spec WIP DB granularity first draft spec Mar 29, 2022
Comment thread specs/agents/tracing-spans-service-target.md Outdated
Comment thread specs/agents/tracing-spans-service-target.md Outdated
Copy link
Copy Markdown
Contributor

@basepi basepi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just one comment about db.instance. Otherwise I think this is looking good!

Comment thread specs/agents/tracing-instrumentation-db.md Outdated
Comment thread specs/agents/tracing-spans-service-target.md
@SylvainJuge SylvainJuge changed the title WIP DB granularity first draft spec Span service target fields specification Apr 13, 2022
@SylvainJuge
Copy link
Copy Markdown
Member Author

This specification is now ready for the next step, I've just renamed it to reflect that.

If we follow "The Process" ® the next step is getting one agent approval, @basepi since you already looked at it, would you mind doing that 🙏 ?

@SylvainJuge SylvainJuge marked this pull request as ready for review April 14, 2022 07:51
@SylvainJuge SylvainJuge requested review from a team as code owners April 14, 2022 07:51
Comment thread specs/agents/tracing-spans-service-target.md Outdated
- `span.context.service.target.type` : [ECS service.type](https://www.elastic.co/guide/en/ecs/current/ecs-service.html#field-service-type)
, optional.
- `span.context.service.target.name` : [ECS service name](https://www.elastic.co/guide/en/ecs/current/ecs-service.html#field-service-name)
, optional, ignored if `span.context.service.target.type` is not provided.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From the APM server's perspective, does this mean that for validating span.context.service.target it should expect span.context.service.target.type to be a required field?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just to be sure, here do you mean 1) in the data sent by agents, 2) to validate data stored to ES or 3) both ? I'll assume 1) here in my answer.

The whole span.context.service.target is optional, but I did not thought that we could maybe enforce a mandatory property within it, so maybe having span.context.service.target.type mandatory (but potentially empty) could be a good option, would that make things simpler on the server side for ingestion ?

However, one good reason to keep most of them optional is that the mandatory fields tends to stay forever for backwards compatibility after they are added, so while we don't have currently any use-case for that yet, it could help us later.
Also, another reason to stay cautious with mandatory fields is that if there is any extra complexity on the apm-server side, we can always fix it through code but the protocol can't be modified as easily. Another example of similar strategy in another context is with protobuf in version >=3 where all the attributes are optional for similar reasons.

Maybe @felixbarny could enlighten us with the agent protocol history and what should be the best strategy here.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After discussing with @SylvainJuge we have a decision to have APM-server expect span.context.service.target with at-least one field, name or type, as non-empty.

Comment thread specs/agents/tracing-spans-service-target.md
Comment thread specs/agents/tracing-spans-service-target.md Outdated
Co-authored-by: Trent Mick <trentm@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[META 621] Spec: Improve Granularity for SQL Databases

7 participants