Dynamic Sampling Context Continuation / Baggage by sl0thentr0py · Pull Request #1869 · getsentry/sentry-ruby

sl0thentr0py · 2022-08-16T15:42:32Z

This PR adds support for the first part of our Dynamic Sampling product, i.e.

parsing incoming W3C Baggage headers and adding them as a Baggage object on the transaction
adding this baggage header to other outgoing HTTP requests in our net/http patch
converting the sentry-* items in the baggage to a DSC (Dynamic Sampling Context) and attaching them to outgoing envelope headers in the trace field

now contains #1898 and #1902

codecov-commenter · 2022-08-17T13:56:12Z

Codecov Report

❌ Patch coverage is 99.75248% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 98.46%. Comparing base (d83b29c) to head (a558764).
⚠️ Report is 509 commits behind head on master.

Files with missing lines	Patch %	Lines
sentry-ruby/lib/sentry/rake.rb	0.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #1869      +/-   ##
==========================================
+ Coverage   98.38%   98.46%   +0.08%     
==========================================
  Files         148      150       +2     
  Lines        8893     9240     +347     
==========================================
+ Hits         8749     9098     +349     
+ Misses        144      142       -2

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

sentry-ruby/lib/sentry/baggage.rb

sentry-ruby/lib/sentry/span.rb

sentry-ruby/lib/sentry/baggage.rb

sentry-ruby/lib/sentry/span.rb

sentry-ruby/lib/sentry/client.rb

st0012

Do we want to release this in the next minor release?

sentry-ruby/lib/sentry/transport.rb

sl0thentr0py · 2022-09-15T09:13:04Z

@st0012 for releasing this, we need 2 more PRs, so maybe let's merge them all together. They're independent as features, but should go out together ideally.

head SDK baggage/DSC creation - feat(tracing) Collect DSC for head of trace #1836
add transaction_info.source annotations - Transaction source #1866

sl0thentr0py · 2022-09-15T12:32:50Z

I'm making this a draft again so we don't merge it accidentally

…upled with sentry_trace

sl0thentr0py · 2022-09-19T12:56:19Z

@st0012 after some discussion, we decided to remove all third party handling since that was mostly unused logic.

Creates new `Baggage` entries when the ruby SDK is the `head SDK`, i.e the first SDK that originates a distributed trace. The new `Baggage` entry is created when `get_baggage` is first called, either * when the first outgoing HTTP request is made * the first transaction finishes and the envelope is sent to sentry

sl0thentr0py · 2022-09-28T13:20:15Z

E2E testing documentation

JS as head SDK / continuation

relevant frontend request headers (baggage)

GET /ds/items HTTP/1.1
Host: localhost:3000
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:105.0) Gecko/20100101 Firefox/105.0
Accept: */*
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate, br
Referer: http://localhost:3000/ds_js_head
sentry-trace: 9821e65c92894d60ab9b05e69d84e218-994f1602ff45d966-1
baggage: sentry-environment=development,sentry-release=foobar,sentry-public_key=d16ae2d36f9249849c7964e9a3a8a608,sentry-trace_id=9821e65c92894d60ab9b05e69d84e218,sentry-sample_rate=1

Outgoing trace envelope header (dynamic sampling context)

[5] pry(#<Sentry::HTTPTransport>)> envelope.headers[:trace]
=> {"environment"=>"development", "release"=>"foobar", "public_key"=>"d16ae2d36f9249849c7964e9a3a8a608", "trace_id"=>"9821e65c92894d60ab9b05e69d84e218", "sample_rate"=>"1"}

correct source annotation in transaction_info

[8] pry(#<Sentry::HTTPTransport>)> envelope.items.first.payload[:transaction_info]
=> {:source=>:view}

Backend-backend propagation of baggage

Rails as head SDK

meta tags on rails template

<meta name="sentry-trace" content="43906b36c4e84d97b310908dfa323ca0-534d009e7d378188-1">
<meta name="baggage" content="sentry-trace_id=43906b36c4e84d97b310908dfa323ca0,sentry-sample_rate=1.0,sentry-environment=development,sentry-release=test-sessions-2022-09-28+14%3A34%3A02+UTC,sentry-public_key=2fb45f003d054a7ea47feb45898f7649">

outgoing request from browser continues that trace

GET /ds/items HTTP/1.1
Host: localhost:3000
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:105.0) Gecko/20100101 Firefox/105.0
Accept: */*
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate, br
Referer: http://localhost:3000/ds_rails_head
sentry-trace: 43906b36c4e84d97b310908dfa323ca0-adc949650e49f6da-1
baggage: sentry-trace_id=43906b36c4e84d97b310908dfa323ca0,sentry-sample_rate=1.0,sentry-environment=development,sentry-release=test-sessions-2022-09-28%2B14%3A34%3A02%2BUTC,sentry-public_key=2fb45f003d054a7ea47feb45898f7649

envelope from browser transaction has correct trace header

"trace":{"trace_id":"43906b36c4e84d97b310908dfa323ca0","sample_rate":"1.0","environment":"development","release":"test-sessions-2022-09-28+14:34:02+UTC","public_key":"2fb45f003d054a7ea47feb45898f7649"}}

final flask request also has correct baggage header

sentry-trace_id=43906b36c4e84d97b310908dfa323ca0,sentry-sample_rate=1.0,sentry-environment=development,sentry-release=test-sessions-2022-09-28%2B14%3A34%3A02%2BUTC,sentry-public_key=2fb45f003d054a7ea47feb45898f7649

sl0thentr0py · 2022-09-28T14:53:22Z

ok @st0012 I've tested this E2E properly in all scenarios, everything seems to work 🎉

antonpirker

Just a couple of question for me to understand. The tests I just skimmed over because my brain is already porridge, but everything looks ok (because it also looks very similar to the python implementation.)

sentry-rails/lib/sentry/rails/action_cable.rb

sentry-ruby/lib/sentry/client.rb

sentry-ruby/lib/sentry/transaction.rb

st0012 · 2022-09-30T09:47:12Z

@sl0thentr0py Should we make this PR ready for review?

sl0thentr0py force-pushed the neel/baggage-dsc-continuation branch from 5e088e8 to 2619677 Compare August 17, 2022 12:58

sl0thentr0py force-pushed the neel/baggage-dsc-continuation branch from 775f94c to f2c1089 Compare August 17, 2022 14:00

sl0thentr0py mentioned this pull request Aug 18, 2022

Remove webmock #1871

Merged

sl0thentr0py force-pushed the neel/baggage-dsc-continuation branch from ed734cd to e44980f Compare August 19, 2022 10:42

sl0thentr0py changed the title ~~Dynamic Sampling Context Continuation / Baggage WIP~~ Dynamic Sampling Context Continuation / Baggage Aug 19, 2022

sl0thentr0py force-pushed the neel/baggage-dsc-continuation branch from b8e3f9e to 0d7350a Compare August 19, 2022 14:50

st0012 reviewed Aug 26, 2022

View reviewed changes

sentry-ruby/lib/sentry/baggage.rb Outdated Show resolved Hide resolved

sentry-ruby/lib/sentry/baggage.rb Outdated Show resolved Hide resolved

sentry-ruby/lib/sentry/span.rb Outdated Show resolved Hide resolved

sl0thentr0py force-pushed the neel/baggage-dsc-continuation branch from 0fbe77a to c08e737 Compare September 5, 2022 14:49

sl0thentr0py marked this pull request as ready for review September 8, 2022 13:30

sl0thentr0py force-pushed the neel/baggage-dsc-continuation branch from c4cfc52 to 38c1603 Compare September 13, 2022 11:15

sl0thentr0py requested a review from st0012 September 13, 2022 14:06

st0012 reviewed Sep 14, 2022

View reviewed changes

sentry-ruby/lib/sentry/baggage.rb Outdated Show resolved Hide resolved

sentry-ruby/lib/sentry/baggage.rb Show resolved Hide resolved

sentry-ruby/lib/sentry/span.rb Outdated Show resolved Hide resolved

sentry-ruby/lib/sentry/client.rb Outdated Show resolved Hide resolved

sl0thentr0py mentioned this pull request Sep 14, 2022

Make transaction required in Span constructor #1890

Closed

sl0thentr0py force-pushed the neel/baggage-dsc-continuation branch from 38c1603 to 5cea861 Compare September 14, 2022 12:17

st0012 reviewed Sep 15, 2022

View reviewed changes

sentry-ruby/lib/sentry/transport.rb Show resolved Hide resolved

st0012 added enhancement sentry-rails sentry-ruby labels Sep 15, 2022

st0012 added this to the 5.5.0 milestone Sep 15, 2022

st0012 assigned sl0thentr0py Sep 15, 2022

sl0thentr0py force-pushed the neel/baggage-dsc-continuation branch 2 times, most recently from 75bcad6 to 522f0f2 Compare September 15, 2022 12:10

sl0thentr0py marked this pull request as draft September 15, 2022 12:32

sl0thentr0py force-pushed the neel/baggage-dsc-continuation branch from f1bd637 to 7490335 Compare September 15, 2022 12:58

sl0thentr0py added 3 commits September 19, 2022 13:08

Expose baggage in Transaction#initialize

57eee75

Implement Baggage class

962658b

Propagate dynamic sampling context to envelope trace header

bfe4e32

sl0thentr0py added 9 commits September 19, 2022 13:08

Span#to_baggage spec

7c41366

Review fixes

2fa553a

net/http spec

1167d88

transport spec

804dcc1

rack spec

1c979e9

rails tracing spec

5230b61

Review fixes

f59152e

changelog entry for dynamic sampling

6ae3ad8

Move baggage to from_sentry_trace constructor because it is always co…

8b8af16

…upled with sentry_trace

sl0thentr0py force-pushed the neel/baggage-dsc-continuation branch from 7490335 to 8b8af16 Compare September 19, 2022 12:08

Remove third_party_items

dde8094

sl0thentr0py modified the milestones: 5.5.0, Dynamic Sampling Sep 28, 2022

antonpirker reviewed Sep 29, 2022

View reviewed changes

sentry-rails/lib/sentry/rails/action_cable.rb Show resolved Hide resolved

sentry-ruby/lib/sentry/client.rb Show resolved Hide resolved

sentry-ruby/lib/sentry/transaction.rb Show resolved Hide resolved

Add transaction source annotation (#1902)

a558764

sl0thentr0py marked this pull request as ready for review September 30, 2022 09:15

sl0thentr0py requested a review from st0012 September 30, 2022 09:15

st0012 added sentry spec Traces sentry-sidekiq sentry-delayed_job sentry-resque labels Sep 30, 2022

st0012 approved these changes Sep 30, 2022

View reviewed changes

sl0thentr0py merged commit c79aa34 into master Oct 3, 2022

sl0thentr0py deleted the neel/baggage-dsc-continuation branch October 3, 2022 12:27

Uh oh!

Conversation

sl0thentr0py commented Aug 16, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codecov-commenter commented Aug 17, 2022 • edited by codecov bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

st0012 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

sl0thentr0py commented Sep 15, 2022

Uh oh!

sl0thentr0py commented Sep 15, 2022

Uh oh!

sl0thentr0py commented Sep 19, 2022

Uh oh!

sl0thentr0py commented Sep 28, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

E2E testing documentation

JS as head SDK / continuation

Backend-backend propagation of baggage

Rails as head SDK

Uh oh!

sl0thentr0py commented Sep 28, 2022

Uh oh!

antonpirker left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

st0012 commented Sep 30, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

sl0thentr0py commented Aug 16, 2022 •

edited

Loading

codecov-commenter commented Aug 17, 2022 •

edited by codecov bot

Loading

sl0thentr0py commented Sep 28, 2022 •

edited

Loading