Add instrumentation for AWS SQS and SNS by estolfo · Pull Request #978 · elastic/apm-agent-ruby

estolfo · 2021-03-04T10:31:41Z

I've added support for SNS and SQS in the same PR because they both depend on some additions to the span context.

SQS

Note that there is no instrumentation for polling, as you can poll a SQS queue by doing one of the following two things:

Set wait_time_seconds when calling client#receive_message. So the action is still RECEIVE, though with a longer timeout. I didn't see that it was necessary to inspect the method params for a wait_time_seconds value and identify the action as POLL instead of RECEIVE in the case that it was set.
Use the QueuePoller class. This class is just a wrapper around client methods. QueuePoller.poll just calls receive_message on a client in a loop. So I didn't think it necessary to instrument this class, as the user will see the individual RECEIVE actions instrumented anyway.

This PR also adds a class Message that allows span.context.message fields to be added.

And lastly, the SQS instrumentation may look repetitive but each method differs slightly. For example, we must iterate over the params[:entries] in order to add trace context info to messages sent via send_message_batch, as opposed to adding the info directly to the params. I thought it would make the code more confusing if I tried to centralize the instrumentation into one method that handled slight differences.

See the spec here

Closes #946

SNS

Closes #945

ghost · 2021-03-04T10:42:18Z

💔 Build Failed

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

Build Cause: Pull request #978 updated
Start Time: 2021-03-04T15:17:57.426+0000
Duration: 12 min 11 sec
Commit: 429557b

Trends 🧪

Steps errors

Expand to view the steps failures

Show only the first 10 steps failures

`Shell Script`

Took 1 min 38 sec . View more details on here
Description: ./spec/scripts/spec.sh ruby:3.0 sinatra-master

`Shell Script`

Took 1 min 47 sec . View more details on here
Description: ./spec/scripts/spec.sh ruby:3.0 grape-master

`Shell Script`

Took 1 min 37 sec . View more details on here
Description: ./spec/scripts/spec.sh ruby:3.0 grape-master

`Shell Script`

Took 2 min 35 sec . View more details on here
Description: ./spec/scripts/spec.sh ruby:2.7 rails-main

`Shell Script`

Took 1 min 25 sec . View more details on here
Description: ./spec/scripts/spec.sh ruby:2.7 rails-main

`Shell Script`

Took 1 min 10 sec . View more details on here
Description: ./spec/scripts/spec.sh ruby:2.7 sinatra-master

`Shell Script`

Took 1 min 55 sec . View more details on here
Description: ./spec/scripts/spec.sh ruby:2.7 sinatra-master

`Shell Script`

Took 1 min 8 sec . View more details on here
Description: ./spec/scripts/spec.sh ruby:2.7 grape-master

`Shell Script`

Took 1 min 57 sec . View more details on here
Description: ./spec/scripts/spec.sh ruby:2.7 grape-master

`Error signal`

Took 0 min 0 sec . View more details on here
Description: Ruby:ruby:3.0#rails-6.1 tests failed : hudson.AbortException: script returned exit code 1

Log output

Expand to view the last 100 lines of log output

[2021-03-04T15:29:32.023Z] [Checks API] No suitable checks publisher found.
[2021-03-04T15:29:32.042Z] Running in /var/lib/jenkins/workspace/t-ruby_apm-agent-ruby-mbp_PR-978/src/github.com/elastic/apm-agent-ruby
[2021-03-04T15:29:32.076Z] Archiving artifacts
[2021-03-04T15:29:32.572Z] Stashed 32 file(s)
[2021-03-04T15:29:32.652Z] Failed in branch Ruby:ruby:2.4#rails-4.2
[2021-03-04T15:29:34.156Z] Coverage may be inaccurate; set the "--debug" command line option, or do JRUBY_OPTS="--debug" or set the "debug.fullTrace=true" option in your .jrubyrc
[2021-03-04T15:29:34.843Z] /usr/local/bundle/jruby/2.5.0/gems/simplecov-0.21.2/lib/simplecov.rb:345: warning: tracing (e.g. set_trace_func) will not capture all events without --debug flag
[2021-03-04T15:29:39.140Z] [INFO] Skipping Sinatra.start spec
[2021-03-04T15:29:43.083Z] [INFO] Skipping Sinatra.start spec
[2021-03-04T15:29:45.824Z] 
[2021-03-04T15:29:45.824Z] An error occurred while loading ./spec/elastic_apm/spies/sns_spec.rb. - Did you mean?
[2021-03-04T15:29:45.824Z]                     rspec ./spec/elastic_apm/spies/sqs_spec.rb
[2021-03-04T15:29:45.824Z]                     rspec ./spec/elastic_apm/spies/mongo_spec.rb
[2021-03-04T15:29:45.824Z]                     rspec ./spec/elastic_apm/spies/sneakers_spec.rb
[2021-03-04T15:29:45.824Z] 
[2021-03-04T15:29:45.824Z] Failure/Error: res = require_without_apm(path)
[2021-03-04T15:29:45.824Z] 
[2021-03-04T15:29:45.824Z] LoadError:
[2021-03-04T15:29:45.824Z]   no such file to load -- aws-sdk-sns
[2021-03-04T15:29:45.824Z] # ./lib/elastic_apm/spies.rb:91:in `require'
[2021-03-04T15:29:45.824Z] # ./spec/elastic_apm/spies/sns_spec.rb:21:in `<main>'
[2021-03-04T15:29:48.430Z] 
[2021-03-04T15:29:48.430Z] An error occurred while loading ./spec/elastic_apm/spies/sns_spec.rb. - Did you mean?
[2021-03-04T15:29:48.430Z]                     rspec ./spec/elastic_apm/spies/sqs_spec.rb
[2021-03-04T15:29:48.430Z]                     rspec ./spec/elastic_apm/spies/mongo_spec.rb
[2021-03-04T15:29:48.430Z]                     rspec ./spec/elastic_apm/spies/sneakers_spec.rb
[2021-03-04T15:29:48.430Z] 
[2021-03-04T15:29:48.430Z] Failure/Error: res = require_without_apm(path)
[2021-03-04T15:29:48.430Z] 
[2021-03-04T15:29:48.430Z] LoadError:
[2021-03-04T15:29:48.430Z]   no such file to load -- aws-sdk-sns
[2021-03-04T15:29:48.430Z] # ./lib/elastic_apm/spies.rb:91:in `require'
[2021-03-04T15:29:48.430Z] # /usr/local/bundle/jruby/2.5.0/gems/activesupport-5.2.4.5/lib/active_support/dependencies.rb:291:in `block in require'
[2021-03-04T15:29:48.430Z] # /usr/local/bundle/jruby/2.5.0/gems/activesupport-5.2.4.5/lib/active_support/dependencies.rb:257:in `load_dependency'
[2021-03-04T15:29:48.430Z] # /usr/local/bundle/jruby/2.5.0/gems/activesupport-5.2.4.5/lib/active_support/dependencies.rb:291:in `require'
[2021-03-04T15:29:48.430Z] # ./spec/elastic_apm/spies/sns_spec.rb:21:in `<main>'
[2021-03-04T15:29:48.430Z] # /usr/local/bundle/jruby/2.5.0/gems/activesupport-5.2.4.5/lib/active_support/dependencies.rb:285:in `block in load'
[2021-03-04T15:29:48.430Z] # /usr/local/bundle/jruby/2.5.0/gems/activesupport-5.2.4.5/lib/active_support/dependencies.rb:257:in `load_dependency'
[2021-03-04T15:29:48.430Z] # /usr/local/bundle/jruby/2.5.0/gems/activesupport-5.2.4.5/lib/active_support/dependencies.rb:285:in `load'
[2021-03-04T15:29:49.151Z] 
[2021-03-04T15:29:49.151Z] 
[2021-03-04T15:29:49.151Z] Finished in 0.00153 seconds (files took 23.35 seconds to load)
[2021-03-04T15:29:49.151Z] 0 examples, 0 failures, 1 error occurred outside of examples
[2021-03-04T15:29:49.151Z] 
[2021-03-04T15:29:49.382Z] 
[2021-03-04T15:29:49.382Z] 
[2021-03-04T15:29:49.382Z] Finished in 0.00127 seconds (files took 15.93 seconds to load)
[2021-03-04T15:29:49.382Z] 0 examples, 0 failures, 1 error occurred outside of examples
[2021-03-04T15:29:49.383Z] 
[2021-03-04T15:29:51.079Z] Coverage report generated for RSpec to /app/coverage/matrix_results/grape-1.5-docker.elastic.co/observability-ci/jruby-9.2-11-jdk. 0 / 3461 LOC (0.0%) covered.
[2021-03-04T15:29:51.079Z] Stopped processing SimpleCov as a previous error not related to SimpleCov has been detected
[2021-03-04T15:29:51.405Z] Coverage report generated for RSpec to /app/coverage/matrix_results/rails-5.2-docker.elastic.co/observability-ci/jruby-9.2-8-jdk. 0 / 3454 LOC (0.0%) covered.
[2021-03-04T15:29:51.406Z] Stopped processing SimpleCov as a previous error not related to SimpleCov has been detected
[2021-03-04T15:29:52.176Z] Recording test results
[2021-03-04T15:29:52.442Z] Recording test results
[2021-03-04T15:29:52.676Z] None of the test reports contained any result
[2021-03-04T15:29:52.676Z] [Checks API] No suitable checks publisher found.
[2021-03-04T15:29:52.686Z] Running in /var/lib/jenkins/workspace/t-ruby_apm-agent-ruby-mbp_PR-978/src/github.com/elastic/apm-agent-ruby
[2021-03-04T15:29:52.715Z] Archiving artifacts
[2021-03-04T15:29:53.118Z] Stashed 32 file(s)
[2021-03-04T15:29:53.175Z] Failed in branch Ruby:docker.elastic.co/observability-ci/jruby:9.2-11-jdk#grape-1.5
[2021-03-04T15:29:53.639Z] None of the test reports contained any result
[2021-03-04T15:29:53.640Z] [Checks API] No suitable checks publisher found.
[2021-03-04T15:29:53.650Z] Running in /var/lib/jenkins/workspace/t-ruby_apm-agent-ruby-mbp_PR-978/src/github.com/elastic/apm-agent-ruby
[2021-03-04T15:29:53.673Z] Archiving artifacts
[2021-03-04T15:29:54.039Z] Stashed 32 file(s)
[2021-03-04T15:29:54.073Z] Failed in branch Ruby:docker.elastic.co/observability-ci/jruby:9.2-8-jdk#rails-5.2
[2021-03-04T15:29:55.023Z] Post stage
[2021-03-04T15:29:55.488Z] Running in /var/lib/jenkins/workspace/t-ruby_apm-agent-ruby-mbp_PR-978/src/github.com/elastic/apm-agent-ruby
[2021-03-04T15:29:57.381Z] Error when executing always post condition:
[2021-03-04T15:29:57.381Z] hudson.AbortException: No such saved stash ‘jruby-9.2-13-jdk-rails-6.1’
[2021-03-04T15:29:57.381Z] 	at org.jenkinsci.plugins.workflow.flow.StashManager.unstash(StashManager.java:159)
[2021-03-04T15:29:57.381Z] 	at org.jenkinsci.plugins.workflow.support.steps.stash.UnstashStep$Execution.run(UnstashStep.java:76)
[2021-03-04T15:29:57.381Z] 	at org.jenkinsci.plugins.workflow.support.steps.stash.UnstashStep$Execution.run(UnstashStep.java:63)
[2021-03-04T15:29:57.381Z] 	at org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution.lambda$start$0(SynchronousNonBlockingStepExecution.java:47)
[2021-03-04T15:29:57.381Z] 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[2021-03-04T15:29:57.381Z] 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[2021-03-04T15:29:57.381Z] 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[2021-03-04T15:29:57.381Z] 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[2021-03-04T15:29:57.381Z] 	at java.lang.Thread.run(Thread.java:748)
[2021-03-04T15:29:57.381Z] 
[2021-03-04T15:29:57.526Z] Failed in branch Tests
[2021-03-04T15:30:06.688Z] [INFO] For detailed information see: https://apm-ci.elastic.co/job/apm-integration-tests-selector-mbp/job/master/14808/display/redirect
[2021-03-04T15:30:06.825Z] Stage "Benchmarks" skipped due to earlier failure(s)
[2021-03-04T15:30:06.849Z] Stage "Benchmarks" skipped due to earlier failure(s)
[2021-03-04T15:30:06.916Z] Stage "Benchmarks" skipped due to earlier failure(s)
[2021-03-04T15:30:07.046Z] Stage "Release" skipped due to earlier failure(s)
[2021-03-04T15:30:07.077Z] Stage "Release" skipped due to earlier failure(s)
[2021-03-04T15:30:07.145Z] Stage "Release" skipped due to earlier failure(s)
[2021-03-04T15:30:07.220Z] Stage "Release" skipped due to earlier failure(s)
[2021-03-04T15:30:07.307Z] Stage "Release" skipped due to earlier failure(s)
[2021-03-04T15:30:07.698Z] Running on Jenkins in /var/lib/jenkins/workspace/t-ruby_apm-agent-ruby-mbp_PR-978
[2021-03-04T15:30:07.745Z] [INFO] getVaultSecret: Getting secrets
[2021-03-04T15:30:08.222Z] Masking supported pattern matches of $VAULT_ADDR or $VAULT_ROLE_ID or $VAULT_SECRET_ID
[2021-03-04T15:30:08.925Z] + chmod 755 generate-build-data.sh
[2021-03-04T15:30:08.925Z] + ./generate-build-data.sh https://apm-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/apm-agent-ruby/apm-agent-ruby-mbp/PR-978/ https://apm-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/apm-agent-ruby/apm-agent-ruby-mbp/PR-978/runs/7 FAILURE 731236
[2021-03-04T15:30:09.176Z] INFO: curl https://apm-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/apm-agent-ruby/apm-agent-ruby-mbp/PR-978/runs/7/steps/?limit=10000 -o steps-info.json
[2021-03-04T15:30:15.657Z] INFO: curl https://apm-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/apm-agent-ruby/apm-agent-ruby-mbp/PR-978/runs/7/tests/?status=FAILED -o tests-errors.json
[2021-03-04T15:30:15.657Z] Retry 1/3 exited 22, retrying in 1 seconds...
[2021-03-04T15:30:16.208Z] Retry 2/3 exited 22, retrying in 2 seconds...

github-actions bot added the agent-ruby label Mar 4, 2021

estolfo added 5 commits March 4, 2021 11:36

Add message object to span context

0403ccd

Add instrumentation for SQS

4cd9cde

Add test for message attributes

747240c

Add newline at end of file

c863735

Set context to a variable for clarity

8022bf1

estolfo force-pushed the aws-spec-sqs branch from c5082f9 to 8022bf1 Compare March 4, 2021 10:36

Add Changelog entry

80974cd

Add support for AWS SNS

92638f7

estolfo changed the title ~~Add instrumentation for AWS SQS~~ Add instrumentation for AWS SQS and SNS Mar 4, 2021

estolfo added 2 commits March 4, 2021 15:33

Add placeholder for getting region from queue_url in SQS

e5291e1

Clearer span name construction

429557b

mikker mentioned this pull request Mar 18, 2021

Add instrumentation for AWS SQS and SNS (cont.) #988

Merged

mikker closed this in #988 Mar 19, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add instrumentation for AWS SQS and SNS#978

Add instrumentation for AWS SQS and SNS#978
estolfo wants to merge 9 commits intoelastic:masterfrom
estolfo:aws-spec-sqs

estolfo commented Mar 4, 2021 •

edited by zube bot

Loading

Uh oh!

ghost commented Mar 4, 2021 •

edited by ghost

Loading

Build stats

Trends 🧪

`Shell Script`

`Shell Script`

`Shell Script`

`Shell Script`

`Shell Script`

`Shell Script`

`Shell Script`

`Shell Script`

`Shell Script`

`Error signal`

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

estolfo commented Mar 4, 2021 • edited by zube bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

SQS

SNS

Uh oh!

ghost commented Mar 4, 2021 • edited by ghost Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

💔 Build Failed

Build stats

Trends 🧪

Steps errors

Shell Script

Shell Script

Shell Script

Shell Script

Shell Script

Shell Script

Shell Script

Shell Script

Shell Script

Error signal

Log output

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

estolfo commented Mar 4, 2021 •

edited by zube bot

Loading

ghost commented Mar 4, 2021 •

edited by ghost

Loading

`Shell Script`

`Shell Script`

`Shell Script`

`Shell Script`

`Shell Script`

`Shell Script`

`Shell Script`

`Shell Script`

`Shell Script`

`Error signal`