Skip to content

Some event sending fail when POST request with non ASCII string #1475

@mkusaka

Description

@mkusaka

Describe the bug
When POST request through fetch with non ASCII string, error event failed to send to sentry server.

With config.debug = true, following log printed in console.

Event sending failed: "\xE3" from ASCII-8BIT to UTF-8
/Users/username/.rbenv/versions/2.7.3/lib/ruby/2.7.0/json/common.rb:224:in `generate'
/Users/username/.rbenv/versions/2.7.3/lib/ruby/2.7.0/json/common.rb:224:in `generate'
/Users/username/.rbenv/versions/2.7.3/lib/ruby/gems/2.7.0/gems/sentry-ruby-core-4.5.1/lib/sentry/transport.rb:102:in `encode'
/Users/username/.rbenv/versions/2.7.3/lib/ruby/gems/2.7.0/gems/sentry-ruby-core-4.5.1/lib/sentry/transport.rb:42:in `send_event'
/Users/username/.rbenv/versions/2.7.3/lib/ruby/gems/2.7.0/gems/sentry-ruby-core-4.5.1/lib/sentry/client.rb:86:in `send_event'
/Users/username/.rbenv/versions/2.7.3/lib/ruby/gems/2.7.0/gems/sentry-ruby-core-4.5.1/lib/sentry/client.rb:110:in `block in dispatch_background_event'
/Users/username/.rbenv/versions/2.7.3/lib/ruby/gems/2.7.0/gems/sentry-rails-4.5.1/lib/sentry/rails/background_worker.rb:7:in `block (2 levels) in perform'
...

To Reproduce
see: https://github.com/mkusaka/sentry-ruby-issue-repro

Expected behavior
Event sending not fail when POST request through fetch with non ASCII string.

Actual behavior
Event sending fail when POST request through fetch with non ASCII string.

Environment

  • Ruby Version: 2.7.3
  • SDK Version:
    • sentry-ruby: 4.5.1
    • sentry-rails: 4.5.1
  • Integration Versions (if any):
    • rails: 5.2.6

Sentry Config

This is not necessary but could be helpful.

Sentry.init do |config|
  config.send_default_pii = true
  config.debug = true
end

additional info
I've been searching for a while, and below is my summary for search.

Why this error occurred?

The direct cause is Rack::Request's body data encoding. It is encoded to ASCII-8BIT, and non ascii string convert to escaped string. And JSON.generate try to convert to UTF-8, but no conversion for escaped string, then raise error.

  • If config.send_default_pii = true and user send POST request from NON HTML form, sentry-ruby try to send event with request body.
  • Rack::Request#body#read returns ASCII-8BIT string.
    • NON ASCII-8BIT string converted to an escaped string, such as \xE5 etc...
  • JSON.generate (which called at Sentry::Transport#encode), try to convert ASCII-8BIT string into UTF-8, then raise this error.

How to fix this issue

Force-encode request body string to UTF-8 make fix this issue.

For example, in:

if request.form_data?
request.POST
elsif request.body # JSON requests, etc
data = request.body.read(MAX_BODY_LIMIT)
request.body.rewind
data
end

      if request.form_data?
        request.POST
      elsif request.body # JSON requests, etc
-       data = request.body.read(MAX_BODY_LIMIT)
+       data = request.body.read(MAX_BODY_LIMIT).force_encoding(Encoding::UTF_8)
        request.body.rewind
        data
      end

But, I'm not sure this is a good solution, because I don't know force_encoding works well in other situations.

  • Rails seems to convert (ASCII) URL to UTF-8 at rails/rails@7e50492, so I believe this solution works well in many cases.

similar: #1303

Metadata

Metadata

Assignees

No fields configured for issues without a type.

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions