Describe the bug
When POST request through fetch with non ASCII string, error event failed to send to sentry server.
With config.debug = true, following log printed in console.
Event sending failed: "\xE3" from ASCII-8BIT to UTF-8
/Users/username/.rbenv/versions/2.7.3/lib/ruby/2.7.0/json/common.rb:224:in `generate'
/Users/username/.rbenv/versions/2.7.3/lib/ruby/2.7.0/json/common.rb:224:in `generate'
/Users/username/.rbenv/versions/2.7.3/lib/ruby/gems/2.7.0/gems/sentry-ruby-core-4.5.1/lib/sentry/transport.rb:102:in `encode'
/Users/username/.rbenv/versions/2.7.3/lib/ruby/gems/2.7.0/gems/sentry-ruby-core-4.5.1/lib/sentry/transport.rb:42:in `send_event'
/Users/username/.rbenv/versions/2.7.3/lib/ruby/gems/2.7.0/gems/sentry-ruby-core-4.5.1/lib/sentry/client.rb:86:in `send_event'
/Users/username/.rbenv/versions/2.7.3/lib/ruby/gems/2.7.0/gems/sentry-ruby-core-4.5.1/lib/sentry/client.rb:110:in `block in dispatch_background_event'
/Users/username/.rbenv/versions/2.7.3/lib/ruby/gems/2.7.0/gems/sentry-rails-4.5.1/lib/sentry/rails/background_worker.rb:7:in `block (2 levels) in perform'
...
To Reproduce
see: https://github.com/mkusaka/sentry-ruby-issue-repro
Expected behavior
Event sending not fail when POST request through fetch with non ASCII string.
Actual behavior
Event sending fail when POST request through fetch with non ASCII string.
Environment
- Ruby Version: 2.7.3
- SDK Version:
- sentry-ruby: 4.5.1
- sentry-rails: 4.5.1
- Integration Versions (if any):
Sentry Config
This is not necessary but could be helpful.
Sentry.init do |config|
config.send_default_pii = true
config.debug = true
end
additional info
I've been searching for a while, and below is my summary for search.
Why this error occurred?
The direct cause is Rack::Request's body data encoding. It is encoded to ASCII-8BIT, and non ascii string convert to escaped string. And JSON.generate try to convert to UTF-8, but no conversion for escaped string, then raise error.
- If
config.send_default_pii = true and user send POST request from NON HTML form, sentry-ruby try to send event with request body.
|
if request.form_data? |
|
request.POST |
|
elsif request.body # JSON requests, etc |
|
data = request.body.read(MAX_BODY_LIMIT) |
|
request.body.rewind |
|
data |
|
end |
Rack::Request#body#read returns ASCII-8BIT string.
- NON ASCII-8BIT string converted to an escaped string, such as
\xE5 etc...
JSON.generate (which called at Sentry::Transport#encode), try to convert ASCII-8BIT string into UTF-8, then raise this error.
How to fix this issue
Force-encode request body string to UTF-8 make fix this issue.
For example, in:
|
if request.form_data? |
|
request.POST |
|
elsif request.body # JSON requests, etc |
|
data = request.body.read(MAX_BODY_LIMIT) |
|
request.body.rewind |
|
data |
|
end |
if request.form_data?
request.POST
elsif request.body # JSON requests, etc
- data = request.body.read(MAX_BODY_LIMIT)
+ data = request.body.read(MAX_BODY_LIMIT).force_encoding(Encoding::UTF_8)
request.body.rewind
data
end
But, I'm not sure this is a good solution, because I don't know force_encoding works well in other situations.
- Rails seems to convert (ASCII) URL to UTF-8 at rails/rails@7e50492, so I believe this solution works well in many cases.
similar: #1303
Describe the bug
When POST request through fetch with non ASCII string, error event failed to send to sentry server.
With
config.debug = true, following log printed in console.To Reproduce
see: https://github.com/mkusaka/sentry-ruby-issue-repro
Expected behavior
Event sending not fail when POST request through fetch with non ASCII string.
Actual behavior
Event sending fail when POST request through fetch with non ASCII string.
Environment
Sentry Config
This is not necessary but could be helpful.
additional info
I've been searching for a while, and below is my summary for search.
Why this error occurred?
The direct cause is Rack::Request's body data encoding. It is encoded to ASCII-8BIT, and non ascii string convert to escaped string. And JSON.generate try to convert to UTF-8, but no conversion for escaped string, then raise error.
config.send_default_pii = trueand user send POST request from NON HTML form, sentry-ruby try to send event with request body.sentry-ruby/sentry-ruby/lib/sentry/interfaces/request.rb
Lines 56 to 62 in 6d2fc60
Rack::Request#body#readreturns ASCII-8BIT string.\xE5etc...JSON.generate(which called atSentry::Transport#encode), try to convert ASCII-8BIT string into UTF-8, then raise this error.How to fix this issue
Force-encode request body string to UTF-8 make fix this issue.
For example, in:
sentry-ruby/sentry-ruby/lib/sentry/interfaces/request.rb
Lines 56 to 62 in 6d2fc60
if request.form_data? request.POST elsif request.body # JSON requests, etc - data = request.body.read(MAX_BODY_LIMIT) + data = request.body.read(MAX_BODY_LIMIT).force_encoding(Encoding::UTF_8) request.body.rewind data endBut, I'm not sure this is a good solution, because I don't know force_encoding works well in other situations.
similar: #1303