Check CONTRIBUTING guideline first and here is the list to help us investigate the problem.
Describe the bug
I use out_http to output json arrays over HTTP. When the server is slow to respond, and ReadTimeout occurs, fluentd retries sending the chunk, but the JSON format is invalid (looks like the end of the buffer is trimmed).
To Reproduce
Send any log over http and make the server not respond for at least read_timeout seconds.
Expected behavior
Fluentd should retry the exact same chunk, or at least not slice the chunk in an arbitrary position.
Your Environment
- Fluentd version 1.11.2
- Operating system: Debian container running on MacOS
Your Configuration
<match **>
@type http
endpoint "http://localhost:8080" # local test server
open_timeout 2
read_timeout 10
content_type "application/json"
retryable_response_codes [503]
json_array true
<format>
@type json
add_newline false
</format>
<buffer>
# Interval for flushing event chunks
flush_interval 5s
# The number of threads to flush/write chunks in parallel
flush_thread_count 2
# The max size of each chunks: events will be written into chunks until
# the size of chunks become this size
chunk_limit_size 16mb
</buffer>
</match>
Your Error Log
Example ReadTimeout error:
[warn]: #0 failed to flush the buffer. retry_time=275 next_retry_seconds=2020-09-29 10:01:02 +0000 chunk="5b070cb025df79d5cd538adb86617115" error_class=Net::ReadTimeout error="Net::ReadTimeout"
Additional context
Example for an invalid JSON output (note the json array closing AFTER the payload is trimmed):
[{"worker=0","service":"fluentd","tag":"fluent.info"},"sourcetype":"fluentd","agent_timestamp":"2020-09-29T09:44:27Z","hostname":]
(the actual payload is larger, ±14kb)
Check CONTRIBUTING guideline first and here is the list to help us investigate the problem.
Describe the bug
I use out_http to output json arrays over HTTP. When the server is slow to respond, and
ReadTimeoutoccurs, fluentd retries sending the chunk, but the JSON format is invalid (looks like the end of the buffer is trimmed).To Reproduce
Send any log over http and make the server not respond for at least
read_timeoutseconds.Expected behavior
Fluentd should retry the exact same chunk, or at least not slice the chunk in an arbitrary position.
Your Environment
Your Configuration
Your Error Log
Example
ReadTimeouterror:Additional context
Example for an invalid JSON output (note the json array closing AFTER the payload is trimmed):
(the actual payload is larger, ±14kb)