Skip to content

feat: add graylog plugin TCP support#9644

Merged
sspaink merged 4 commits intoinfluxdata:masterfrom
bonitoo-io:feat/gelf-tcp-output
Sep 21, 2021
Merged

feat: add graylog plugin TCP support#9644
sspaink merged 4 commits intoinfluxdata:masterfrom
bonitoo-io:feat/gelf-tcp-output

Conversation

@alespour
Copy link
Copy Markdown
Contributor

@alespour alespour commented Aug 18, 2021

Required for all PRs:

  • Updated associated README.md.
  • Wrote appropriate unit tests.
  • Pull request title or commits are in conventional commit format (e.g. feat: or fix:)

This PR resolves #9034.

It adds support for TCP. Servers (endpoints) may now be specified as URLs with either udp or tcp scheme, eg. "udp://127.0.0.1:12201" or "tcp://127.0.0.1:12201". For compatiblity with existing configurations, if scheme is missing (eg. "127.0.0.1:12201") or is not udp neither tcp, UDP protocol is assumed.

Sample config has been changed to reflect it:

  ## Endpoints for your graylog instances.
  servers = ["udp://127.0.0.1:12201"]

  ## Connection timeout.
  # timeout = "5s"

  ## The field to use as the GELF short_message, if unset the static string
  ## "telegraf" will be used.
  ##   example: short_message_field = "message"
  # short_message_field = ""

Config option timeout allows to change connection dialup timeout.

Connection handling has been refactored. Although I'm not quite sure, it may be OK for UDP to have a connection created for each outgoing message, but it would not work well for TCP where network resources would get depleted quickly. Therefore connections are reused (and recreated whenever needed, ie. typically after write error occurs).

@telegraf-tiger telegraf-tiger bot added the feat Improvement on an existing feature such as adding a new setting/mode to an existing plugin label Aug 18, 2021
@alespour alespour changed the title feat: add TCP support feat: add graylog plugin TCP support Aug 18, 2021
@alespour alespour marked this pull request as ready for review August 18, 2021 13:17
@sjwang90 sjwang90 added the plugin/output 1. Request for new output plugins 2. Issues/PRs that are related to out plugins label Aug 26, 2021
@alespour alespour force-pushed the feat/gelf-tcp-output branch from dcd1ac5 to a07efd2 Compare September 15, 2021 14:15
@1tft
Copy link
Copy Markdown

1tft commented Sep 16, 2021

Played through some (TCP) test cases. I think it is working fine. So we are glad, that after so many months gelf output plugin speaks TCP. Maybe in future this plugin will also speak secure TCP (TLS).

We noticed a little issue: Input plugin name field (e.g. "name":"log_tail") are not sent with "_" prefix. Regarding Gelf spezification (https://docs.graylog.org/en/4.1/pages/gelf.html#gelf-payload-specification), all non standard fields should always be prefexed with "_".
But most Gelf receivers can live with this issue, so its not soooo important.

@sspaink sspaink merged commit a9898f1 into influxdata:master Sep 21, 2021
@alespour alespour deleted the feat/gelf-tcp-output branch October 5, 2021 13:12
@tamirhad
Copy link
Copy Markdown

Hey @alespour ,
is it possible to add TTL to the connection?
we are using multiple GL servers under AWS NLB LB, my concern is that single node will get overloaded with traffic since the connection will never be terminated.
i came across this post: https://medium.com/@manoj.senguttuvan/setting-up-a-load-balanced-logstash-behind-and-aws-elb-cc793bf9fda4

@alespour
Copy link
Copy Markdown
Contributor Author

alespour commented Jan 5, 2022

Hi @tamirhad I've created feature request #10367 for the connection TTL.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

feat Improvement on an existing feature such as adding a new setting/mode to an existing plugin plugin/output 1. Request for new output plugins 2. Issues/PRs that are related to out plugins

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Graylog output plugin: Add TCP option

6 participants