Skip to content

turn gzip on by default for influxdb v1#8269

Merged
ssoroka merged 1 commit intomasterfrom
influxdb_gzip
Oct 16, 2020
Merged

turn gzip on by default for influxdb v1#8269
ssoroka merged 1 commit intomasterfrom
influxdb_gzip

Conversation

@ssoroka
Copy link
Copy Markdown
Contributor

@ssoroka ssoroka commented Oct 14, 2020

resolves #7397

@reimda
Copy link
Copy Markdown
Contributor

reimda commented Oct 15, 2020

I wish gzip would have been the default from the beginning but I have reservations about changing the default at this point. It was uncompressed from the beginning of telegraf (almost 5 years ago) and default uncompressed since the content encoding option was added 3 years ago.

It should be safe in most cases to switch to gzip, but there are cases where it might not be, such as when the device running telegraf was sized with the expectation of low cpu usage (think embedded device or small virtual instance on a cloud provider). The same goes for influxdb which has to gunzip if telegraf is sending gzipped.

I like the idea of gradually defaulting to gzip. The v2 output is already default gzip, so people upgrading that way will get it. I'd also be be fine changing the v1 output default to gzip in a major release (2.0) where backwards compatibility isn't as expected as a minor release (1.x).

I haven't seen a lot of user reports of uncompressed v1 being a problem. You mentioned one case in #7397 (comment) where enabling gzip helped. I don't think it's worth the risk when people aren't asking for it.

@ssoroka
Copy link
Copy Markdown
Contributor Author

ssoroka commented Oct 15, 2020

I understand where you're coming from, but the potential gains here outweigh any potential risk, especially since you can opt to turn it off. Anyone who has set this setting explicitly will not be affected, and we can document the change.

Some other things to consider:

  • I've talked to almost a dozen customers now where the answer was to enable gzip. Nobody has ever complained about gzip support in v2 being the default, and the server has no problems handling this.
  • the cost of gzip is order of magnitude less than the network costs of transmitting the data.
  • gzip compression is the standard everywhere. This is an extremely reasonable and safe default.
  • No server has ever died or become slow, to my knowledge, due to the cost of doing gzip decompression. TLS will be an issue before gzip is.
  • this isn't breaking backwards compatibility, because all past users will continue to work (our worst case scenario for those users is slightly higher cpu)

lastly, we can't be afraid that we might make performance worse for a few people with underpowered devices at the cost of performance at high-volume for the majority.

@ssoroka ssoroka merged commit 78cf0b7 into master Oct 16, 2020
@ssoroka ssoroka deleted the influxdb_gzip branch October 16, 2020 17:07
arstercz pushed a commit to arstercz/telegraf that referenced this pull request Mar 5, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

influxdb output (v1) should gzip by default

2 participants