Performance: Added fast_double_parser to improve double conversion time#683
Merged
Conversation
4a2f69b to
72ba867
Compare
72ba867 to
c8e90ea
Compare
danni-m
previously approved these changes
Apr 1, 2021
danni-m
previously approved these changes
Apr 1, 2021
rafie
reviewed
Apr 1, 2021
rafie
reviewed
Apr 1, 2021
rafie
reviewed
Apr 1, 2021
rafie
approved these changes
Apr 1, 2021
danni-m
approved these changes
Apr 1, 2021
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR improves string to double conversion by leveraging a C Wrapper for the C++ Eisel-Lemire ParseFloat algorithm.
This optimizes primarily our write workflows and we've measured an improvement of ~=8% on the achievable ops/sec.
Bellow you can find the top-down tree of cpu usage that made us focus on this optimization.
Top-Down tree of hotspots by CPU utilization for TS.MADD....
Please notice the 16.7% line, which we're optimizing on this PR.
Validating improvements with a tsbs dataset
on a quick look, using a single shard to test we move from 1.4M datapoints/sec ( 140K ops/sec of TS.MADD with 10 points ) to 1.53M datapoints.
According to Eisel-Lemire ParseFloat algorithm numbers, we should see a 4X improvement on the string to double, which means we would move from 17% CPU time on conversion to 0.25*17%=4.25% on best case scenario. I'm seeing an real improvement of ~8-9% on ingestion (meaning we're at 2X improvement ) :