Skip to content

modelindexer optimisations#6408

Merged
axw merged 4 commits intoelastic:masterfrom
axw:modelindexer-optimisations
Oct 27, 2021
Merged

modelindexer optimisations#6408
axw merged 4 commits intoelastic:masterfrom
axw:modelindexer-optimisations

Conversation

@axw
Copy link
Copy Markdown
Member

@axw axw commented Oct 21, 2021

Motivation/summary

  • Use json-iterator for faster decoding of _bulk responses.
  • Use fastjson for faster (and fewer allocations) encoding of events for _bulk requests, and for encoding _bulk request metadata

We currently still use reflection for both encoding and decoding. We should generate code for both the response decoding and for encoding events and requests.

benchstat -delta-test=ttest /tmp/old.txt /tmp/new.txt 
name             old time/op    new time/op    delta
ModelIndexer-12    2.55µs ± 5%    2.03µs ±12%  -20.50%  (p=0.036 n=3+3)

name             old alloc/op   new alloc/op   delta
ModelIndexer-12    1.53kB ± 5%    1.31kB ± 1%  -14.15%  (p=0.031 n=3+3)

name             old allocs/op  new allocs/op  delta
ModelIndexer-12      12.0 ± 0%      10.0 ± 0%     ~     (zero variance)

Checklist

- [ ] Update CHANGELOG.asciidoc
- [ ] Documentation has been updated

How to test these changes

Non-functional change.

Related issues

#6360

@axw axw added the backport-skip Skip notification from the automated backport with mergify label Oct 21, 2021
@ghost
Copy link
Copy Markdown

ghost commented Oct 21, 2021

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview preview

Expand to view the summary

Build stats

  • Duration: 41 min 9 sec

🤖 GitHub comments

To re-run your PR in the CI, just comment with:

  • /test : Re-trigger the build.

  • /hey-apm : Run the hey-apm benchmark.

  • /package : Generate and publish the docker images.

@axw axw added the v8.0.0 label Oct 25, 2021
@axw axw force-pushed the modelindexer-optimisations branch from 227cf5d to f91955f Compare October 26, 2021 06:23
@axw axw requested a review from a team October 26, 2021 06:25
@axw axw marked this pull request as ready for review October 26, 2021 06:25
@axw axw merged commit caf6ecb into elastic:master Oct 27, 2021
@axw axw deleted the modelindexer-optimisations branch October 27, 2021 07:32
@axw axw added the backport-7.16 Automated backport with mergify to the 7.16 branch label Nov 16, 2021
mergify bot pushed a commit that referenced this pull request Nov 16, 2021
* modelindexer: optimise bulk response decoding

* modelindexer: use fastjson for encoding meta

* modelindexer: optimise event encoding

(cherry picked from commit caf6ecb)
@mergify mergify bot removed the backport-skip Skip notification from the automated backport with mergify label Nov 16, 2021
axw added a commit that referenced this pull request Nov 16, 2021
* modelindexer: optimise bulk response decoding

* modelindexer: use fastjson for encoding meta

* modelindexer: optimise event encoding

(cherry picked from commit caf6ecb)

Co-authored-by: Andrew Wilkins <axw@elastic.co>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backport-7.16 Automated backport with mergify to the 7.16 branch test-plan-skip v8.0.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants