Skip to content

test/erasure-code: increase ceph_erasure_code_benchmark default buffe…#59486

Merged
SrinivasaBharath merged 1 commit intomainfrom
ceph-ec-benchmark-increase-default-buffer-size
Nov 4, 2024
Merged

test/erasure-code: increase ceph_erasure_code_benchmark default buffe…#59486
SrinivasaBharath merged 1 commit intomainfrom
ceph-ec-benchmark-increase-default-buffer-size

Conversation

@jamiepryde
Copy link
Contributor

@jamiepryde jamiepryde commented Aug 28, 2024

…r size

The ceph_erasure_code_benchmark tool and bench.sh scripts use small default buffer sizes, and also contain comments that suggest running the tool with a buffer size that is too small. It can be observed from flame graphs captured while using the tool that a majority of CPU time is spent allocating, freeing and aligning buffers, rather than testing the performance of the erasure code plugin.

This PR updates the default settings and comments to suggest a far larger buffer size and far fewer number of encode/decode iterations. Flame graphs show that the vast majority of CPU time is now spent encoding and decoding, rather than aligning buffers.

Contribution Guidelines

  • To sign and title your commits, please refer to Submitting Patches to Ceph.

  • If you are submitting a fix for a stable branch (e.g. "quincy"), please refer to Submitting Patches to Ceph - Backports for the proper workflow.

  • When filling out the below checklist, you may click boxes directly in the GitHub web UI. When entering or editing the entire PR message in the GitHub web UI editor, you may also select a checklist item by adding an x between the brackets: [x]. Spaces and capitalization matter when checking off items this way.

Checklist

  • Tracker (select at least one)
    • References tracker ticket
    • Very recent bug; references commit where it was introduced
    • New feature (ticket optional)
    • Doc update (no ticket needed)
    • Code cleanup (no ticket needed)
  • Component impact
    • Affects Dashboard, opened tracker ticket
    • Affects Orchestrator, opened tracker ticket
    • No impact that needs to be tracked
  • Documentation (select at least one)
    • Updates relevant documentation
    • No doc update is appropriate
  • Tests (select at least one)
Show available Jenkins commands
  • jenkins retest this please
  • jenkins test classic perf
  • jenkins test crimson perf
  • jenkins test signed
  • jenkins test make check
  • jenkins test make check arm64
  • jenkins test submodules
  • jenkins test dashboard
  • jenkins test dashboard cephadm
  • jenkins test api
  • jenkins test docs
  • jenkins render docs
  • jenkins test ceph-volume all
  • jenkins test ceph-volume tox
  • jenkins test windows
  • jenkins test rook e2e

…r size

Signed-off-by: Jamie Pryde <jamiepry@uk.ibm.com>
@jamiepryde jamiepryde self-assigned this Aug 28, 2024
@jamiepryde jamiepryde requested a review from a team as a code owner August 28, 2024 16:17
@jamiepryde
Copy link
Contributor Author

Here are some flamegraphs showing CPU utilisation for the bench script run with suggested value:

#  TOTAL_SIZE=$((4 * 1024 * 1024 * 1024)) \
#  CEPH_ERASURE_CODE_BENCHMARK=src/ceph_erasure_code_benchmark  \
#  PLUGIN_DIRECTORY=build/lib \
#      qa/workunits/erasure-code/bench.sh fplot jerasure |
#      tee qa/workunits/erasure-code/bench.js

image

ISA vandermonde encode - ~15% gf encode functions, ~55% aligning buffers, ~20% freeing buffers

image

Jerasure vandermonde encode

image

Bench results

@jamiepryde
Copy link
Contributor Author

And here they are with the new suggested values:

#  TOTAL_SIZE=$((100 * 80 * 1024 * 1024)) SIZE=$((80 * 1024 * 1024)) \
#  CEPH_ERASURE_CODE_BENCHMARK=build/bin/ceph_erasure_code_benchmark  \
#  PLUGIN_DIRECTORY=build/lib \
#      qa/workunits/erasure-code/bench.sh fplot jerasure |
#      tee qa/workunits/erasure-code/bench.js
image

ISA vandermonde encode

image

Jerasure vandermonde encode

image

Bench results.

It's worth noting that with these benchmark settings, we see a far bigger performance improvement when using ISA compared to Jerasure.

@jamiepryde
Copy link
Contributor Author

jenkins test api

@jamiepryde
Copy link
Contributor Author

jenkins test make check

1 similar comment
@jamiepryde
Copy link
Contributor Author

jenkins test make check

Copy link
Member

@markhpc markhpc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@jamiepryde
Copy link
Contributor Author

jenkins test make check

@jamiepryde
Copy link
Contributor Author

jenkins test make check

2 similar comments
@jamiepryde
Copy link
Contributor Author

jenkins test make check

@jamiepryde
Copy link
Contributor Author

jenkins test make check

@Naveenaidu
Copy link
Contributor

@SrinivasaBharath SrinivasaBharath merged commit 28887e2 into main Nov 4, 2024
@SrinivasaBharath SrinivasaBharath deleted the ceph-ec-benchmark-increase-default-buffer-size branch November 4, 2024 07:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants