Skip to content

benchmark: Adds TF modules, benchmark workspace#7966

Merged
marclop merged 3 commits intoelastic:mainfrom
marclop:f/add-deployment-module-worker-module-benchmark-workspace
Apr 27, 2022
Merged

benchmark: Adds TF modules, benchmark workspace#7966
marclop merged 3 commits intoelastic:mainfrom
marclop:f/add-deployment-module-worker-module-benchmark-workspace

Conversation

@marclop
Copy link
Copy Markdown
Contributor

@marclop marclop commented Apr 26, 2022 •
edited
Loading

Motivation/summary

Adds two new modules under testing/infra/terraform/modules:

  • ec_deployment: Provisions a new deployment in ESS / ECE.
  • benchmark_executor: Provisions a VM in the configured region and
    uploads a compiled version of apmbench to it with an .envrc file
    with credentials to index data into a configured deployment.

Most of the codebase has been copied from ecetest and modified to work
in a standalone module.

Additionally, a new testing/benchmark workspace has been added which
facilitates using the two modules with light automation via make and
a few pre-configured settings.

The result is a folder which can be used by devs or automation to create
a configurable deployment (in ESS by default) which can be benchmarked
using apmbench running in a remote VM which can also be configured.

Checklist

- [ ] Update CHANGELOG.asciidoc
- [ ] Update package changelog.yml (only if changes to apmpackage have been made)

  • Documentation has been updated

How to test these changes

  1. Read the documentation.
  2. cd testing/benchmark && make
  3. make run-benchmark BENCHMARK_RUN=BenchmarkAgent BENCHMARK_TIME=2m

Related issues

@marclop
benchmark: Adds TF modules, benchmark workspace
aba3799 
Adds two new modules under `infra/terraform/modules`:

- `ec_deployment`: Provisions a new deployment in ESS / ECE.
- `benchmark_executor`: Provisions a VM in the configured region and
   uploads a compiled version of `apmbench` to it with an `.envrc` file
   with credentials to index data into a configured deployment.

Most of the codebase has been copied from `ecetest` and modified to work
in a standalone module.

Additionally, a new `testing/benchmark` workspace has been added which
facilitates using the two modules with light automation via `make` and
a few pre-configured settings.

The result is a folder which can be used by devs or automation to create
a configurable deployment (in ESS by default) which can be benchmarked
using `apmbench` running in a remote VM which can also be configured.

Signed-off-by: Marc Lopez Rubio <marc5.12@outlook.com>
@mergify
Copy link
Copy Markdown
Contributor

mergify bot commented Apr 26, 2022

This pull request does not have a backport label. Could you fix it @marclop? 🙏
To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

  • backport-7.x is the label to automatically backport to the 7.x branch.
  • backport-7./d is the label to automatically backport to the 7./d branch. /d is the digit

NOTE: backport-skip has been added to this pull request.

@mergify mergify bot added the backport-skip Skip notification from the automated backport with mergify label Apr 26, 2022
@ghost
Copy link
Copy Markdown

ghost commented Apr 26, 2022 •
edited by ghost
Loading

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview preview

Expand to view the summary

Build stats

  • Start Time: 2022-04-27T01:22:02.371+0000

  • Duration: 27 min 37 sec

Test stats 🧪

Test Results
Failed 0
Passed 3978
Skipped 13
Total 3991

🤖 GitHub comments

To re-run your PR in the CI, just comment with:

  • /test : Re-trigger the build.

  • /hey-apm : Run the hey-apm benchmark.

  • /package : Generate and publish the docker images.

  • /test windows : Build & tests on Windows.

  • run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

@marclop marclop marked this pull request as ready for review April 26, 2022 08:21
@marclop marclop requested a review from a team April 26, 2022 08:21
axw
axw approved these changes Apr 27, 2022
View reviewed changes
Copy link
Copy Markdown
Member

@axw axw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

Do you anticipate the TF modules being useful outside of benchmarking/performance testing? I wonder if we should move infra to testing/infra. Perhaps later we should also move apmbench so everything is close together.

Makefile Outdated

.PHONY: tf-docs
tf-docs: $(TERRAFORMDOCS)
@$(foreach dir, $(wildcard infra/terraform/modules/*),terraform-docs markdown --hide-empty --header-from header.md --output-file=README.md --output-mode replace $(dir);)
Copy link
Copy Markdown
Member

@axw axw Apr 27, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
@$(foreach dir, $(wildcard infra/terraform/modules/*),terraform-docs markdown --hide-empty --header-from header.md --output-file=README.md --output-mode replace $(dir);)
@$(foreach dir, $(wildcard infra/terraform/modules/*),$(TERRAFORMDOCS) markdown --hide-empty --header-from header.md --output-file=README.md --output-mode replace $(dir);)

I think?

Copy link
Copy Markdown
Member

@axw axw Apr 27, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe use a pattern rule?

Copy link
Copy Markdown
Contributor Author

@marclop marclop Apr 27, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed and added a pattern rule to make it more "make"

@marclop
Copy link
Copy Markdown
Contributor Author

marclop commented Apr 27, 2022

That's a good point. Initially it probably makes sense to move it within the testing/ tree and if we use it outside of testing / benchmarking then we can consider moving it out of there.

marclop added 2 commits April 27, 2022 09:21
@marclop
Address review comments
2cab780 
Signed-off-by: Marc Lopez Rubio <marc5.12@outlook.com>
@marclop
Merge remote-tracking branch 'upstream/main'
4b86124
@marclop marclop enabled auto-merge (squash) April 27, 2022 01:26
@ghost
Copy link
Copy Markdown

ghost commented Apr 27, 2022

🌐 Coverage report

Name Metrics % (covered/total) Diff
Packages 100.0% (43/43) 💚
Files 91.878% (181/197) 👍
Classes 93.407% (425/455) 👍
Methods 89.283% (1083/1213) 👍
Lines 77.382% (13151/16995) 👍 0.006
Conditionals 100.0% (0/0) 💚

@marclop marclop merged commit 3daf298 into elastic:main Apr 27, 2022
@marclop marclop deleted the f/add-deployment-module-worker-module-benchmark-workspace branch April 27, 2022 01:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@axw axw axw approved these changes

Assignees

No one assigned

Labels

backport-skip Skip notification from the automated backport with mergify benchmarking enhancement test-plan v8.3.0

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

benchmark: Create terraform module that allows the creation of apmbench "workers" benchmark: Create APM Server deployment terraform module

3 participants

@marclop @axw @simitt