docs: refactor docs to enable the local development by mdelapenya · Pull Request #2454 · elastic/e2e-testing

mdelapenya · 2022-04-30T07:41:19Z

What does this PR do?

This PR adds a guide to create and setup AWS instance using the same deployment mechanism of the CI. For that we are adding make goals to:

setup your local machine to have python, ansible and all runtime dependencies, using a virtual env.
use ansible to create and configure AWS instances.
create and configure a stack node for the elastic-stack
destroy the elastic-stack
recreate the fleet-server
destroy the stack node
create and configure a test node
destroy the test node
ssh into both nodes
run the tests for a suite and/or for specific tags

Besides, we have added docs to improve the experience of running this from the local machine. The user will only need to export their AWS credentials, and the automation will create and configure everything.

Why is it important?

Local experience must be as close to the CI as possible, if not equals. With this, we are executing the same commands as the CI to create and configure AWS instances, and run the test scenarios, with the ability to filter the execution using tags.

We honestly expect this change makes an impact in how the developers reproduce errors, or even test their stuff.

Checklist

My code follows the style guidelines of this project
I have commented my code, particularly in hard-to-understand areas
I have made corresponding changes to the documentation
I have made corresponding change to the default configuration files
I have added tests that prove my fix is effective or that my feature works
I have run the Unit tests (make unit-test), and they are passing locally
I have run the End-2-End tests for the suite I'm working on, and they are passing locally
I have noticed new Go dependencies (run make notice in the proper directory)

Author's Checklist

I would need to test this on Windows / Linux

How to test this PR locally

Read the .ci/README.md file, and follow the instructions 😃

For example:

Add your github username to the github-ssh-keys file (could be done without committing the file)
export AWS settings and the SSH_KEY="$HOME/.ssh/my-github-key" used on Github, to enable SSH access to the machines
make -C .ci setup-env to prepare Ansible
make -C .ci create-stack to create and run the VM running the elastic-stack
make -C .ci create-node to create the VM to run the tests (default is Debian AMD)
make -C .ci run-tests TAGS="fleet_mode_agent && install" to run only the install scenario.
make -C .ci run-tests TAGS="system_integration && deploy-system_integration-with-core" ELASTIC_AGENT_VERSION=8.2-SNAPSHOT BEAT_VERSION=8.2-SNAPSHOT to run only the system integration scenarios for core metrics only, having the agent and any other beat in 8.2-SNAPSHOT versions

Follow-ups

Next steps would be having the CI pipeline using these new make goals

They will be used in the automation to locate the stack and node

.ci/README.md

Co-authored-by: Anderson Queiroz <me@andersonq.me> Co-authored-by: Mike Place <mike.place@elastic.co>

This will allow setting the agent version in lower ones: ELASTIC_AGENT_VERSION=8.1.3-SNAPSHOT make -C .ci create-node

It will allow to run those scenarios that do not need a remote stack, such as dockerised ones See apm-server feature file

* docs: update stack creation * fix: add default value for gitSha * docs: nodeUser too * chore: always use .venv * docs: make snippets copiable * docs: do not hardcode path to repo, use current dir * chore: add nodeUser to test nodes * chore: update nodeLabel to debian AMD * chore: consistency provisioning node tags * chore: use current dir for workspace * fix: check that suite is defined * chore: automate the creation of cloud VMs * chore: add runID as a tag in the VMs * chore: make exporting variables explicit * chore: reference the sshhosts files to get the IP addresses * fix: remove variable name from runID file * chore: store IP addresses in file They will be used in the automation to locate the stack and node * chore: support configuring the suite * chore: support destroying the stack and nodes * chore: remove node files * chore: remove wait for strategy on fleet-server * chore: command to recreate the stack * chore: simplify creation of the nodes in one single command * chore: support running the tests remotely * chore: pass TAGS as a variable * chore: configure tags for running the tests * docs: document how to run the tests in the VMs * chore: add goals to SSH into the machines * chore: add a command to destroy the stack * fix: typo * fix: conflicts in goal name * docs: document how to recreate the stack * docs: refactor docs * fix: pass bootstrap tag to the stack creation * fix: set the tags in the ansible code * fix: pass bootstrap tag to the stack creation * chore: apply suggestions from code review Co-authored-by: Anderson Queiroz <me@andersonq.me> Co-authored-by: Mike Place <mike.place@elastic.co> * fix: use proper hidden file name * docs: explain how to locate the feature files and tags * chore: check for runID where needed * chore: adds commands to inspect the nodes * fix: proper goal name * fix: lookup env This will allow setting the agent version in lower ones: ELASTIC_AGENT_VERSION=8.1.3-SNAPSHOT make -C .ci create-node * fix: close double quote * fix: goal name was incorrect * fix: bootstrap elastic stack properly * docs: reference to the environment variables affecting the build * chore: support configuring the remote It will allow to run those scenarios that do not need a remote stack, such as dockerised ones See apm-server feature file * chore: use the stack VM to enroll the stand-alone agent Co-authored-by: Anderson Queiroz <me@andersonq.me> Co-authored-by: Mike Place <mike.place@elastic.co> (cherry picked from commit 0995a83)

* docs: update stack creation * fix: add default value for gitSha * docs: nodeUser too * chore: always use .venv * docs: make snippets copiable * docs: do not hardcode path to repo, use current dir * chore: add nodeUser to test nodes * chore: update nodeLabel to debian AMD * chore: consistency provisioning node tags * chore: use current dir for workspace * fix: check that suite is defined * chore: automate the creation of cloud VMs * chore: add runID as a tag in the VMs * chore: make exporting variables explicit * chore: reference the sshhosts files to get the IP addresses * fix: remove variable name from runID file * chore: store IP addresses in file They will be used in the automation to locate the stack and node * chore: support configuring the suite * chore: support destroying the stack and nodes * chore: remove node files * chore: remove wait for strategy on fleet-server * chore: command to recreate the stack * chore: simplify creation of the nodes in one single command * chore: support running the tests remotely * chore: pass TAGS as a variable * chore: configure tags for running the tests * docs: document how to run the tests in the VMs * chore: add goals to SSH into the machines * chore: add a command to destroy the stack * fix: typo * fix: conflicts in goal name * docs: document how to recreate the stack * docs: refactor docs * fix: pass bootstrap tag to the stack creation * fix: set the tags in the ansible code * fix: pass bootstrap tag to the stack creation * chore: apply suggestions from code review Co-authored-by: Anderson Queiroz <me@andersonq.me> Co-authored-by: Mike Place <mike.place@elastic.co> * fix: use proper hidden file name * docs: explain how to locate the feature files and tags * chore: check for runID where needed * chore: adds commands to inspect the nodes * fix: proper goal name * fix: lookup env This will allow setting the agent version in lower ones: ELASTIC_AGENT_VERSION=8.1.3-SNAPSHOT make -C .ci create-node * fix: close double quote * fix: goal name was incorrect * fix: bootstrap elastic stack properly * docs: reference to the environment variables affecting the build * chore: support configuring the remote It will allow to run those scenarios that do not need a remote stack, such as dockerised ones See apm-server feature file * chore: use the stack VM to enroll the stand-alone agent Co-authored-by: Anderson Queiroz <me@andersonq.me> Co-authored-by: Mike Place <mike.place@elastic.co> (cherry picked from commit 0995a83) # Conflicts: # e2e/_suites/fleet/fleet.go

* main: docs: refactor docs to enable the local development (elastic#2454)

* main: docs: refactor docs to enable the local development (elastic#2454) bump stack version 8.3.0-a1c5cfff (elastic#2474) bump stack version 8.3.0-7f585873 (elastic#2469)

* docs: update stack creation * fix: add default value for gitSha * docs: nodeUser too * chore: always use .venv * docs: make snippets copiable * docs: do not hardcode path to repo, use current dir * chore: add nodeUser to test nodes * chore: update nodeLabel to debian AMD * chore: consistency provisioning node tags * chore: use current dir for workspace * fix: check that suite is defined * chore: automate the creation of cloud VMs * chore: add runID as a tag in the VMs * chore: make exporting variables explicit * chore: reference the sshhosts files to get the IP addresses * fix: remove variable name from runID file * chore: store IP addresses in file They will be used in the automation to locate the stack and node * chore: support configuring the suite * chore: support destroying the stack and nodes * chore: remove node files * chore: remove wait for strategy on fleet-server * chore: command to recreate the stack * chore: simplify creation of the nodes in one single command * chore: support running the tests remotely * chore: pass TAGS as a variable * chore: configure tags for running the tests * docs: document how to run the tests in the VMs * chore: add goals to SSH into the machines * chore: add a command to destroy the stack * fix: typo * fix: conflicts in goal name * docs: document how to recreate the stack * docs: refactor docs * fix: pass bootstrap tag to the stack creation * fix: set the tags in the ansible code * fix: pass bootstrap tag to the stack creation * chore: apply suggestions from code review Co-authored-by: Anderson Queiroz <me@andersonq.me> Co-authored-by: Mike Place <mike.place@elastic.co> * fix: use proper hidden file name * docs: explain how to locate the feature files and tags * chore: check for runID where needed * chore: adds commands to inspect the nodes * fix: proper goal name * fix: lookup env This will allow setting the agent version in lower ones: ELASTIC_AGENT_VERSION=8.1.3-SNAPSHOT make -C .ci create-node * fix: close double quote * fix: goal name was incorrect * fix: bootstrap elastic stack properly * docs: reference to the environment variables affecting the build * chore: support configuring the remote It will allow to run those scenarios that do not need a remote stack, such as dockerised ones See apm-server feature file * chore: use the stack VM to enroll the stand-alone agent Co-authored-by: Anderson Queiroz <me@andersonq.me> Co-authored-by: Mike Place <mike.place@elastic.co> (cherry picked from commit 0995a83) Co-authored-by: Manuel de la Peña <mdelapenya@gmail.com>

…pment (#2480) * docs: refactor docs to enable the local development (#2454) * docs: update stack creation * fix: add default value for gitSha * docs: nodeUser too * chore: always use .venv * docs: make snippets copiable * docs: do not hardcode path to repo, use current dir * chore: add nodeUser to test nodes * chore: update nodeLabel to debian AMD * chore: consistency provisioning node tags * chore: use current dir for workspace * fix: check that suite is defined * chore: automate the creation of cloud VMs * chore: add runID as a tag in the VMs * chore: make exporting variables explicit * chore: reference the sshhosts files to get the IP addresses * fix: remove variable name from runID file * chore: store IP addresses in file They will be used in the automation to locate the stack and node * chore: support configuring the suite * chore: support destroying the stack and nodes * chore: remove node files * chore: remove wait for strategy on fleet-server * chore: command to recreate the stack * chore: simplify creation of the nodes in one single command * chore: support running the tests remotely * chore: pass TAGS as a variable * chore: configure tags for running the tests * docs: document how to run the tests in the VMs * chore: add goals to SSH into the machines * chore: add a command to destroy the stack * fix: typo * fix: conflicts in goal name * docs: document how to recreate the stack * docs: refactor docs * fix: pass bootstrap tag to the stack creation * fix: set the tags in the ansible code * fix: pass bootstrap tag to the stack creation * chore: apply suggestions from code review Co-authored-by: Anderson Queiroz <me@andersonq.me> Co-authored-by: Mike Place <mike.place@elastic.co> * fix: use proper hidden file name * docs: explain how to locate the feature files and tags * chore: check for runID where needed * chore: adds commands to inspect the nodes * fix: proper goal name * fix: lookup env This will allow setting the agent version in lower ones: ELASTIC_AGENT_VERSION=8.1.3-SNAPSHOT make -C .ci create-node * fix: close double quote * fix: goal name was incorrect * fix: bootstrap elastic stack properly * docs: reference to the environment variables affecting the build * chore: support configuring the remote It will allow to run those scenarios that do not need a remote stack, such as dockerised ones See apm-server feature file * chore: use the stack VM to enroll the stand-alone agent Co-authored-by: Anderson Queiroz <me@andersonq.me> Co-authored-by: Mike Place <mike.place@elastic.co> (cherry picked from commit 0995a83) # Conflicts: # e2e/_suites/fleet/fleet.go * fix: resolve conflicts Co-authored-by: Manuel de la Peña <mdelapenya@gmail.com>

* docs: update stack creation * fix: add default value for gitSha * docs: nodeUser too * chore: always use .venv * docs: make snippets copiable * docs: do not hardcode path to repo, use current dir * chore: add nodeUser to test nodes * chore: update nodeLabel to debian AMD * chore: consistency provisioning node tags * chore: use current dir for workspace * fix: check that suite is defined * chore: automate the creation of cloud VMs * chore: add runID as a tag in the VMs * chore: make exporting variables explicit * chore: reference the sshhosts files to get the IP addresses * fix: remove variable name from runID file * chore: store IP addresses in file They will be used in the automation to locate the stack and node * chore: support configuring the suite * chore: support destroying the stack and nodes * chore: remove node files * chore: remove wait for strategy on fleet-server * chore: command to recreate the stack * chore: simplify creation of the nodes in one single command * chore: support running the tests remotely * chore: pass TAGS as a variable * chore: configure tags for running the tests * docs: document how to run the tests in the VMs * chore: add goals to SSH into the machines * chore: add a command to destroy the stack * fix: typo * fix: conflicts in goal name * docs: document how to recreate the stack * docs: refactor docs * fix: pass bootstrap tag to the stack creation * fix: set the tags in the ansible code * fix: pass bootstrap tag to the stack creation * chore: apply suggestions from code review Co-authored-by: Anderson Queiroz <me@andersonq.me> Co-authored-by: Mike Place <mike.place@elastic.co> * fix: use proper hidden file name * docs: explain how to locate the feature files and tags * chore: check for runID where needed * chore: adds commands to inspect the nodes * fix: proper goal name * fix: lookup env This will allow setting the agent version in lower ones: ELASTIC_AGENT_VERSION=8.1.3-SNAPSHOT make -C .ci create-node * fix: close double quote * fix: goal name was incorrect * fix: bootstrap elastic stack properly * docs: reference to the environment variables affecting the build * chore: support configuring the remote It will allow to run those scenarios that do not need a remote stack, such as dockerised ones See apm-server feature file * chore: use the stack VM to enroll the stand-alone agent Co-authored-by: Anderson Queiroz <me@andersonq.me> Co-authored-by: Mike Place <mike.place@elastic.co> (cherry picked from commit 0995a83) Co-authored-by: Manuel de la Peña <mdelapenya@gmail.com>

mdelapenya added 30 commits April 29, 2022 10:15

docs: update stack creation

c704e71

fix: add default value for gitSha

01ca4ff

docs: nodeUser too

3fb89f9

chore: always use .venv

f6166ca

docs: make snippets copiable

9ec86f8

docs: do not hardcode path to repo, use current dir

03e1d88

chore: add nodeUser to test nodes

ab3aa6c

chore: update nodeLabel to debian AMD

61d0358

chore: consistency provisioning node tags

4155b75

chore: use current dir for workspace

fbc90e7

fix: check that suite is defined

0102c36

chore: automate the creation of cloud VMs

1de8223

chore: add runID as a tag in the VMs

78e05f9

chore: make exporting variables explicit

1ed7a70

chore: reference the sshhosts files to get the IP addresses

0d1c661

fix: remove variable name from runID file

be57bd4

chore: store IP addresses in file

69439be

They will be used in the automation to locate the stack and node

chore: support configuring the suite

1587dbb

chore: support destroying the stack and nodes

dec1709

chore: remove node files

429b113

chore: remove wait for strategy on fleet-server

d1afaa2

chore: command to recreate the stack

f65a5e6

chore: simplify creation of the nodes in one single command

90d1478

chore: support running the tests remotely

3d38bac

chore: pass TAGS as a variable

e0d75a5

chore: configure tags for running the tests

24c41f7

docs: document how to run the tests in the VMs

46ea4ad

chore: add goals to SSH into the machines

078dd71

chore: add a command to destroy the stack

4a91f6e

fix: typo

674e1c1

mdelapenya commented May 2, 2022

View reviewed changes

.ci/README.md Outdated Show resolved Hide resolved

mdelapenya and others added 7 commits May 2, 2022 15:57

chore: apply suggestions from code review

15ed5d8

Co-authored-by: Anderson Queiroz <me@andersonq.me> Co-authored-by: Mike Place <mike.place@elastic.co>

fix: use proper hidden file name

a71ea70

docs: explain how to locate the feature files and tags

e6ec3e1

chore: check for runID where needed

cfe27e0

chore: adds commands to inspect the nodes

b371d8e

fix: proper goal name

74d308e

fix: lookup env

1dfb3fe

This will allow setting the agent version in lower ones: ELASTIC_AGENT_VERSION=8.1.3-SNAPSHOT make -C .ci create-node

cachedout approved these changes May 3, 2022

View reviewed changes

mdelapenya added 7 commits May 3, 2022 16:38

Merge branch 'main' into ansible-deployments

a03cebd

fix: close double quote

5fcfc47

fix: goal name was incorrect

07df9ee

fix: bootstrap elastic stack properly

fa329ed

docs: reference to the environment variables affecting the build

f1b4d94

chore: support configuring the remote

2b6e5f4

It will allow to run those scenarios that do not need a remote stack, such as dockerised ones See apm-server feature file

chore: use the stack VM to enroll the stand-alone agent

c756fb4

mdelapenya merged commit 0995a83 into elastic:main May 5, 2022

mergify bot mentioned this pull request May 5, 2022

[8.2](backport #2454) docs: refactor docs to enable the local development #2478

Merged

mergify bot mentioned this pull request May 5, 2022

[8.1](backport #2454) docs: refactor docs to enable the local development #2479

Merged

mergify bot mentioned this pull request May 5, 2022

[7.17](backport #2454) docs: refactor docs to enable the local development #2480

Merged

mdelapenya deleted the ansible-deployments branch May 5, 2022 14:00

mdelapenya added a commit to mdelapenya/e2e-testing that referenced this pull request May 5, 2022

Merge branch 'main' into fix-stale-agent-scenario

248d67b

* main: docs: refactor docs to enable the local development (elastic#2454)

mdelapenya mentioned this pull request May 6, 2022

chore: use build system to create the VMs on CI #2490

Merged

8 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

docs: refactor docs to enable the local development#2454

docs: refactor docs to enable the local development#2454
mdelapenya merged 50 commits intoelastic:mainfrom
mdelapenya:ansible-deployments

mdelapenya commented Apr 30, 2022 •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

mdelapenya commented Apr 30, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What does this PR do?

Why is it important?

Checklist

Author's Checklist

How to test this PR locally

Follow-ups

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

mdelapenya commented Apr 30, 2022 •

edited

Loading