This pull request does not have a backport label. Could you fix it @ChrsMark? 🙏
To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

• backport-v./d./d./d is the label to automatically backport to the 7./d branch. /d is the digit
  NOTE: backport-skip has been added to this pull request.

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

💚 Flaky test report

Tests succeeded.

🤖 GitHub comments

To re-run your PR in the CI, just comment with:

• /test : Re-trigger the build.

The errors are still there. If you think you need to wait before executing certain code, the tests allow you to wait for a certain condition, using a retry with backoff strategy, or more hard, using a Sleep.

@mdelapenya I didn't have the time to deal with it today but the thing is that running the suite locally it doesn't fail specially with the change applied on this PR. I believe that the issue in general is that we don't properly wait until ES is started properly. Since it seems that the init container does not really help here, do we have any specific alternative leveraging the framework in order to ensure that ES is up and running before starting the scenario? Sth like:

Scenario: Logs collection from running pod
  Given "elastic-agent" is running with "logs generic"
    And "elasticsearch" is running
   When "a pod" is deployed
   Then "elastic-agent" collects events with "kubernetes.pod.name:a-pod"

do we have any specific alternative leveraging the framework in order to ensure that ES is up and running before starting the scenario? Sth like:

Scenario: Logs collection from running pod
  Given "elastic-agent" is running with "logs generic"
    And "elasticsearch" is running
   When "a pod" is deployed
   Then "elastic-agent" collects events with "kubernetes.pod.name:a-pod"

Let me think about it 🤔 I'll fetch this branch and try locally

I think you are right, we could decouple the elasticsearch k8s resources (configMap, deployment and service) and start them before the elastic agent is started.

I have two ideas:

1. rephrase your scenario like this (elasticsearch first):

Scenario: Logs collection from running pod
  Given "elasticsearch" is running
    And "elastic-agent" is running with "logs generic"
   When "a pod" is deployed
   Then "elastic-agent" collects events with "kubernetes.pod.name:a-pod"

2. set elasticsearch as a required resource, and install it in the ctx.BeforeSuite methods, right after initialising the kind cluster. I like this approach the most, as it reflects a real scenario where the elasticsearch instance does exist before any agent connected to it. OTOH, this corner case represents a good question for the agent team: what happens to the agent if the elasticsearch is not there on time?

Thanks for looking into this!

I think I had tried already the option 1 but with no success. I will try to give it another shot.
Regarding option 2: Will this mean that ES will be installed once in the startup of the suite? If this is true then the various scenarios will share the same ES and this might mean that the data will also be shared along the runs which is something that we don't want since we might find fields to be present from previous scenarios will the current scenario is actually failing. Is this concern valid?

We may need to rewrite the scenario and leverage the different life cycle hooks:

• move the elasticsearch installation to beforeSuite, so that it's started once
• make sure the elasticsearch indices of interest are clean in the afterScenario, so that we are not adding dirty data affecting other tests
• make sure the data is namespaced. Is it possible to rename the indices where the data is stored so that they not collide in future, possible parallel tests?

After checking all options I will go with the basic one for now which seems simpler. Changing the suite implementation seems to me a bit overkill since we are gonna to re-use the same utils that we already have and can just work well with the current implementation of the scenario. I still believe that we can reach deadlines if ES image takes too long to get downloaded but not sure how we can handle it better.

Great team work resolving the issue! Thanks for your time here

@Mergifyio refresh