[Cloud Security] POC - basic ui tests for Create Environment workflow#183801
[Cloud Security] POC - basic ui tests for Create Environment workflow#183801gurevichdmitry merged 17 commits intoelastic:mainfrom
Conversation
da36265 to
1d05710
Compare
1d05710 to
16f7fc6
Compare
kfirpeled
left a comment
There was a problem hiding this comment.
looks ok, but please refactor the settings into new files
| }, | ||
| }; | ||
| // If TEST_CLOUD environment varible is defined, return the configuration for cloud testing | ||
| if (process.env.TEST_CLOUD !== '1') { |
There was a problem hiding this comment.
instead of if statement please refactor the following into a new config file: config.cloud.ts
| loadTestFile(require.resolve('./findings_old_data')); | ||
| loadTestFile(require.resolve('./vulnerabilities')); | ||
| loadTestFile(require.resolve('./vulnerabilities_grouping')); | ||
| if (process.env.TEST_CLOUD === '1') { |
There was a problem hiding this comment.
instead of an if statement please refactor the following into a new file
kfirpeled
left a comment
There was a problem hiding this comment.
Besides removing it from running on every pull request, do you mind to enhance our documentation and share how to run these sanity tests?
Other then that, it looks pretty solid to me
Added a README file describing how to configure and execute tests, both for development purposes and on demand. |
|
@elasticmachine merge upstream |
opauloh
left a comment
There was a problem hiding this comment.
It looks good, just got some questions regarding the assertions
|
|
||
| it('displays correct number of resources evaluated', async () => { | ||
| const resourcesEvaluated = await dashboard.getKubernetesResourcesEvaluated(); | ||
| expect((await resourcesEvaluated.getVisibleText()) === '199').to.be(true); |
There was a problem hiding this comment.
So it's ensured these numbers won't change over time? otherwise, I would recommend changing to something like the previous test leaving some room for fluctuation:
expect(resourcesEvaluatedCount).greaterThan(150);
There was a problem hiding this comment.
@opauloh, since we are deploying the same resources, the number should not change. However, the timing of findings retrieval can affect this. Therefore, I have taken your suggestion and updated the test to allow some room for fluctuation
| it('displays accurate summary compliance score', async () => { | ||
| await pageObjects.header.waitUntilLoadingHasFinished(); | ||
| const scoreElement = await dashboard.getKubernetesComplianceScore(); | ||
| expect((await scoreElement.getVisibleText()) === '83%').to.be(true); |
There was a problem hiding this comment.
More like the previous question, will the agents be installed pointing to a controlled environment that the score doesn't (or at least shouldn't) change over time? otherwise, we could perform an assertion that are more flexible:
expect(score).greaterThan(80);The main reason is having to avoid having to update the tests in Kibana whenever these values change if the findings of the environments scanned are mutable
💛 Build succeeded, but was flaky
Failed CI StepsTest FailuresMetrics [docs]
History
To update your PR or re-run it, just comment with: |
Summary
This PR is essential for a Proof of Concept (POC) aimed at testing the feasibility of running UI tests written in the Kibana repository within a separate repository using GitHub Actions.