Skip to content

[Cloud Security] Fix flaky test#183262

Merged
CohenIdo merged 5 commits intoelastic:mainfrom
CohenIdo:fix-flaky-test
May 15, 2024
Merged

[Cloud Security] Fix flaky test#183262
CohenIdo merged 5 commits intoelastic:mainfrom
CohenIdo:fix-flaky-test

Conversation

@CohenIdo
Copy link
Copy Markdown
Contributor

@CohenIdo CohenIdo commented May 13, 2024 •
edited
Loading

Solves:

The tests mentioned above failed with the following error message:


ResponseError: {"_shards":{"total":8,"successful":7,"failed":1,"failures":[{"shard":0,"index":".ds-metrics-fleet_server.agent_status-default-2024.03.05-000001","status":"NOT_FOUND","reason":{"type":"index_not_found_exception","reason":"no such index [.ds-metrics-fleet_server.agent_status-default...

I consulted with the fleet team and discovered that there is a task responsible for writing to the data stream named ds-metrics-fleet_server, and it was recommended to disable this task during FTRs.

After updating the test configuration to disable this task and re-enabling the previously skipped tests, I executed the tests using the flaky test runner, and it appears that this action has resolved the issue.

image

@CohenIdo CohenIdo added backport:skip This PR does not require backporting Team:Cloud Security Cloud Security team related labels May 13, 2024
@elasticmachine
Copy link
Copy Markdown
Contributor

elasticmachine commented May 13, 2024

Pinging @elastic/kibana-cloud-security-posture (Team:Cloud Security)

@CohenIdo CohenIdo added the release_note:skip Skip the PR/issue when compiling release notes label May 13, 2024
@CohenIdo CohenIdo linked an issue May 13, 2024 that may be closed by this pull request
Failing test: Serverless Security API Integration Tests.x-pack/test_serverless/api_integration/test_suites/security/cloud_security_posture/benchmark/v1·ts - Serverless security API cloud_security_posture GET /internal/cloud_security_posture/benchmark "before each" hook for "Should return non-empty array filled with Rules if user has CSP integrations" #182570
Closed
@CohenIdo CohenIdo requested a review from a team as a code owner May 13, 2024 15:38
This was linked to issues May 13, 2024
Failing test: Serverless Security API Integration Tests.x-pack/test_serverless/api_integration/test_suites/security/cloud_security_posture/status/status_indexed·ts - Serverless security API cloud_security_posture GET /internal/cloud_security_posture/status STATUS = INDEXED TEST "before each" hook for "Return kspm status indexed when logs-cloud_security_posture.findings_latest-default contains new kspm documents" #178027
Closed
Failing test: Serverless Security API Integration Tests.x-pack/test_serverless/api_integration/test_suites/security/cloud_security_posture/status/status_indexing·ts - Serverless security API cloud_security_posture GET /internal/cloud_security_posture/status STATUS = INDEXING TEST "before each" hook for "Return kspm status indexing when logs-cloud_security_posture.findings_latest-default doesn't contain new kspm documents, but has newly connected agents" #181777
Closed
@kfirpeled
Copy link
Copy Markdown
Contributor

kfirpeled commented May 13, 2024

@CohenIdo please elaborate on the ticket's description of what stands behind this fix. This will serve us in the future as a knowledge base

@CohenIdo
Copy link
Copy Markdown
Contributor Author

CohenIdo commented May 15, 2024

@elasticmachine merge upstream

@CohenIdo
Copy link
Copy Markdown
Contributor Author

CohenIdo commented May 15, 2024

@elasticmachine merge upstream

@kibana-ci
Copy link
Copy Markdown

kibana-ci commented May 15, 2024

💚 Build Succeeded

Metrics [docs]

✅ unchanged

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

@CohenIdo CohenIdo merged commit 433b67d into elastic:main May 15, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@JordanSh JordanSh JordanSh approved these changes

Assignees

No one assigned

Labels

backport:skip This PR does not require backporting release_note:skip Skip the PR/issue when compiling release notes Team:Cloud Security Cloud Security team related v8.15.0

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Failing test: Serverless Security API Integration Tests.x-pack/test_serverless/api_integration/test_suites/security/cloud_security_posture/benchmark/v1·ts - Serverless security API cloud_security_posture GET /internal/cloud_security_posture/benchmark "before each" hook for "Should return non-empty array filled with Rules if user has CSP integrations" Failing test: Serverless Security API Integration Tests.x-pack/test_serverless/api_integration/test_suites/security/cloud_security_posture/status/status_indexing·ts - Serverless security API cloud_security_posture GET /internal/cloud_security_posture/status STATUS = INDEXING TEST "before each" hook for "Return kspm status indexing when logs-cloud_security_posture.findings_latest-default doesn't contain new kspm documents, but has newly connected agents" Failing test: Serverless Security API Integration Tests.x-pack/test_serverless/api_integration/test_suites/security/cloud_security_posture/status/status_indexed·ts - Serverless security API cloud_security_posture GET /internal/cloud_security_posture/status STATUS = INDEXED TEST "before each" hook for "Return kspm status indexed when logs-cloud_security_posture.findings_latest-default contains new kspm documents"

6 participants

@CohenIdo @elasticmachine @kfirpeled @kibana-ci @JordanSh @kibanamachine