Skip to content

Adding support for integration tests with remote cluster.#298

Merged
saratvemulapalli merged 1 commit intoopensearch-project:mainfrom
saratvemulapalli:main
Nov 11, 2021
Merged

Adding support for integration tests with remote cluster.#298
saratvemulapalli merged 1 commit intoopensearch-project:mainfrom
saratvemulapalli:main

Conversation

@saratvemulapalli
Copy link
Copy Markdown
Member

@saratvemulapalli saratvemulapalli commented Nov 10, 2021 •
edited
Loading

Signed-off-by: Sarat Vemulapalli vemulapallisarat@gmail.com

Description

Adding support for running integration tests with remote cluster.

Plugin integration tests depend on opensearch.testclusters gradle plugin exported by OpenSearch. Plugins use the same framework to run integration tests against a remote cluster. The problem is it internally downloads the right distribution to setup the test cluster even though it's not used.

Until now that worked since the min artifact is published for every release.
In order to step up our testing flow as part of the release, we now would like to run integration tests before the artifacts are published to verify the bundle via https://github.com/opensearch-project/opensearch-build#integration-tests. But since the artifact is not published the tests would see:

Before:

* What went wrong:
Execution failed for task ':integTest'.
> Could not resolve all files for configuration ':opensearch_distro_extracted_testclusters--integTest-0-1.1.0-'.
   > Could not resolve opensearch-distribution:opensearch:1.1.0.
     Required by:
         project :
      > Could not resolve opensearch-distribution:opensearch:1.1.0.
         > Could not get resource 'https://artifacts.opensearch.org/releases/core/opensearch/1.1.0/opensearch-min-1.1.0-linux-x64.tar.gz'.
            > Could not HEAD 'https://artifacts.opensearch.org/releases/core/opensearch/1.1.0/opensearch-min-1.1.0-linux-x64.tar.gz'. Received status code 403 from server: Forbidden
      > Could not resolve opensearch-distribution:opensearch:1.1.0.
         > Could not get resource 'https://artifacts.opensearch.org/release-candidates/core/opensearch/1.1.0/opensearch-min-1.1.0-linux-x64.tar.gz'.
            > Could not HEAD 'https://artifacts.opensearch.org/release-candidates/core/opensearch/1.1.0/opensearch-min-1.1.0-linux-x64.tar.gz'. Received status code 403 from server: Forbidden

After:

% ./gradlew integTestRemote -Dtests.rest.cluster=localhost:9200 -Dtests.cluster=localhost:9200 -Dtests.clustername="docker-cluster" -Dhttps=true -Duser=admin -Dpassword=admin
82 tests completed, 12 failed, 1 skipped

So this PR adds support to run integration tests for remote clusters without the min artifacts being published at artifacts.opensearch.org.

Issues Resolved

opensearch-project/OpenSearch#1240
opensearch-project/opensearch-plugins#101

Check List

  • Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@saratvemulapalli
Adding support for integration tests with remote cluster.
2f4f964 
Signed-off-by: Sarat Vemulapalli <vemulapallisarat@gmail.com>
@codecov-commenter
Copy link
Copy Markdown

codecov-commenter commented Nov 10, 2021 •
edited by codecov bot
Loading

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 75.93%. Comparing base (e10ce9d) to head (2f4f964).
⚠️ Report is 307 commits behind head on main.

Additional details and impacted files

Impacted file tree graph

@@             Coverage Diff              @@
##               main     #298      +/-   ##
============================================
- Coverage     76.32%   75.93%   -0.40%     
+ Complexity     3967     3950      -17     
============================================
  Files           295      295              
  Lines         17165    17165              
  Branches       1812     1812              
============================================
- Hits          13101    13034      -67     
- Misses         3238     3299      +61     
- Partials        826      832       +6
Flag Coverage Δ
plugin 75.93% <ø> (-0.40%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.
see 9 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@ohltyler
Copy link
Copy Markdown
Member

ohltyler commented Nov 10, 2021

@saratvemulapalli is it reasonable for this to be skipped for 1.2 specifically? Only concern is backporting to 1.2 branch can delay the bundle builds more.

@saratvemulapalli
Copy link
Copy Markdown
Member Author

saratvemulapalli commented Nov 10, 2021

@ohltyler how much of an effort/time is it to run the builds again?
This will help avoid running manual tests for the release for the plugin owners.
cc: @peternied @peterzhuamazon

@ohltyler
Copy link
Copy Markdown
Member

ohltyler commented Nov 10, 2021

@saratvemulapalli considering we already have bug fixes backported to 1.2 today, we can add this as well if it gets merged today. Changes make sense to me.

@peterzhuamazon
Copy link
Copy Markdown
Member

peterzhuamazon commented Nov 10, 2021

@ohltyler how much of an effort/time is it to run the builds again? This will help avoid running manual tests for the release for the plugin owners. cc: @peternied @peterzhuamazon

Build is automatic every 2 hours for around 20+10 = 30min for tar + docker.
So I think I agree with @ohltyler to approve and merge.

Thanks.

@saratvemulapalli saratvemulapalli merged commit e592d31 into opensearch-project:main Nov 11, 2021
@amitgalitz amitgalitz added infra Changes to infrastructure, testing, CI/CD, pipelines, etc. v1.2.0 labels Nov 11, 2021
This was referenced Nov 11, 2021
Closed
Closed
Closed
Closed
Closed
Closed
Closed
Closed
Closed
Closed
Closed
Closed
Closed
Closed
Closed
This was referenced Nov 11, 2021
Closed
Closed
saratvemulapalli added a commit that referenced this pull request Nov 11, 2021
@saratvemulapalli
Adding support for integration tests with remote cluster. (#298) (#312)
1258da5 
Signed-off-by: Sarat Vemulapalli <vemulapallisarat@gmail.com>
saratvemulapalli added a commit that referenced this pull request Nov 11, 2021
@saratvemulapalli
Adding support for integration tests with remote cluster. (#298) (#311)
3fad91d 
Signed-off-by: Sarat Vemulapalli <vemulapallisarat@gmail.com>
@saratvemulapalli saratvemulapalli mentioned this pull request Nov 16, 2021
Closed
@saratvemulapalli saratvemulapalli mentioned this pull request Jun 24, 2022
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@ohltyler ohltyler ohltyler approved these changes

@dblock dblock Awaiting requested review from dblock

@ylwu-amzn ylwu-amzn Awaiting requested review from ylwu-amzn ylwu-amzn is a code owner

+1 more reviewer

@peternied peternied peternied approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

infra Changes to infrastructure, testing, CI/CD, pipelines, etc. v1.2.0

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

@saratvemulapalli @codecov-commenter @ohltyler @peterzhuamazon @peternied @amitgalitz