Add CI for Windows and MacOS platforms by peternied · Pull Request #2190 · opensearch-project/security

peternied · 2022-10-24T22:46:00Z

Description

Add CI for Windows and MacOS platforms

Due to the increase in the number of platforms, I've separated the newer integration tests into their own workflow. Until retries have been enabled they will automatically pass - but still run and report logs. As soon as we have full confidence we will allow them to start blocking pull requests from merging. #2184

Switch the gradle commands to be platform agnostic via the gradle/gradle-build-action@v2, dropping the 'clean' step to the build which allows us to reuse the gradle cache - if we see any problems pulling in more recent snapshots we can disable this setting quickly.

Found and fixed an issued with config value replacement via environment variables, long story short Windows and MacOS allow for many more characters that are used in the unix environment variable landscape. Added new tests to cover these interesting scenarios as well.

Found an encoding issue with user names from config files, still unclear of the source of this issue, be it test setup specific or a problem in the broader OpenSearch ecosystem, disabling the testSpecialUsernames until we can dive deeper. #2194

Disabled the HeapBasedRateTrackerTests - it was depending on system timing and was very brittle if the system under test experienced any undo load, created follow up issue #2193

Fixed a test issue in testDlsWithMinDocCountZeroAggregations where there was a random chance for a test failure, easier to find intermittent tests when they are run so often.

OpenSSL has open questions - while it is supported for our Linux JDK11 builds, it seems like a stopgap measure. I've disabled the tests on windows environment while we determine if we should support OpenSSL at all on Windows JDK11. #2195

Issues Resolved

Closes Add Windows Support #1474

Check List

New functionality includes testing
~~New functionality has been documented~~
Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: Peter Nied <petern@amazon.com>

src/test/java/org/opensearch/security/ssl/OpenSSLTest.java

peternied · 2022-10-25T13:33:27Z

Signed-off-by: Peter Nied <petern@amazon.com>

codecov-commenter · 2022-10-25T21:52:53Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 61.04%. Comparing base (966b3fc) to head (7fe3292).
⚠️ Report is 1385 commits behind head on main.

Additional details and impacted files

@@             Coverage Diff              @@
##               main    #2190      +/-   ##
============================================
- Coverage     61.12%   61.04%   -0.08%     
- Complexity     3264     3268       +4     
============================================
  Files           259      259              
  Lines         18335    18335              
  Branches       3248     3248              
============================================
- Hits          11207    11193      -14     
- Misses         5537     5557      +20     
+ Partials       1591     1585       -6

Files with missing lines	Coverage Δ
...org/opensearch/security/support/SecurityUtils.java	`72.30% <100.00%> (ø)`

... and 8 files with indirect coverage changes

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Tracked with opensearch-project#2193 Signed-off-by: Peter Nied <petern@amazon.com>

opensearch-project#2194 Signed-off-by: Peter Nied <petern@amazon.com>

Signed-off-by: Peter Nied <petern@amazon.com>

opensearch-project#2195 Signed-off-by: Peter Nied <petern@amazon.com>

Signed-off-by: Peter Nied <petern@amazon.com>

peternied · 2022-10-26T13:11:44Z

Signed-off-by: Peter Nied <petern@amazon.com>

peternied · 2022-10-26T14:16:42Z

src/test/java/org/opensearch/security/dlic/dlsfls/DlsTest.java

        // Significant Text Aggregation is not impacted.
        // Non-admin user with setting "min_doc_count=0". Expected to only have access to buckets for dept_manager".
-        String query3 = "{\"aggregations\":{\"significant_termX\":{\"significant_terms\":{\"field\":\"termX.keyword\",\"min_doc_count\":0}}}}";
+        String query3 = "{\"size\":100,\"aggregations\":{\"significant_termX\":{\"significant_terms\":{\"field\":\"termX.keyword\",\"min_doc_count\":0}}}}";


This test inserts 17 documents and then does a query for matches on them checking the 'hits'. This hits have a default limit of 10... so this test was failing all the time because it depends if you got the right mix of 10/17 items in the output. By making this size much larger this should prevent this issue from happening again.

Always be careful with defaults!

src/test/java/org/opensearch/security/test/helper/file/FileHelper.java

src/test/java/org/opensearch/security/ssl/OpenSSLTest.java

Signed-off-by: Peter Nied <petern@amazon.com>

peternied · 2022-10-27T20:24:33Z

@opensearch-project/security This change is ready for review. Note; I assuming this change is approved I'll adjust the branch protection rules for the new matrix of builds

opensearch-trigger-bot · 2022-10-28T16:10:16Z

The backport to 2.x failed:

The process '/usr/bin/git' failed with exit code 1

To backport manually, run these commands in your terminal:

# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add .worktrees/backport-2.x 2.x
# Navigate to the new working tree
cd .worktrees/backport-2.x
# Create a new branch
git switch --create backport/backport-2190-to-2.x
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 a57fd0a745a398b1fb7ba005bda44d0e55f5fc88
# Push it to GitHub
git push --set-upstream origin backport/backport-2190-to-2.x
# Go back to the original working tree
cd ../..
# Delete the working tree
git worktree remove .worktrees/backport-2.x

Then, create a pull request where the base branch is 2.x and the compare/head branch is backport/backport-2190-to-2.x.

Add CI for Windows and MacOS platforms Due to the increase in the number of platforms, I've separated the newer integration tests into their own workflow. Until retries have been enabled they will automatically pass - but still run and report logs. As soon as we have full confidence we will allow them to start blocking pull requests from merging. #2184 Switch the gradle commands to be platform agnostic via the `gradle/gradle-build-action@v2`, dropping the 'clean' step to the build which allows us to reuse the gradle cache - if we see any problems pulling in more recent snapshots we can disable this setting quickly. Found and fixed an issued with config value replacement via environment variables, long story short Windows and MacOS allow for many more characters that are used in the unix environment variable landscape. Added new tests to cover these interesting scenarios as well. Found an encoding issue with user names from config files, still unclear of the source of this issue, be it test setup specific or a problem in the broader OpenSearch ecosystem, disabling the `testSpecialUsernames` until we can dive deeper. #2194 Disabled the HeapBasedRateTrackerTests - it was depending on system timing and was very brittle if the system under test experienced any undo load, created follow up issue #2193 Fixed a test issue in testDlsWithMinDocCountZeroAggregations where there was a random chance for a test failure, easier to find intermittent tests when they are run so often. OpenSSL has open questions - while it is supported for our Linux JDK11 builds, it seems like a stopgap measure. I've disabled the tests on windows environment while we determine if we should support OpenSSL at all on Windows JDK11. #2195 Signed-off-by: Peter Nied <petern@amazon.com> (cherry picked from commit a57fd0a)

peternied · 2022-10-28T17:05:46Z

Manually backporting in #2205

Add CI for Windows and MacOS platforms Due to the increase in the number of platforms, I've separated the newer integration tests into their own workflow. Until retries have been enabled they will automatically pass - but still run and report logs. As soon as we have full confidence we will allow them to start blocking pull requests from merging. #2184 Switch the gradle commands to be platform agnostic via the `gradle/gradle-build-action@v2`, dropping the 'clean' step to the build which allows us to reuse the gradle cache - if we see any problems pulling in more recent snapshots we can disable this setting quickly. Found and fixed an issued with config value replacement via environment variables, long story short Windows and MacOS allow for many more characters that are used in the unix environment variable landscape. Added new tests to cover these interesting scenarios as well. Found an encoding issue with user names from config files, still unclear of the source of this issue, be it test setup specific or a problem in the broader OpenSearch ecosystem, disabling the `testSpecialUsernames` until we can dive deeper. #2194 Disabled the HeapBasedRateTrackerTests - it was depending on system timing and was very brittle if the system under test experienced any undo load, created follow up issue #2193 Fixed a test issue in testDlsWithMinDocCountZeroAggregations where there was a random chance for a test failure, easier to find intermittent tests when they are run so often. OpenSSL has open questions - while it is supported for our Linux JDK11 builds, it seems like a stopgap measure. I've disabled the tests on windows environment while we determine if we should support OpenSSL at all on Windows JDK11. #2195 Signed-off-by: Peter Nied <petern@amazon.com> (cherry picked from commit a57fd0a)

…nsearch-project#2205) Add CI for Windows and MacOS platforms Due to the increase in the number of platforms, I've separated the newer integration tests into their own workflow. Until retries have been enabled they will automatically pass - but still run and report logs. As soon as we have full confidence we will allow them to start blocking pull requests from merging. opensearch-project#2184 Switch the gradle commands to be platform agnostic via the `gradle/gradle-build-action@v2`, dropping the 'clean' step to the build which allows us to reuse the gradle cache - if we see any problems pulling in more recent snapshots we can disable this setting quickly. Found and fixed an issued with config value replacement via environment variables, long story short Windows and MacOS allow for many more characters that are used in the unix environment variable landscape. Added new tests to cover these interesting scenarios as well. Found an encoding issue with user names from config files, still unclear of the source of this issue, be it test setup specific or a problem in the broader OpenSearch ecosystem, disabling the `testSpecialUsernames` until we can dive deeper. opensearch-project#2194 Disabled the HeapBasedRateTrackerTests - it was depending on system timing and was very brittle if the system under test experienced any undo load, created follow up issue opensearch-project#2193 Fixed a test issue in testDlsWithMinDocCountZeroAggregations where there was a random chance for a test failure, easier to find intermittent tests when they are run so often. OpenSSL has open questions - while it is supported for our Linux JDK11 builds, it seems like a stopgap measure. I've disabled the tests on windows environment while we determine if we should support OpenSSL at all on Windows JDK11. opensearch-project#2195 Signed-off-by: Peter Nied <petern@amazon.com> (cherry picked from commit a57fd0a)

Add CI for Windows and MacOS platforms Due to the increase in the number of platforms, I've separated the newer integration tests into their own workflow. Until retries have been enabled they will automatically pass - but still run and report logs. As soon as we have full confidence we will allow them to start blocking pull requests from merging. opensearch-project#2184 Switch the gradle commands to be platform agnostic via the `gradle/gradle-build-action@v2`, dropping the 'clean' step to the build which allows us to reuse the gradle cache - if we see any problems pulling in more recent snapshots we can disable this setting quickly. Found and fixed an issued with config value replacement via environment variables, long story short Windows and MacOS allow for many more characters that are used in the unix environment variable landscape. Added new tests to cover these interesting scenarios as well. Found an encoding issue with user names from config files, still unclear of the source of this issue, be it test setup specific or a problem in the broader OpenSearch ecosystem, disabling the `testSpecialUsernames` until we can dive deeper. opensearch-project#2194 Disabled the HeapBasedRateTrackerTests - it was depending on system timing and was very brittle if the system under test experienced any undo load, created follow up issue opensearch-project#2193 Fixed a test issue in testDlsWithMinDocCountZeroAggregations where there was a random chance for a test failure, easier to find intermittent tests when they are run so often. OpenSSL has open questions - while it is supported for our Linux JDK11 builds, it seems like a stopgap measure. I've disabled the tests on windows environment while we determine if we should support OpenSSL at all on Windows JDK11. opensearch-project#2195 Signed-off-by: Peter Nied <petern@amazon.com> Signed-off-by: Stephen Crawford <steecraw@amazon.com>

…nsearch-project#2205) Add CI for Windows and MacOS platforms Due to the increase in the number of platforms, I've separated the newer integration tests into their own workflow. Until retries have been enabled they will automatically pass - but still run and report logs. As soon as we have full confidence we will allow them to start blocking pull requests from merging. opensearch-project#2184 Switch the gradle commands to be platform agnostic via the `gradle/gradle-build-action@v2`, dropping the 'clean' step to the build which allows us to reuse the gradle cache - if we see any problems pulling in more recent snapshots we can disable this setting quickly. Found and fixed an issued with config value replacement via environment variables, long story short Windows and MacOS allow for many more characters that are used in the unix environment variable landscape. Added new tests to cover these interesting scenarios as well. Found an encoding issue with user names from config files, still unclear of the source of this issue, be it test setup specific or a problem in the broader OpenSearch ecosystem, disabling the `testSpecialUsernames` until we can dive deeper. opensearch-project#2194 Disabled the HeapBasedRateTrackerTests - it was depending on system timing and was very brittle if the system under test experienced any undo load, created follow up issue opensearch-project#2193 Fixed a test issue in testDlsWithMinDocCountZeroAggregations where there was a random chance for a test failure, easier to find intermittent tests when they are run so often. OpenSSL has open questions - while it is supported for our Linux JDK11 builds, it seems like a stopgap measure. I've disabled the tests on windows environment while we determine if we should support OpenSSL at all on Windows JDK11. opensearch-project#2195 Signed-off-by: Peter Nied <petern@amazon.com> (cherry picked from commit a57fd0a) Signed-off-by: Stephen Crawford <steecraw@amazon.com>

opensearch-trigger-bot · 2022-12-01T19:13:42Z

The backport to 1.3 failed:

The process '/usr/bin/git' failed with exit code 1

To backport manually, run these commands in your terminal:

# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add .worktrees/backport-1.3 1.3
# Navigate to the new working tree
cd .worktrees/backport-1.3
# Create a new branch
git switch --create backport/backport-2190-to-1.3
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 a57fd0a745a398b1fb7ba005bda44d0e55f5fc88
# Push it to GitHub
git push --set-upstream origin backport/backport-2190-to-1.3
# Go back to the original working tree
cd ../..
# Delete the working tree
git worktree remove .worktrees/backport-1.3

Then, create a pull request where the base branch is 1.3 and the compare/head branch is backport/backport-2190-to-1.3.

Add CI for Windows and MacOS platforms Due to the increase in the number of platforms, I've separated the newer integration tests into their own workflow. Until retries have been enabled they will automatically pass - but still run and report logs. As soon as we have full confidence we will allow them to start blocking pull requests from merging. opensearch-project#2184 Switch the gradle commands to be platform agnostic via the `gradle/gradle-build-action@v2`, dropping the 'clean' step to the build which allows us to reuse the gradle cache - if we see any problems pulling in more recent snapshots we can disable this setting quickly. Found and fixed an issued with config value replacement via environment variables, long story short Windows and MacOS allow for many more characters that are used in the unix environment variable landscape. Added new tests to cover these interesting scenarios as well. Found an encoding issue with user names from config files, still unclear of the source of this issue, be it test setup specific or a problem in the broader OpenSearch ecosystem, disabling the `testSpecialUsernames` until we can dive deeper. opensearch-project#2194 Disabled the HeapBasedRateTrackerTests - it was depending on system timing and was very brittle if the system under test experienced any undo load, created follow up issue opensearch-project#2193 Fixed a test issue in testDlsWithMinDocCountZeroAggregations where there was a random chance for a test failure, easier to find intermittent tests when they are run so often. OpenSSL has open questions - while it is supported for our Linux JDK11 builds, it seems like a stopgap measure. I've disabled the tests on windows environment while we determine if we should support OpenSSL at all on Windows JDK11. opensearch-project#2195 Signed-off-by: Peter Nied <petern@amazon.com>

…nsearch-project#2205) Add CI for Windows and MacOS platforms Due to the increase in the number of platforms, I've separated the newer integration tests into their own workflow. Until retries have been enabled they will automatically pass - but still run and report logs. As soon as we have full confidence we will allow them to start blocking pull requests from merging. opensearch-project#2184 Switch the gradle commands to be platform agnostic via the `gradle/gradle-build-action@v2`, dropping the 'clean' step to the build which allows us to reuse the gradle cache - if we see any problems pulling in more recent snapshots we can disable this setting quickly. Found and fixed an issued with config value replacement via environment variables, long story short Windows and MacOS allow for many more characters that are used in the unix environment variable landscape. Added new tests to cover these interesting scenarios as well. Found an encoding issue with user names from config files, still unclear of the source of this issue, be it test setup specific or a problem in the broader OpenSearch ecosystem, disabling the `testSpecialUsernames` until we can dive deeper. opensearch-project#2194 Disabled the HeapBasedRateTrackerTests - it was depending on system timing and was very brittle if the system under test experienced any undo load, created follow up issue opensearch-project#2193 Fixed a test issue in testDlsWithMinDocCountZeroAggregations where there was a random chance for a test failure, easier to find intermittent tests when they are run so often. OpenSSL has open questions - while it is supported for our Linux JDK11 builds, it seems like a stopgap measure. I've disabled the tests on windows environment while we determine if we should support OpenSSL at all on Windows JDK11. opensearch-project#2195 Signed-off-by: Peter Nied <petern@amazon.com> (cherry picked from commit a57fd0a)

peternied added 2 commits October 24, 2022 17:27

Add CI for Windows and MacOS platforms

c3995e7

Signed-off-by: Peter Nied <petern@amazon.com>

Remove OpenSSL tests with platform quirks

3fb3d1a

Signed-off-by: Peter Nied <petern@amazon.com>

stephen-crawford reviewed Oct 25, 2022

View reviewed changes

src/test/java/org/opensearch/security/ssl/OpenSSLTest.java Show resolved Hide resolved

Support for environment variables in other platforms

2a4ce68

Signed-off-by: Peter Nied <petern@amazon.com>

peternied force-pushed the more-platforms branch from 43fa3f5 to 2a4ce68 Compare October 25, 2022 20:05

peternied added 2 commits October 25, 2022 15:22

Handle dash character for MacOS

a44c85a

Signed-off-by: Peter Nied <petern@amazon.com>

Use lazy lookahead to find the env key name

f876447

Signed-off-by: Peter Nied <petern@amazon.com>

Disabling flaky tests in HeapBasedRateTrackerTest

4b31bca

Tracked with opensearch-project#2193 Signed-off-by: Peter Nied <petern@amazon.com>

peternied force-pushed the more-platforms branch from 2d66168 to 4b31bca Compare October 25, 2022 23:35

peternied added 5 commits October 25, 2022 18:41

Disable testSpecialUsernames broken on Windows

d753335

opensearch-project#2194 Signed-off-by: Peter Nied <petern@amazon.com>

Enable retries on integration tests

e704db9

Signed-off-by: Peter Nied <petern@amazon.com>

Disable OpenSSL on Windows and Mac

d9aa6cf

opensearch-project#2195 Signed-off-by: Peter Nied <petern@amazon.com>

Remove env variable for OpenSSL setup

d2a8952

Signed-off-by: Peter Nied <petern@amazon.com>

Fix checkstyle issue

49016cd

Signed-off-by: Peter Nied <petern@amazon.com>

peternied added 2 commits October 26, 2022 08:44

Move validation steps in separate jobs

c937165

Signed-off-by: Peter Nied <petern@amazon.com>

Protect against random failures due to document count

df79cef

Signed-off-by: Peter Nied <petern@amazon.com>

peternied force-pushed the more-platforms branch from cee4700 to df79cef Compare October 26, 2022 13:45

Ensure integrationTest is not run in primary workflow

dbe4669

Signed-off-by: Peter Nied <petern@amazon.com>

peternied commented Oct 26, 2022

View reviewed changes

peternied added 4 commits October 26, 2022 14:49

Verify that removing the clean step unblocks the tests

5c7123e

Signed-off-by: Peter Nied <petern@amazon.com>

Run all impacted tests only

97d1f77

Signed-off-by: Peter Nied <petern@amazon.com>

Merge remote-tracking branch 'origin/main' into more-platforms

5a7154f

Signed-off-by: Peter Nied <petern@amazon.com>

Verify SSL tests are in order

70a4cf3

Signed-off-by: Peter Nied <petern@amazon.com>

peternied mentioned this pull request Oct 26, 2022

Add Windows Support opensearch-project/security-dashboards-plugin#845

Closed

3 tasks

peternied added 3 commits October 26, 2022 16:28

Disable OpenSSL for only Windows

e167fd7

Signed-off-by: Peter Nied <petern@amazon.com>

Fix build break

221ece1

Signed-off-by: Peter Nied <petern@amazon.com>

Clean up for review readiness

5f8211c

Signed-off-by: Peter Nied <petern@amazon.com>

peternied marked this pull request as ready for review October 26, 2022 22:06

peternied requested a review from a team October 26, 2022 22:06

Remove testEnsureOpenSSLAvailabilit that wass unreliable

7fe3292

Signed-off-by: Peter Nied <petern@amazon.com>

peternied added backport 2.x v2.4.0 'Issues and PRs related to version v2.4.0' labels Oct 27, 2022

davidlago approved these changes Oct 28, 2022

View reviewed changes

peternied merged commit a57fd0a into opensearch-project:main Oct 28, 2022

peternied deleted the more-platforms branch October 28, 2022 16:40

peternied mentioned this pull request Oct 31, 2022

[Backport 2.x] Add CI for Windows and MacOS platforms (#2190) #2205

Merged

3 tasks

cwperks mentioned this pull request Nov 3, 2022

2.4 Release Notes for Security Plugin #2231

Merged

3 tasks

stephen-crawford added the backport 1.3 backport to 1.3 branch label Dec 1, 2022

stephen-crawford mentioned this pull request Dec 1, 2022

[BACKPORT] Backport Windows CI #2289

Closed

3 tasks

cwperks mentioned this pull request Dec 1, 2022

Backport Windows support to 1.x branch #2281

Closed

13 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add CI for Windows and MacOS platforms#2190

Add CI for Windows and MacOS platforms#2190
peternied merged 22 commits intoopensearch-project:mainfrom
peternied:more-platforms

peternied commented Oct 24, 2022 •

edited

Loading

Uh oh!

Uh oh!

peternied commented Oct 25, 2022 •

edited

Loading

Uh oh!

codecov-commenter commented Oct 25, 2022 •

edited by codecov bot

Loading

Uh oh!

peternied commented Oct 26, 2022 •

edited

Loading

Uh oh!

peternied Oct 26, 2022

Uh oh!

Uh oh!

Uh oh!

peternied commented Oct 27, 2022

Uh oh!

opensearch-trigger-bot bot commented Oct 28, 2022

Uh oh!

peternied commented Oct 28, 2022

Uh oh!

opensearch-trigger-bot bot commented Dec 1, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

peternied commented Oct 24, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Issues Resolved

Check List

Uh oh!

Uh oh!

peternied commented Oct 25, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

MacOS

Windows

Uh oh!

codecov-commenter commented Oct 25, 2022 • edited by codecov bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

peternied commented Oct 26, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Latest failures to investigate

JDK11

Linux

Windows

MacOS

JDK17

Linux

Windows

MacOS

Uh oh!

peternied Oct 26, 2022

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

peternied commented Oct 27, 2022

Uh oh!

opensearch-trigger-bot bot commented Oct 28, 2022

Uh oh!

peternied commented Oct 28, 2022

Uh oh!

opensearch-trigger-bot bot commented Dec 1, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

peternied commented Oct 24, 2022 •

edited

Loading

peternied commented Oct 25, 2022 •

edited

Loading

codecov-commenter commented Oct 25, 2022 •

edited by codecov bot

Loading

peternied commented Oct 26, 2022 •

edited

Loading