Skip to content

Auto-convert V6 configuration instances into V7 configuration instances (for OpenSearch 2.x only)#4753

Merged
cwperks merged 9 commits intoopensearch-project:2.xfrom
nibix:config-improvements-2.x
Oct 4, 2024
Merged

Auto-convert V6 configuration instances into V7 configuration instances (for OpenSearch 2.x only)#4753
cwperks merged 9 commits intoopensearch-project:2.xfrom
nibix:config-improvements-2.x

Conversation

@nibix
Copy link
Copy Markdown
Collaborator

@nibix nibix commented Sep 24, 2024 •
edited
Loading

Description

This is secondary to #4546. Please review #4546 first.

This relates to the proposal "Behind the scenes conversion from V6 to V7" from #4493.
This converts the V6 config instances to V7 config instances "behind the scenes". This allows to save a lot of duplicate implementation regarding to privilege evaluation and possibly also further logic.

As an additional benefit, this makes it possible to use SecurityDynamicConfiguration<> instances in a generically type-safe manner. So far, one usually needs to use SecurityDynamicConfiguration<?> instances and do unsafe casts.

This only touches the runtime representation of the configuration. If there's a V6 configuration in the configuration index, it remains untouched. The possibility to update configuration in V6 format will remain.

This is only for OpenSearch 2.x. For the OpenSearch main branch, #4546 removes V6 config support alltogether.

Behind the scenes conversion

Some more detail on what is changed by this PR:

The configuration architecture consists of these levels:

  1. JSON documents in the configuration index
  2. ConfigurationRepository which manages SecurityDynamicConfiguration<> instances which contain as entry instances of org.opensearch.security.securityconf.impl.v6.RoleV6, org.opensearch.security.securityconf.impl.v6.RoleMappingsV6, org.opensearch.security.securityconf.impl.v7.RoleV7, org.opensearch.security.securityconf.impl.v7.RoleMappingsV7, etc.
  3. DynamicConfigFactory which consumes SecurityDynamicConfiguration<> instances and creates ConfigModel and DynamicConfigModel instances. If it finds V6 classes in SecurityDynamicConfiguration<>, it creates ConfigModelV6 instances, if it finds V7 classes, it creates ConfigModelV7 instances.
  4. The ConfigModelV6 and ConfigModelV7 classes contain business logic which evaluates roles and permissions.
  5. This is used by PrivilegesEvaluator.

This PR changes the architecture at level 2:

  1. The JSON documents in the index do not change
  2. ConfigurationRepository is changed in this way: If it finds V6 instances like org.opensearch.security.securityconf.impl.v6.RoleV6, it auto-converts these to V7 instances like org.opensearch.security.securityconf.impl.v7.RoleV7. The V7 classes already have functionality for that which is already used for the securityadmin migration command.
  3. This makes the distinction of V6/V7 on all layers below unnecessary. DynamicConfigFactory does no longer need a distinction between two config versions.
  4. ConfigModelV6 becomes redundant.
  5. As PrivilegeEvaluator consumes the super interface ConfigModel, it does not notice anything about this change.

  • Category: Refactoring

  • Why these changes are required? - Minimize duplicate logic for privilege evaluation

  • What is the old behavior before changes and new behavior after changes? - none

  • Category: Refactoring

  • Why these changes are required? - Minimize duplicate logic for privilege evaluation

  • What is the old behavior before changes and new behavior after changes? no behavior changes

Issues Resolved

Testing

  • New int test LegacyConfigV6AutoConversionTest

Check List

  • New functionality includes testing
  • Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@nibix nibix mentioned this pull request Sep 24, 2024
Merged
1 task
@nibix
V6 configurations now get auto-converted into V7 configurations
ed0a7d2 
Signed-off-by: Nils Bandener <nils.bandener@eliatra.com>
@nibix
Delete rebase error
8b3fc4c 
Signed-off-by: Nils Bandener <nils.bandener@eliatra.com>
@nibix
Fixes
b43655c 
Signed-off-by: Nils Bandener <nils.bandener@eliatra.com>
@nibix
Fixes
51dbfb5 
Signed-off-by: Nils Bandener <nils.bandener@eliatra.com>
@nibix nibix force-pushed the config-improvements-2.x branch from 59758a7 to 51dbfb5 Compare September 25, 2024 05:27
@nibix
Removed unnecessary stubbings
50bc999 
Signed-off-by: Nils Bandener <nils.bandener@eliatra.com>
@nibix
Fixes
35481aa 
Signed-off-by: Nils Bandener <nils.bandener@eliatra.com>
@nibix nibix marked this pull request as ready for review September 25, 2024 06:19
@nibix
Deleted leftovers
ade7c8a 
Signed-off-by: Nils Bandener <nils.bandener@eliatra.com>
@codecov
Copy link
Copy Markdown

codecov bot commented Sep 25, 2024 •
edited
Loading

Codecov Report

Attention: Patch coverage is 72.84345% with 85 lines in your changes missing coverage. Please review.

Project coverage is 64.61%. Comparing base (f353b0e) to head (ab9a982).
Report is 127 commits behind head on 2.x.

Files with missing lines Patch % Lines
...ch/security/securityconf/DynamicConfigFactory.java 60.00% 9 Missing and 7 partials ⚠️
...ecurityconf/impl/SecurityDynamicConfiguration.java 77.41% 9 Missing and 5 partials ⚠️
...earch/security/configuration/ConfigurationMap.java 67.74% 7 Missing and 3 partials ⚠️
...rg/opensearch/security/securityconf/Migration.java 67.85% 8 Missing and 1 partial ⚠️
...a/org/opensearch/security/tools/SecurityAdmin.java 0.00% 8 Missing ⚠️
...g/opensearch/security/securityconf/impl/CType.java 89.85% 5 Missing and 2 partials ⚠️
...n/java/org/opensearch/security/tools/Migrater.java 0.00% 6 Missing ⚠️
...ty/configuration/ConfigurationLoaderSecurity7.java 66.66% 4 Missing and 1 partial ⚠️
...a/org/opensearch/security/DefaultObjectMapper.java 33.33% 3 Missing and 1 partial ⚠️
...ecurity/configuration/ConfigurationRepository.java 80.95% 2 Missing and 2 partials ⚠️
... and 1 more
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##              2.x    #4753      +/-   ##
==========================================
- Coverage   65.38%   64.61%   -0.78%     
==========================================
  Files         317      320       +3     
  Lines       22182    22357     +175     
  Branches     3584     3603      +19     
==========================================
- Hits        14504    14445      -59     
- Misses       5881     6140     +259     
+ Partials     1797     1772      -25
Files with missing lines Coverage Δ
...arch/security/dlic/rest/api/AbstractApiAction.java 82.81% <100.00%> (-0.14%) ⬇️
...earch/security/dlic/rest/api/AccountApiAction.java 60.56% <ø> (ø)
.../security/dlic/rest/api/ActionGroupsApiAction.java 78.18% <ø> (ø)
...rch/security/dlic/rest/api/AllowlistApiAction.java 89.47% <ø> (ø)
...nsearch/security/dlic/rest/api/AuditApiAction.java 90.76% <ø> (ø)
...curity/dlic/rest/api/AuthTokenProcessorAction.java 90.90% <ø> (ø)
.../security/dlic/rest/api/CertificatesApiAction.java 35.13% <ø> (ø)
...security/dlic/rest/api/ConfigUpgradeApiAction.java 72.97% <100.00%> (ø)
...ch/security/dlic/rest/api/FlushCacheApiAction.java 65.38% <ø> (ø)
...security/dlic/rest/api/InternalUsersApiAction.java 76.85% <ø> (ø)
... and 29 more

... and 12 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

cwperks
cwperks reviewed Oct 2, 2024
View reviewed changes
Copy link
Copy Markdown
Member

@cwperks cwperks left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @nibix! Left a few comments. This change looks good to me. This will greatly simplify the security codebase to only have to keep track of the v7 models.

nibix added 2 commits October 2, 2024 23:45
@nibix
Formatting fix
f8d31aa 
Signed-off-by: Nils Bandener <nils.bandener@eliatra.com>
@nibix
Removed leftover
ab9a982 
Signed-off-by: Nils Bandener <nils.bandener@eliatra.com>
@cwperks
Copy link
Copy Markdown
Member

cwperks commented Oct 4, 2024

@DarshitChanpura @willyborankin @derek-ho @peternied Could we get one more review on this? This change helps pave the way forward for Optimized Privileges Evaluation as it simplifies the code greatly to only depend on one version of the models.

@cwperks cwperks merged commit fbe2dd3 into opensearch-project:2.x Oct 4, 2024
@cwperks cwperks mentioned this pull request Mar 19, 2025
Closed
@cwperks cwperks added the v2.18.0 Issues targeting release v2.18.0 label Mar 20, 2025
@cwperks cwperks mentioned this pull request Mar 20, 2025
Merged
5 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@willyborankin willyborankin willyborankin approved these changes

@cwperks cwperks cwperks approved these changes

@DarshitChanpura DarshitChanpura Awaiting requested review from DarshitChanpura DarshitChanpura is a code owner

@peternied peternied Awaiting requested review from peternied

@RyanL1997 RyanL1997 Awaiting requested review from RyanL1997 RyanL1997 is a code owner

@stephen-crawford stephen-crawford Awaiting requested review from stephen-crawford

@reta reta Awaiting requested review from reta reta is a code owner

Assignees

No one assigned

Labels

v2.18.0 Issues targeting release v2.18.0

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@nibix @cwperks @willyborankin