[2.19] Add runtimePermission for localeServiceProvider to fix docker startup issue in 2.19.5 by cwperks · Pull Request #20786 · opensearch-project/OpenSearch

cwperks · 2026-03-05T21:28:10Z

Description

Add runtimePermission for localeServiceProvider to fix docker startup issue in 2.19.5

Note: I haven't been able to reproduce this on my mac and plan to try on my dev desktop as well.

Current theory is that it could be related to -Djava.locale.providers=SPI,COMPAT system prop but not exactly sure what changed from 2.19.4 to 2.19.5 to introduce this exception on bootup.

Stack Trace:

Caused by: java.security.AccessControlException: access denied ("java.lang.RuntimePermission" "localeServiceProvider")
	at java.base/java.security.AccessControlContext.checkPermission(AccessControlContext.java:488)
	at java.base/java.security.AccessController.checkPermission(AccessController.java:1071)
	at java.base/java.lang.SecurityManager.checkPermission(SecurityManager.java:411)
	at java.base/java.util.spi.LocaleServiceProvider.checkPermission(LocaleServiceProvider.java:161)
	at java.base/java.util.spi.LocaleServiceProvider.<init>(LocaleServiceProvider.java:175)
	at java.base/java.util.spi.CalendarDataProvider.<init>(CalendarDataProvider.java:45)
	at org.opensearch.common.time.IsoCalendarDataProvider.<init>(IsoCalendarDataProvider.java:47)
	at java.base/jdk.internal.reflect.DirectConstructorHandleAccessor.newInstance(DirectConstructorHandleAccessor.java:62)
	at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:502)
	at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:486)
	at java.base/java.util.ServiceLoader$ProviderImpl$2.run(ServiceLoader.java:797)
	at java.base/java.security.AccessController.doPrivileged(AccessController.java:714)
	at java.base/java.util.ServiceLoader$ProviderImpl.newInstance(ServiceLoader.java:802)
	... 44 more
java.lang.IllegalStateException: failed to load plugin class [org.opensearch.security.OpenSearchSecurityPlugin]
Likely root cause: java.security.AccessControlException: access denied ("java.lang.RuntimePermission" "localeServiceProvider")
	at java.base/java.security.AccessControlContext.checkPermission(AccessControlContext.java:488)
	at java.base/java.security.AccessController.checkPermission(AccessController.java:1071)
	at java.base/java.lang.SecurityManager.checkPermission(SecurityManager.java:411)
	at java.base/java.util.spi.LocaleServiceProvider.checkPermission(LocaleServiceProvider.java:161)
	at java.base/java.util.spi.LocaleServiceProvider.<init>(LocaleServiceProvider.java:175)
	at java.base/java.util.spi.CalendarDataProvider.<init>(CalendarDataProvider.java:45)
	at org.opensearch.common.time.IsoCalendarDataProvider.<init>(IsoCalendarDataProvider.java:47)
	at java.base/jdk.internal.reflect.DirectConstructorHandleAccessor.newInstance(DirectConstructorHandleAccessor.java:62)
	at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:502)
	at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:486)
	at java.base/java.util.ServiceLoader$ProviderImpl$2.run(ServiceLoader.java:797)
	at java.base/java.security.AccessController.doPrivileged(AccessController.java:714)
	at java.base/java.util.ServiceLoader$ProviderImpl.newInstance(ServiceLoader.java:802)
	at java.base/java.util.ServiceLoader$ProviderImpl.get(ServiceLoader.java:729)
	at java.base/java.util.ServiceLoader$3.next(ServiceLoader.java:1403)
	at java.base/sun.util.locale.provider.SPILocaleProviderAdapter$1.run(SPILocaleProviderAdapter.java:83)
	at java.base/sun.util.locale.provider.SPILocaleProviderAdapter$1.run(SPILocaleProviderAdapter.java:76)
	at java.base/java.security.AccessController.doPrivileged(AccessController.java:571)
	at java.base/sun.util.locale.provider.SPILocaleProviderAdapter.findInstalledProvider(SPILocaleProviderAdapter.java:76)
	at java.base/sun.util.locale.provider.AuxLocaleProviderAdapter.getLocaleServiceProvider(AuxLocaleProviderAdapter.java:73)
	at java.base/sun.util.locale.provider.LocaleServiceProviderPool.findProviders(LocaleServiceProviderPool.java:313)
	at java.base/sun.util.locale.provider.LocaleServiceProviderPool.getLocalizedObjectImpl(LocaleServiceProviderPool.java:283)
	at java.base/sun.util.locale.provider.LocaleServiceProviderPool.getLocalizedObject(LocaleServiceProviderPool.java:265)
	at java.base/sun.util.locale.provider.CalendarDataUtility.retrieveFirstDayOfWeek(CalendarDataUtility.java:76)
	at java.base/java.util.Calendar.setWeekCountData(Calendar.java:3413)
	at java.base/java.util.Calendar.<init>(Calendar.java:1612)
	at java.base/java.util.GregorianCalendar.<init>(GregorianCalendar.java:741)
	at java.base/java.util.Calendar$Builder.build(Calendar.java:1497)
	at java.base/sun.util.locale.provider.CalendarProviderImpl.getInstance(CalendarProviderImpl.java:87)
	at java.base/java.util.Calendar.createCalendar(Calendar.java:1700)
	<<<truncated>>>
For complete error details, refer to the log at /usr/share/opensearch/logs/docker-cluster.log

Check List

Functionality includes testing.
API changes companion pull request created, if applicable.
Public documentation issue/PR created, if applicable.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

…startup issue in 2.19.5 Signed-off-by: Craig Perkins <cwperx@amazon.com>

github-actions · 2026-03-05T21:28:44Z

PR Code Analyzer ❗

AI-powered 'Code-Diff-Analyzer' found issues on commit da52269.

Path	Line	Severity	Description
server/src/main/resources/org/opensearch/bootstrap/security.policy	201	low	Addition of 'localeServiceProvider' RuntimePermission to the global grant block. This permission allows code to register as a Java locale service provider (via java.util.spi.LocaleServiceProvider), which could theoretically be used to intercept or manipulate locale-sensitive operations (e.g., string formatting, date/number parsing). However, this is a standard Java i18n permission with no direct malicious utility on its own, and the change appears consistent with adding internationalization support. No evidence of malicious intent, but the necessity of this permission should be verified against actual code that requires it.

The table above displays the top 10 most important findings.

Total: 1 | Critical: 0 | High: 0 | Medium: 0 | Low: 1

Pull Requests Author(s): Please update your Pull Request according to the report above.

Repository Maintainer(s): You can bypass diff analyzer by adding label skip-diff-analyzer after reviewing the changes carefully, then re-run failed actions. To re-enable the analyzer, remove the label, then re-run all actions.

⚠️ Note: The Code-Diff-Analyzer helps protect against potentially harmful code patterns. Please ensure you have thoroughly reviewed the changes beforehand.

Thanks.

github-actions · 2026-03-05T21:29:25Z

PR Reviewer Guide 🔍

(Review updated until commit `735a494`)

Here are some key observations to aid the review process:

🧪 No relevant tests
🔒 Security concerns Permission escalation: The `localeServiceProvider` RuntimePermission is added to the global `grant` block, which applies to all code without restriction. While this fixes the immediate startup issue, granting broad runtime permissions globally could weaken the security sandbox. It should be confirmed that this permission cannot be exploited by untrusted code and that a more targeted grant (scoped to a specific codebase) is not feasible.
✅ No TODO sections
🔀 No multiple PR themes
⚡ Recommended focus areas for review Permission Scope The `localeServiceProvider` RuntimePermission is being granted globally in the `grant` block without any codebase restriction. This means all code running in the JVM will have this permission. It should be verified whether this permission can be scoped to a specific codebase (e.g., the OpenSearch security plugin or the specific class `IsoCalendarDataProvider`) rather than granted universally, to follow the principle of least privilege. permission java.lang.RuntimePermission "localeServiceProvider";

github-actions · 2026-03-05T22:06:19Z

❌ Gradle check result for da52269: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions · 2026-03-06T00:19:27Z

❌ Gradle check result for da52269: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions · 2026-03-06T01:14:58Z

❌ Gradle check result for da52269: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions · 2026-03-06T02:32:34Z

❌ Gradle check result for da52269: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions · 2026-03-06T04:14:55Z

❌ Gradle check result for da52269: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

cwperks · 2026-03-06T12:09:53Z

Is there any issue with the gradle check because I see BUILD SUCCESSFUL in the output

BUILD SUCCESSFUL in 48m 28s
3238 actionable tasks: 3230 executed, 2 from cache, 6 up-to-date
[Pipeline] }
[Pipeline] // withSecrets
[Pipeline] sh
+ docker logout
Removing login credentials for https://index.docker.io/v1/
[Pipeline] }
[Pipeline] // withSecrets
[Pipeline] }
[Pipeline] // script
Post stage
[Pipeline] sh
+ find search/build/reports/jacoco/ -name *.xml
+ head -n 1
+ cp -v search/build/reports/jacoco/testCodeCoverageReport/testCodeCoverageReport.xml codeCoverage.xml
'search/build/reports/jacoco/testCodeCoverageReport/testCodeCoverageReport.xml' -> 'codeCoverage.xml'
[Pipeline] junit
Recording test results
[Checks API] No suitable checks publisher found.
[Pipeline] archiveArtifacts
Archiving artifacts
[Pipeline] script
[Pipeline] {
[Pipeline] library
Only using first definition of library jenkins
[Pipeline] echo
No test failed.
[Pipeline] writeFile
[Pipeline] }
[Pipeline] // script
Error when executing always post condition:
an exception which occurred:
	in field com.cloudbees.groovy.cps.impl.BlockScopeEnv.locals
	in object com.cloudbees.groovy.cps.impl.BlockScopeEnv@1a5af277
	in field com.cloudbees.groovy.cps.impl.CallEnv.caller
	in object com.cloudbees.groovy.cps.impl.FunctionCallEnv@3ae0e8e1
	in field com.cloudbees.groovy.cps.Continuable.e
	in object org.jenkinsci.plugins.workflow.cps.SandboxContinuable@2173ab52
	in field org.jenkinsci.plugins.workflow.cps.CpsThread.program
	in object org.jenkinsci.plugins.workflow.cps.CpsThread@4018236f
	in field org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.threads
	in object org.jenkinsci.plugins.workflow.cps.CpsThreadGroup@45b8ac0f
	in object org.jenkinsci.plugins.workflow.cps.CpsThreadGroup@45b8ac0f
Also:   org.jenkinsci.plugins.workflow.actions.ErrorAction$ErrorId: c5bc5f6d-6f75-49a9-8af4-cf46411ac76c
Caused: java.io.NotSerializableException: hudson.tasks.junit.TestResultAction
	at PluginClassLoader for workflow-support//org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:278)
	at PluginClassLoader for workflow-support//org.jboss.marshalling.river.BlockMarshaller.doWriteObject(BlockMarshaller.java:65)
	at PluginClassLoader for workflow-support//org.jboss.marshalling.river.BlockMarshaller.writeObject(BlockMarshaller.java:56)
	at PluginClassLoader for workflow-support//org.jboss.marshalling.MarshallerObjectOutputStream.writeObjectOverride(MarshallerObjectOutputStream.java:50)
	at PluginClassLoader for workflow-

github-actions · 2026-03-06T13:03:56Z

❌ Gradle check result for da52269: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions · 2026-03-06T14:39:26Z

PR Code Analyzer ❗

AI-powered 'Code-Diff-Analyzer' found issues on commit 735a494.

Path	Line	Severity	Description
server/src/main/resources/org/opensearch/bootstrap/security.policy	201	low	Addition of 'localeServiceProvider' RuntimePermission to the global grant block. This is a well-documented, narrow Java permission for locale SPI lookups and is consistent with internationalization support. No evidence of malicious intent; flagged only because any security policy modification warrants review.

The table above displays the top 10 most important findings.

Total: 1 | Critical: 0 | High: 0 | Medium: 0 | Low: 1

Pull Requests Author(s): Please update your Pull Request according to the report above.

Repository Maintainer(s): You can bypass diff analyzer by adding label skip-diff-analyzer after reviewing the changes carefully, then re-run failed actions. To re-enable the analyzer, remove the label, then re-run all actions.

⚠️ Note: The Code-Diff-Analyzer helps protect against potentially harmful code patterns. Please ensure you have thoroughly reviewed the changes beforehand.

Thanks.

github-actions · 2026-03-06T14:40:07Z

Persistent review updated to latest commit 735a494

github-actions · 2026-03-06T15:37:10Z

❌ Gradle check result for 735a494: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

peterzhuamazon · 2026-03-06T18:38:32Z

Due to opensearch-project/opensearch-build-libraries@bcd46d8 the gradle check is failing in post action, not gradle check itself.

In order to unblock 2.19 RC2 will merge this for now.

cc: @Divyaasm @prudhvigodithi on the fix soon.

Thanks.

[2.19] Add runtimePermission for localeServiceProvider to fix docker …

da52269

…startup issue in 2.19.5 Signed-off-by: Craig Perkins <cwperx@amazon.com>

cwperks requested a review from a team as a code owner March 5, 2026 21:28

cwperks added the skip-changelog label Mar 5, 2026

Merge branch '2.19' into locale-service-provider-2.19

735a494

peterzhuamazon approved these changes Mar 6, 2026

View reviewed changes

peterzhuamazon merged commit 688434c into opensearch-project:2.19 Mar 6, 2026
42 of 64 checks passed

Conversation

cwperks commented Mar 5, 2026

Description

Check List

Uh oh!

github-actions bot commented Mar 5, 2026

PR Code Analyzer ❗

Uh oh!

github-actions bot commented Mar 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR Reviewer Guide 🔍

(Review updated until commit 735a494)

Uh oh!

github-actions bot commented Mar 5, 2026

Uh oh!

github-actions bot commented Mar 6, 2026

Uh oh!

github-actions bot commented Mar 6, 2026

Uh oh!

github-actions bot commented Mar 6, 2026

Uh oh!

github-actions bot commented Mar 6, 2026

Uh oh!

cwperks commented Mar 6, 2026

Uh oh!

github-actions bot commented Mar 6, 2026

Uh oh!

github-actions bot commented Mar 6, 2026

PR Code Analyzer ❗

Uh oh!

github-actions bot commented Mar 6, 2026

Uh oh!

github-actions bot commented Mar 6, 2026

Uh oh!

peterzhuamazon commented Mar 6, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

github-actions bot commented Mar 5, 2026 •

edited

Loading

(Review updated until commit `735a494`)