Skip to content

Added OpenSearch 3.0.#459

Merged
Xtansia merged 7 commits intoopensearch-project:mainfrom
dblock:opensearch-3.0
Aug 4, 2024
Merged

Added OpenSearch 3.0.#459
Xtansia merged 7 commits intoopensearch-project:mainfrom
dblock:opensearch-3.0

Conversation

@dblock
Copy link
Copy Markdown
Member

@dblock dblock commented Aug 1, 2024 •
edited
Loading

Description

  • Adds OpenSearch 3.0 to CI/CD.
  • Dumps docker logs in case tests fail to debug server-side failures.
  • Increases memory limit threshold for SQL that fails intermittently.
  • Removes unneeded quotes in docker compose.

Issues Resolved

N/A

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Aug 1, 2024 •
edited
Loading

Changes Analysis

Commit SHA: d909cb5
Comparing To SHA: 8bf497c

API Changes

Summary

└─┬Components
  └─┬nodes._common:ShardSearchBackpressureTaskStats
    └──[➕] properties (46405:9)
Document Element Total Changes Breaking Changes
components 1 0
  • Total Changes: 1
  • Additions: 1

Report

The full API changes report is available at: https://github.com/opensearch-project/opensearch-api-specification/actions/runs/10228661142/artifacts/1772277099

API Coverage

Before After Δ
Covered (%) 510 (49.95 %) 510 (49.95 %) 0 (0 %)
Uncovered (%) 511 (50.05 %) 511 (50.05 %) 0 (0 %)
Unknown 24 24 0

@dblock dblock mentioned this pull request Aug 1, 2024
Merged
6 tasks
@dblock dblock mentioned this pull request Aug 1, 2024
Closed
@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Aug 1, 2024 •
edited
Loading

Spec Test Coverage Analysis

Total Tested
534 118 (22.1 %)

@dblock dblock marked this pull request as ready for review August 1, 2024 13:14
@dblock dblock mentioned this pull request Aug 1, 2024
Merged
@dblock dblock marked this pull request as draft August 1, 2024 13:45
@dblock
Copy link
Copy Markdown
Member Author

dblock commented Aug 1, 2024

The intermittent SQL failure is an OOM.

2024-08-01T14:02:08.9247018Z [2024-08-01T14:02:05,858][INFO ][o.o.s.s.a.SQLSyntaxParser] [6080215090a3] New Engine Request Query: ( SELECT * FROM table )
2024-08-01T14:02:08.9247664Z [2024-08-01T14:02:05,983][WARN ][o.o.s.o.m.OpenSearchMemoryHealthy] [6080215090a3] Memory usage:1054384512 exceed limit:912680550
2024-08-01T14:02:08.9248242Z [2024-08-01T14:02:05,983][WARN ][o.o.s.o.m.OpenSearchMemoryHealthy] [6080215090a3] Fast failure the current request
2024-08-01T14:02:08.9249007Z [2024-08-01T14:02:05,985][ERROR][o.o.s.l.p.RestSqlAction  ] [6080215090a3] ef30b895-53e0-4444-8034-3480c19b181c Server side error during query execution
2024-08-01T14:02:08.9249390Z java.lang.IllegalStateException: resource is not enough to run the query, quit.
2024-08-01T14:02:08.9250329Z 	at org.opensearch.sql.opensearch.executor.protector.ResourceMonitorPlan.open(ResourceMonitorPlan.java:47) ~[opensearch-2.15.0.0.jar:?]
2024-08-01T14:02:08.9250707Z 	at java.base/java.util.Collections$SingletonList.forEach(Collections.java:5186) ~[?:?]
2024-08-01T14:02:08.9251234Z 	at org.opensearch.sql.planner.physical.PhysicalPlan.open(PhysicalPlan.java:29) ~[core-2.15.0.0.jar:?]
2024-08-01T14:02:08.9252023Z 	at org.opensearch.sql.opensearch.executor.OpenSearchExecutionEngine.lambda$execute$0(OpenSearchExecutionEngine.java:49) [opensearch-2.15.0.0.jar:?]
2024-08-01T14:02:08.9252708Z 	at org.opensearch.sql.opensearch.client.OpenSearchNodeClient.schedule(OpenSearchNodeClient.java:177) [opensearch-2.15.0.0.jar:?]
2024-08-01T14:02:08.9253640Z 	at org.opensearch.sql.opensearch.executor.OpenSearchExecutionEngine.execute(OpenSearchExecutionEngine.java:43) [opensearch-2.15.0.0.jar:?]
2024-08-01T14:02:08.9254412Z 	at org.opensearch.sql.executor.QueryService.lambda$executePlan$1(QueryService.java:66) [core-2.15.0.0.jar:?]
2024-08-01T14:02:08.9254706Z 	at java.base/java.util.Optional.ifPresentOrElse(Optional.java:198) [?:?]
2024-08-01T14:02:08.9255209Z 	at org.opensearch.sql.executor.QueryService.executePlan(QueryService.java:63) [core-2.15.0.0.jar:?]
2024-08-01T14:02:08.9255688Z 	at org.opensearch.sql.executor.QueryService.execute(QueryService.java:41) [core-2.15.0.0.jar:?]
2024-08-01T14:02:08.9256179Z 	at org.opensearch.sql.executor.execution.QueryPlan.execute(QueryPlan.java:63) [core-2.15.0.0.jar:?]
2024-08-01T14:02:08.9256928Z 	at org.opensearch.sql.opensearch.executor.OpenSearchQueryManager.lambda$submit$0(OpenSearchQueryManager.java:31) [opensearch-2.15.0.0.jar:?]
2024-08-01T14:02:08.9257734Z 	at org.opensearch.sql.opensearch.executor.OpenSearchQueryManager.lambda$withCurrentContext$1(OpenSearchQueryManager.java:45) [opensearch-2.15.0.0.jar:?]
2024-08-01T14:02:08.9258452Z 	at org.opensearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:882) [opensearch-2.15.0.jar:2.15.0]
2024-08-01T14:02:08.9258901Z 	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144) [?:?]
2024-08-01T14:02:08.9259319Z 	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642) [?:?]
2024-08-01T14:02:08.9259532Z 	at java.base/java.lang.Thread.run(Thread.java:1583) [?:?]

@dblock dblock marked this pull request as ready for review August 1, 2024 15:13
@dblock dblock force-pushed the opensearch-3.0 branch 6 times, most recently from c272e84 to 36ae2ac Compare August 2, 2024 19:09
dblock added 3 commits August 3, 2024 10:17
@dblock
Added OpenSearch 3.0.
eca59ff 
Signed-off-by: dblock <dblock@amazon.com>
@dblock
Removed unneeded quotes.
73681e8 
Signed-off-by: dblock <dblock@amazon.com>
@dblock
Add --verbose to debug SQL failure.
9a8de55 
Signed-off-by: dblock <dblock@amazon.com>
dblock added 4 commits August 3, 2024 10:17
@dblock
Increase plugin memory circuit breaker limit.
50390ca 
Signed-off-by: dblock <dblock@amazon.com>
@dblock
Make pretty.
2bcdefe 
Signed-off-by: dblock <dblock@amazon.com>
@dblock
Remove verbose.
a98eb96 
Signed-off-by: dblock <dblock@amazon.com>
@dblock
Get and set container ID as output.
d909cb5 
Signed-off-by: dblock <dblock@amazon.com>
@Xtansia Xtansia merged commit 13a603c into opensearch-project:main Aug 4, 2024
@dblock dblock deleted the opensearch-3.0 branch August 4, 2024 22:52
This was referenced Aug 5, 2024
Closed
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@Xtansia Xtansia Xtansia approved these changes

@harshavamsi harshavamsi Awaiting requested review from harshavamsi harshavamsi is a code owner

@sachetalva sachetalva Awaiting requested review from sachetalva sachetalva is a code owner

@nhtruong nhtruong Awaiting requested review from nhtruong

@VachaShah VachaShah Awaiting requested review from VachaShah VachaShah is a code owner

@Tokesh Tokesh Awaiting requested review from Tokesh Tokesh is a code owner

@aabeshov aabeshov Awaiting requested review from aabeshov aabeshov is a code owner

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@dblock @Xtansia