[Refactor] XContentFactory contentType introspection to MediaTypeRegistry by nknize · Pull Request #8826 · opensearch-project/OpenSearch

nknize · 2023-07-21T22:58:36Z

XContentFactory is tightly coupled to concrete XContentType. This PR builds on the MediaType abstractions, specifically MediaTypeRegistry, to decouple contentType( introspection (e.g., determining type from byte streams) from concrete XContentType implementations. This removes the tight coupling of :libs:opensearch-x-content with abstract interface contracts currently defined in :server further enabling the ability to refactor those contracts from :server to the proper library for modularity and extensibility.

relates #5910
relates #8110

depends on #8636

nknize · 2023-07-21T23:02:08Z

Reminder: this PR is larger than it's actual change because it includes (depends on) #8636

nknize · 2023-07-21T23:10:33Z

I ran across a flaky test w/ RemoteFSTranslogTests when running :server:test locally. It does not repro on it's own (hence "flaky") so I could not chase the failure.

./gradlew ':server:test' --tests "org.opensearch.index.translog.RemoteFSTranslogTests.testConcurrentWriteViewsAndSnapshot" -Dtests.seed=11209ED8456F2E6A -Dtests.security.manager=true -Dtests.jvm.argline="-XX:TieredStopAtLevel=1 -XX:ReservedCodeCacheSize=64m" -Dtests.locale=sr-ME -Dtests.timezone=Pacific/Johnston -Druntime.java=20

I documented this on #5823 for posterity.

github-actions · 2023-07-21T23:21:35Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/20702/
CommitID: f2da36a
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

github-actions · 2023-07-28T16:23:57Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/21280/
CommitID: bdecf35
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

…erRegistry XContentFactory is tightly coupled to concrete XContentType. This commit builds on the MediaType abstractions, specifically MediaTypeParserRegistry, to decouple contentType introspection (e.g., determining type from byte streams) from concrete XContentTypes. This enables downstream extensions (e.g., serverless or cloud native implementations) to register their own custom XContentType and define the serialization format for proper content introspection. This also removes the tight coupling of :libs:opensearch-x-content with abstract interface contracts further enabling modularity. Signed-off-by: Nicholas Walter Knize <nknize@apache.org>

nknize · 2023-07-28T18:28:39Z

@reta this one will be ready to go as well

github-actions · 2023-07-28T19:03:41Z

Gradle Check (Jenkins) Run Completed with:

RESULT: SUCCESS ✅
URL: https://build.ci.opensearch.org/job/gradle-check/21300/
CommitID: 4899c0d

…erRegistry (opensearch-project#8826) XContentFactory is tightly coupled to concrete XContentType. This commit builds on the MediaType abstractions, specifically MediaTypeParserRegistry, to decouple contentType introspection (e.g., determining type from byte streams) from concrete XContentTypes. This enables downstream extensions (e.g., serverless or cloud native implementations) to register their own custom XContentType and define the serialization format for proper content introspection. This also removes the tight coupling of :libs:opensearch-x-content with abstract interface contracts further enabling modularity. Signed-off-by: Nicholas Walter Knize <nknize@apache.org>

…3069) There are multiple PRs in core affecting the security plugin that the security plugin needs to adapt to. - opensearch-project/OpenSearch#7792 - opensearch-project/OpenSearch#8826 - opensearch-project/OpenSearch#8668 I am opening a Draft PR that includes a fix for the Lucene-related test failures which was caused by opensearch-project/OpenSearch#7792 Resolves: #3064 Signed-off-by: Craig Perkins <cwperx@amazon.com>

…pensearch-project#3069) There are multiple PRs in core affecting the security plugin that the security plugin needs to adapt to. - opensearch-project/OpenSearch#7792 - opensearch-project/OpenSearch#8826 - opensearch-project/OpenSearch#8668 I am opening a Draft PR that includes a fix for the Lucene-related test failures which was caused by opensearch-project/OpenSearch#7792 Resolves: opensearch-project#3064 Signed-off-by: Craig Perkins <cwperx@amazon.com> (cherry picked from commit 08d1734)

opensearch-trigger-bot · 2023-08-01T16:46:12Z

The backport to 2.x failed:

The process '/usr/bin/git' failed with exit code 128

To backport manually, run these commands in your terminal:

# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add ../.worktrees/backport-2.x 2.x
# Navigate to the new working tree
pushd ../.worktrees/backport-2.x
# Create a new branch
git switch --create backport/backport-8826-to-2.x
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 36c37ace30cdf9d2f92176a3aa38b60865c33056
# Push it to GitHub
git push --set-upstream origin backport/backport-8826-to-2.x
# Go back to the original working tree
popd
# Delete the working tree
git worktree remove ../.worktrees/backport-2.x

Then, create a pull request where the base branch is 2.x and the compare/head branch is backport/backport-8826-to-2.x.

opensearch-trigger-bot · 2023-08-01T22:05:39Z

The backport to 2.x failed:

The process '/usr/bin/git' failed with exit code 128

To backport manually, run these commands in your terminal:

# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add ../.worktrees/backport-2.x 2.x
# Navigate to the new working tree
pushd ../.worktrees/backport-2.x
# Create a new branch
git switch --create backport/backport-8826-to-2.x
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 36c37ace30cdf9d2f92176a3aa38b60865c33056
# Push it to GitHub
git push --set-upstream origin backport/backport-8826-to-2.x
# Go back to the original working tree
popd
# Delete the working tree
git worktree remove ../.worktrees/backport-2.x

Then, create a pull request where the base branch is 2.x and the compare/head branch is backport/backport-8826-to-2.x.

…erRegistry (opensearch-project#8826) XContentFactory is tightly coupled to concrete XContentType. This commit builds on the MediaType abstractions, specifically MediaTypeParserRegistry, to decouple contentType introspection (e.g., determining type from byte streams) from concrete XContentTypes. This enables downstream extensions (e.g., serverless or cloud native implementations) to register their own custom XContentType and define the serialization format for proper content introspection. This also removes the tight coupling of :libs:opensearch-x-content with abstract interface contracts further enabling modularity. Signed-off-by: Nicholas Walter Knize <nknize@apache.org> (cherry picked from commit 36c37ac)

…erRegistry (#8826) (#9050) XContentFactory is tightly coupled to concrete XContentType. This commit builds on the MediaType abstractions, specifically MediaTypeParserRegistry, to decouple contentType introspection (e.g., determining type from byte streams) from concrete XContentTypes. This enables downstream extensions (e.g., serverless or cloud native implementations) to register their own custom XContentType and define the serialization format for proper content introspection. This also removes the tight coupling of :libs:opensearch-x-content with abstract interface contracts further enabling modularity. Signed-off-by: Nicholas Walter Knize <nknize@apache.org> (cherry picked from commit 36c37ac)

…erRegistry (opensearch-project#8826) XContentFactory is tightly coupled to concrete XContentType. This commit builds on the MediaType abstractions, specifically MediaTypeParserRegistry, to decouple contentType introspection (e.g., determining type from byte streams) from concrete XContentTypes. This enables downstream extensions (e.g., serverless or cloud native implementations) to register their own custom XContentType and define the serialization format for proper content introspection. This also removes the tight coupling of :libs:opensearch-x-content with abstract interface contracts further enabling modularity. Signed-off-by: Nicholas Walter Knize <nknize@apache.org> Signed-off-by: Kaushal Kumar <ravi.kaushal97@gmail.com>

…erRegistry (opensearch-project#8826) XContentFactory is tightly coupled to concrete XContentType. This commit builds on the MediaType abstractions, specifically MediaTypeParserRegistry, to decouple contentType introspection (e.g., determining type from byte streams) from concrete XContentTypes. This enables downstream extensions (e.g., serverless or cloud native implementations) to register their own custom XContentType and define the serialization format for proper content introspection. This also removes the tight coupling of :libs:opensearch-x-content with abstract interface contracts further enabling modularity. Signed-off-by: Nicholas Walter Knize <nknize@apache.org> Signed-off-by: Ivan Brusic <ivan.brusic@flocksafety.com>

…erRegistry (opensearch-project#8826) XContentFactory is tightly coupled to concrete XContentType. This commit builds on the MediaType abstractions, specifically MediaTypeParserRegistry, to decouple contentType introspection (e.g., determining type from byte streams) from concrete XContentTypes. This enables downstream extensions (e.g., serverless or cloud native implementations) to register their own custom XContentType and define the serialization format for proper content introspection. This also removes the tight coupling of :libs:opensearch-x-content with abstract interface contracts further enabling modularity. Signed-off-by: Nicholas Walter Knize <nknize@apache.org> Signed-off-by: Shivansh Arora <hishiv@amazon.com>

nknize added enhancement Enhancement or improvement to existing feature or request skip-changelog v2.10.0 labels Jul 21, 2023

nknize requested review from Bukhtawar, CEHENKLE, Rishikesh1159, VachaShah, anasalkouz, andrross, dblock, dbwiddis, dreamer-89, gbbafna, kartg, kotwanikunal, mch2, owaiskazi19, reta, ryanbogan, sachinpkale, saratvemulapalli, setiah, shwetathareja and tlfeng as code owners July 21, 2023 22:58

nknize mentioned this pull request Jul 21, 2023

[BUG] Reproducible test failure .RemoteFSTranslogTests.testConcurrentWriteViewsAndSnapshot #5823

Closed

nknize mentioned this pull request Jul 21, 2023

[Refactor] MediaTypeParser to MediaTypeParserRegistry #8636

Merged

nknize force-pushed the xcontentFactoryAbstraction branch from f2da36a to d3787ae Compare July 21, 2023 23:35

nknize force-pushed the xcontentFactoryAbstraction branch from e6cf81f to bdecf35 Compare July 28, 2023 15:39

nknize force-pushed the xcontentFactoryAbstraction branch from bdecf35 to 4899c0d Compare July 28, 2023 18:16

reta approved these changes Jul 28, 2023

View reviewed changes

reta merged commit 36c37ac into opensearch-project:main Jul 28, 2023

martin-gaievski mentioned this pull request Jul 28, 2023

Fixing compilation errors after refactoring in core OpenSearch opensearch-project/neural-search#237

Merged

2 tasks

Yury-Fridlyand mentioned this pull request Jul 29, 2023

Fix breaking changes. opensearch-project/sql#1914

Merged

6 tasks

cwperks mentioned this pull request Jul 29, 2023

[Build Break] Fix build after Lucene upgrade and breaking XContentFactory changes opensearch-project/security#3069

Merged

Hailong-am mentioned this pull request Jul 31, 2023

Fix core XcontentFactory refactor opensearch-project/notifications#732

Merged

1 task

This was referenced Aug 1, 2023

Resolve core Xcontent refactor opensearch-project/alerting#1061

Closed

Resolve core Xcontent and CircuitBreaker refactor opensearch-project/alerting#1062

Merged

nknize added the backport 2.x Backport to 2.x branch label Aug 1, 2023

martin-gaievski mentioned this pull request Aug 1, 2023

Fix compilation errors after recent core pushes opensearch-project/ml-commons#1177

Merged

2 tasks

nknize added backport 2.x Backport to 2.x branch and removed backport 2.x Backport to 2.x branch labels Aug 1, 2023

nknize mentioned this pull request Aug 1, 2023

[Backport 2.x] [Refactor] XContentFactory contentType introspection to MediaTypeRegistry (#8826) #9050

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Refactor] XContentFactory contentType introspection to MediaTypeRegistry#8826

[Refactor] XContentFactory contentType introspection to MediaTypeRegistry#8826
reta merged 1 commit intoopensearch-project:mainfrom
nknize:xcontentFactoryAbstraction

nknize commented Jul 21, 2023 •

edited

Loading

Uh oh!

nknize commented Jul 21, 2023

Uh oh!

nknize commented Jul 21, 2023 •

edited

Loading

Uh oh!

github-actions bot commented Jul 21, 2023

Uh oh!

github-actions bot commented Jul 28, 2023

Uh oh!

nknize commented Jul 28, 2023

Uh oh!

github-actions bot commented Jul 28, 2023

Uh oh!

opensearch-trigger-bot bot commented Aug 1, 2023

Uh oh!

opensearch-trigger-bot bot commented Aug 1, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

nknize commented Jul 21, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

nknize commented Jul 21, 2023

Uh oh!

nknize commented Jul 21, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Jul 21, 2023

Gradle Check (Jenkins) Run Completed with:

Uh oh!

github-actions bot commented Jul 28, 2023

Gradle Check (Jenkins) Run Completed with:

Uh oh!

nknize commented Jul 28, 2023

Uh oh!

github-actions bot commented Jul 28, 2023

Gradle Check (Jenkins) Run Completed with:

Uh oh!

opensearch-trigger-bot bot commented Aug 1, 2023

Uh oh!

opensearch-trigger-bot bot commented Aug 1, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

nknize commented Jul 21, 2023 •

edited

Loading

nknize commented Jul 21, 2023 •

edited

Loading