Support for fields containing dots in their name as literals by mohit10011999 · Pull Request #19958 · opensearch-project/OpenSearch

mohit10011999 · 2025-11-11T09:46:06Z

Description

Add the disable_objects field with default value of false to ObjectMapper

Implement builder methods for setting disable_objects
Implement getter methods (disableObjects() and disableObjectsExplicit())
Add serialization/deserialization support for disable_objects parameter

Related Issues

Resolves #[Issue number to be closed when this PR is merged]
#11749
RFC - #20125

Check List

Functionality includes testing.
API changes companion pull request created, if applicable.
Public documentation issue/PR created, if applicable.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: Mohit Kumar <mohitamg@amazon.com>

github-actions · 2025-11-11T09:59:06Z

❌ Gradle check result for 659aa26: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Signed-off-by: Mohit Kumar <mohitamg@amazon.com>

github-actions · 2025-11-11T11:16:01Z

❌ Gradle check result for 5405ee6: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Signed-off-by: Mohit Kumar <mohitamg@amazon.com>

github-actions · 2025-11-11T14:01:03Z

❌ Gradle check result for f9fa2ba: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Signed-off-by: Mohit Kumar <mohitamg@amazon.com>

github-actions · 2025-11-12T17:56:12Z

❌ Gradle check result for bd8e0c6: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Signed-off-by: Mohit Kumar <mohitamg@amazon.com>

github-actions · 2025-11-13T12:56:54Z

❌ Gradle check result for ecf3e8b: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Signed-off-by: Mohit Kumar <mohitamg@amazon.com>

github-actions · 2025-11-14T07:00:40Z

❌ Gradle check result for e505b3e: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Signed-off-by: Mohit Kumar <mohitamg@amazon.com>

github-actions · 2025-11-14T10:19:28Z

❌ Gradle check result for 3c56a97: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Signed-off-by: Mohit Kumar <mohitamg@amazon.com>

coderabbitai · 2025-11-29T17:04:59Z

Important

Review skipped

Auto incremental reviews are disabled on this repository.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

📝 Walkthrough

Walkthrough

Add a new ObjectMapper disable_objects option, thread it through mapper construction and metadata collection, propagate disable_objects across template/request mappings before merge, implement disable_objects-aware parsing with flattening and dynamic handling, and add extensive unit tests and validations.

Changes

Cohort / File(s)	Summary
Core mapping config `server/src/main/java/org/opensearch/index/mapper/ObjectMapper.java`, `server/src/main/java/org/opensearch/index/mapper/RootObjectMapper.java`	Add public `disable_objects` default and Explicit wrapper, builder setter, new constructor parameters/signatures to carry `disableObjects`, accessors, XContent emission, and build/merge validations (immutability, nested rejection, flat-field checks).
Document parsing & flattening `server/src/main/java/org/opensearch/index/mapper/DocumentParser.java`	Integrate `disable_objects` into parsing/dynamic flow: new detection/handling helpers, flattening utilities for objects/arrays, updated mapper resolution and dynamic/update creation to treat dotted names as literal when applicable.
Template collection & metadata `server/src/main/java/org/opensearch/cluster/metadata/MetadataCreateIndexService.java`	Rework mapping collection to assemble templates first and request mapping last; add `applyDisableObjectsOverrides(List<Map<String,Object>>)` and field-aware merge helpers (`mergeTemplateFieldMappings`, `mergeFieldProperties`) to propagate `disable_objects`; remove `XContentHelper` usage in merge flow.
Unit tests — metadata `server/src/test/java/org/opensearch/cluster/metadata/MetadataCreateIndexServiceTests.java`	Add comprehensive tests covering template `disable_objects` semantics, override/priority logic, V1/V2 equivalence, edge cases, and field-property override scenarios.
Unit tests — document parsing `server/src/test/java/org/opensearch/index/mapper/DocumentParserTests.java`	Add extensive tests for root/nested flattening, arrays, dynamic mapping interactions, existing-field interactions, mapper lookup/flattening scenarios, and strict-dynamic error cases under `disable_objects`.
Unit tests — ObjectMapper validations `server/src/test/java/org/opensearch/index/mapper/ObjectMapperTests.java`	Add tests for `disable_objects` immutability, nested-object rejection, and flat-field compatibility during build/merge.
Test call-site adjustments `server/src/test/java/org/opensearch/index/mapper/FieldAliasMapperValidationTests.java`	Update imports and ObjectMapper constructor invocations in tests to match the new constructor signature (additional Explicit param and mapper Map casting).
Changelog `CHANGELOG.md`	Add Unreleased entry: support for fields containing dots in their name as literals.

Sequence Diagram(s)

sequenceDiagram
    autonumber
    participant Client
    participant Templates as "Index Templates"
    participant Metadata as "MetadataCreateIndexService"
    participant MapperFactory as "ObjectMapper / RootObjectMapper"
    participant Parser as "DocumentParser"
    participant Mappings as "MapperService / Mappings"

    Client->>Templates: define templates (may set disable_objects)
    Client->>Metadata: create index / merge request (includes request mapping)
    Templates->>Metadata: provide template mappings
    Metadata->>Metadata: collect mappings (templates first, request last)
    Metadata->>Metadata: applyDisableObjectsOverrides() to compute effective disable_objects
    Metadata->>MapperFactory: build/register mapping with effective disable_objects
    Client->>Parser: send document to index
    Parser->>MapperFactory: resolve mapper for field path
    alt effective disable_objects = true
        Parser->>Parser: flatten object/array into dotted fields
        Parser->>Mappings: create/update flat dotted-field mappers
    else
        Parser->>Parser: normal nested traversal / create intermediates
        Parser->>Mappings: create/update nested mappers
    end
    Mappings-->>Client: indexing acknowledged

Estimated code review effort

🎯 5 (Critical) | ⏱️ ~120 minutes

Suggested labels

v3.4.0, backport 3.4

🚥 Pre-merge checks | ✅ 2 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 27.93% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (2 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title accurately summarizes the main change: adding support for fields with dots as literals via the disable_objects feature.
Description check	✅ Passed	The description includes a clear summary of the changes (disable_objects field implementation with getters/setters), properly references related issues and RFC, and confirms testing and documentation requirements.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

Signed-off-by: Mohit Kumar <mohitamg@amazon.com>

github-actions · 2025-11-29T17:19:43Z

❌ Gradle check result for bd83c0b: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Signed-off-by: Mohit Kumar <mohitamg@amazon.com>

github-actions · 2025-12-03T08:57:35Z

❌ Gradle check result for c711dcb: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

…rch-project#19958) * Add preserve_dots paramater to ObjectMapper Signed-off-by: Mohit Kumar <mohitamg@amazon.com> * Extend RootObjectMapper to support preserve_dots Signed-off-by: Mohit Kumar <mohitamg@amazon.com> * Implement flat field parsing in DocumentParser Signed-off-by: Mohit Kumar <mohitamg@amazon.com> * Implement validation in MapperService Signed-off-by: Mohit Kumar <mohitamg@amazon.com> * Add index template support for preserve_dots tests Signed-off-by: Mohit Kumar <mohitamg@amazon.com> * error handling with descriptive messages Signed-off-by: Mohit Kumar <mohitamg@amazon.com> * Removing redundancy Signed-off-by: Mohit Kumar <mohitamg@amazon.com> * Adding more edge cases, autoflatten and test cases Signed-off-by: Mohit Kumar <mohitamg@amazon.com> * Adding more edge cases, autoflatten and test cases Signed-off-by: Mohit Kumar <mohitamg@amazon.com> * Adding autoflattening at nested level as well Signed-off-by: Mohit Kumar <mohitamg@amazon.com> * Fixing gradle spotless checks Signed-off-by: Mohit Kumar <mohitamg@amazon.com> * Stab at optimizing the code via removing redundant code and fixing multiple functions Signed-off-by: Mohit Kumar <mohitamg@amazon.com> * Removing splitPathRespectingDisableObjects and trying to achieve it through existing getMapper function Signed-off-by: Mohit Kumar <mohitamg@amazon.com> * Reduction of if else statements by extracting conditional logic into focused methods Signed-off-by: Mohit Kumar <mohitamg@amazon.com> * Reduction of if else statements by extracting conditional logic into focused methods Signed-off-by: Mohit Kumar <mohitamg@amazon.com> * Reduction of if else statements by extracting conditional logic into focused methods Signed-off-by: Mohit Kumar <mohitamg@amazon.com> * Reduction of if else statements by extracting conditional logic into focused methods Signed-off-by: Mohit Kumar <mohitamg@amazon.com> * Reduction of if else statements by extracting conditional logic into focused methods Signed-off-by: Mohit Kumar <mohitamg@amazon.com> * Reduction of if else statements by extracting conditional logic into focused methods Signed-off-by: Mohit Kumar <mohitamg@amazon.com> * Reduction of if else statements by extracting conditional logic into focused methods Signed-off-by: Mohit Kumar <mohitamg@amazon.com> * Revert "Reduction of if else statements by extracting conditional logic into focused methods" This reverts commit 9ca7e6d. Signed-off-by: Mohit Kumar <mohitamg@amazon.com> * [Plugins] Relax jar hell check when extended plugins share transitive dependencies (opensearch-project#20103) * plugin service allow multiple extended plugins both extend the same module/plugin Signed-off-by: Karen X <karenxyr@gmail.com> * fix tests Signed-off-by: Karen X <karenxyr@gmail.com> --------- Signed-off-by: Karen X <karenxyr@gmail.com> Co-authored-by: karenx <karenx@uber.com> * Reverting change in CHANGELOG.md Signed-off-by: Mohit Kumar <mohitamg@amazon.com> * Add integ test for simulating node join left event when data node clu… (opensearch-project#19907) * Add integration test for simulating node join left event when data node cluster state publication lag because the cluster applier thread being busy. Signed-off-by: Swetha Guptha <gupthasg@amazon.com> Signed-off-by: Mohit Kumar <mohitamg@amazon.com> * Adding few more UTs in DocumentParserTests and ObjectMapperTests for code patch coverage Signed-off-by: Mohit Kumar <mohitamg@amazon.com> * Adding changes in CHANGELOG.md Signed-off-by: Mohit Kumar <mohitamg@amazon.com> * Adding more tests for code patch coverage for DocumentParser and ObjectMapper Signed-off-by: Mohit Kumar <mohitamg@amazon.com> * Trigger workflow1 Signed-off-by: Mohit Kumar <mohitamg@amazon.com> * Trigger workflow2 Signed-off-by: Mohit Kumar <mohitamg@amazon.com> * Trigger workflow3 Signed-off-by: Mohit Kumar <mohitamg@amazon.com> * Adding more tests for code patch coverage for DocumentParser and ObjectMapper second round and refactoring of code especially dynamic update portion Signed-off-by: Mohit Kumar <mohitamg@amazon.com> * Removing dead end code from Document Parser related to parseArray and parseObject Signed-off-by: Mohit Kumar <mohitamg@amazon.com> * Trigger workflow4 Signed-off-by: Mohit Kumar <mohitamg@amazon.com> * Trigger workflow4 Signed-off-by: Mohit Kumar <mohitamg@amazon.com> * Trigger workflow5 Signed-off-by: Mohit Kumar <mohitamg@amazon.com> * Trigger workflow6 Signed-off-by: Mohit Kumar <mohitamg@amazon.com> * Trigger workflow6 Signed-off-by: Mohit Kumar <mohitamg@amazon.com> * Trigger workflow7 Signed-off-by: Mohit Kumar <mohitamg@amazon.com> * Trigger workflow8 Signed-off-by: Mohit Kumar <mohitamg@amazon.com> * Fixing breaking changes in ObjectMapper builder method Signed-off-by: Mohit Kumar <mohitamg@amazon.com> * Fixing spotless checks Signed-off-by: Mohit Kumar <mohitamg@amazon.com> * Trigger workflow9 Signed-off-by: Mohit Kumar <mohitamg@amazon.com> * Trigger workflow9 Signed-off-by: Mohit Kumar <mohitamg@amazon.com> * Handling v2 templates for dotted field feature and stab at using existing methods in switch case Signed-off-by: Mohit Kumar <mohitamg@amazon.com> * Handling v2 templates for dotted field feature and stab at using existing methods in switch case Signed-off-by: Mohit Kumar <mohitamg@amazon.com> * Fixing org.opensearch.indices.template.SimpleIndexTemplateIT IT Signed-off-by: Mohit Kumar <mohitamg@amazon.com> * Fixing array mapping with nested disable_objects as true Signed-off-by: Mohit Kumar <mohitamg@amazon.com> * Addressed review comments and removed redundant code Signed-off-by: Mohit Kumar <mohitamg@amazon.com> --------- Signed-off-by: Mohit Kumar <mohitamg@amazon.com> Signed-off-by: mohit10011999 <pinkijyoti1995@gmail.com> Signed-off-by: Karen X <karenxyr@gmail.com> Signed-off-by: Swetha Guptha <gupthasg@amazon.com> Co-authored-by: Mohit Kumar <mohitamg@amazon.com> Co-authored-by: Karen X <karenxyr@gmail.com> Co-authored-by: karenx <karenx@uber.com> Co-authored-by: Swetha Guptha <gupthasg@amazon.com>

Add preserve_dots paramater to ObjectMapper

659aa26

Signed-off-by: Mohit Kumar <mohitamg@amazon.com>

Extend RootObjectMapper to support preserve_dots

5405ee6

Signed-off-by: Mohit Kumar <mohitamg@amazon.com>

Implement flat field parsing in DocumentParser

f9fa2ba

Signed-off-by: Mohit Kumar <mohitamg@amazon.com>

mohit10011999 changed the title ~~Add preserve_dots paramater to ObjectMapper~~ Support for fields containing dots in their name as literals Nov 11, 2025

Implement validation in MapperService

bd8e0c6

Signed-off-by: Mohit Kumar <mohitamg@amazon.com>

Add index template support for preserve_dots tests

ecf3e8b

Signed-off-by: Mohit Kumar <mohitamg@amazon.com>

error handling with descriptive messages

e505b3e

Signed-off-by: Mohit Kumar <mohitamg@amazon.com>

Removing redundancy

3c56a97

Signed-off-by: Mohit Kumar <mohitamg@amazon.com>

mohit10011999 mentioned this pull request Nov 29, 2025

[RFC] Support for dotted field names as standalone fields #20125

Open

Adding more edge cases, autoflatten and test cases

96af23b

Signed-off-by: Mohit Kumar <mohitamg@amazon.com>

Adding more edge cases, autoflatten and test cases

bd83c0b

Signed-off-by: Mohit Kumar <mohitamg@amazon.com>

Adding autoflattening at nested level as well

c711dcb

Signed-off-by: Mohit Kumar <mohitamg@amazon.com>

mohit10011999 marked this pull request as ready for review December 8, 2025 10:13

mohit10011999 requested review from Bukhtawar, CEHENKLE, anasalkouz, andrross, ashking94, cwperks and dbwiddis as code owners December 8, 2025 10:13

newtonne mentioned this pull request Mar 9, 2026

[BUG] _field_caps returns empty response for indexes with disable_objects: true mappings #20811

Closed

This was referenced Mar 18, 2026

[AUTOCUT] Gradle Check Flaky Test Report for TransferManagerBlobContainerReaderTests #18872

Closed

[AUTOCUT] Gradle Check Flaky Test Report for RemoteDiscoveryNodesTests #20932

Open

Conversation

mohit10011999 commented Nov 11, 2025 • edited by coderabbitai bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Related Issues

Check List

Uh oh!

github-actions bot commented Nov 11, 2025

Uh oh!

github-actions bot commented Nov 11, 2025

Uh oh!

github-actions bot commented Nov 11, 2025

Uh oh!

github-actions bot commented Nov 12, 2025

Uh oh!

github-actions bot commented Nov 13, 2025

Uh oh!

github-actions bot commented Nov 14, 2025

Uh oh!

github-actions bot commented Nov 14, 2025

Uh oh!

coderabbitai bot commented Nov 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Review skipped

Walkthrough

Changes

Sequence Diagram(s)

Estimated code review effort

Suggested labels

Uh oh!

github-actions bot commented Nov 29, 2025

Uh oh!

github-actions bot commented Dec 3, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

mohit10011999 commented Nov 11, 2025 •

edited by coderabbitai bot

Loading

coderabbitai bot commented Nov 29, 2025 •

edited

Loading