Skip to content

Revert "Deprecate _source.mode in mappings (#116689)"#117150

Merged
martijnvg merged 1 commit intoelastic:mainfrom
martijnvg:revert_116689
Nov 20, 2024
Merged

Revert "Deprecate _source.mode in mappings (#116689)"#117150
martijnvg merged 1 commit intoelastic:mainfrom
martijnvg:revert_116689

Conversation

@martijnvg
Copy link
Copy Markdown
Member

@martijnvg martijnvg commented Nov 20, 2024 •
edited
Loading

This reverts #116689. Bwc tests fail, because older nodes are killed with the following error:

[2024-11-20T10:54:58,600][ERROR][o.e.b.ElasticsearchUncaughtExceptionHandler] [v8.17.0-0] fatal error in thread [elasticsearch[v8.17.0-0
][clusterApplierService#updateTask][T#1]], exiting java.lang.AssertionError: provided source [{"_doc":{"_data_stream_timestamp":{"enabled":true},"_source":{},"properties":{"@timestamp":{"type":"date"},"k8s":{"properties":{"pod":{"properties":{"ip":{"type":"ip"},"name":{"type":"keyword"},"network":{"properties":{"rx":{"type":"long"},"tx":{"type":"long"}}},"uid":{"type":"keyword","time_series_dimension":true}}}}},"metricset":{"type":"keyword","time_series_dimension":true}}}}] differs from mapping [{"_doc":{"_data_stream_timestamp":{"enabled":true},"_source":{"mode":"synthetic"},"properties":{"@timestamp":{"type":"date"},"k8s":{"properties":{"pod":{"properties":{"ip":{"type":"ip"},"name":{"type":"keyword"},"network":{"properties":{"rx":{"type":"long"},"tx":{"type":"long"}}},"uid":{"type":"keyword","time_series_dimension":true}}}}},"metricset":{"type":"keyword","time_series_dimension":true}}}}]
        at org.elasticsearch.server@9.0.0-SNAPSHOT/org.elasticsearch.index.mapper.DocumentMapper.<init>(DocumentMapper.java:66)
        at org.elasticsearch.server@9.0.0-SNAPSHOT/org.elasticsearch.index.mapper.MapperService.newDocumentMapper(MapperService.java:588)
        at org.elasticsearch.server@9.0.0-SNAPSHOT/org.elasticsearch.index.mapper.MapperService.updateMapping(MapperService.java:346)
        at org.elasticsearch.server@9.0.0-SNAPSHOT/org.elasticsearch.index.IndexService.updateMapping(IndexService.java:840)
        at org.elasticsearch.server@9.0.0-SNAPSHOT/org.elasticsearch.indices.cluster.IndicesClusterStateService.createIndicesAndUpdateShards(IndicesClusterStateService.java:583)
        at org.elasticsearch.server@9.0.0-SNAPSHOT/org.elasticsearch.indices.cluster.IndicesClusterStateService.doApplyClusterState(IndicesClusterStateService.java:306)
        at org.elasticsearch.server@9.0.0-SNAPSHOT/org.elasticsearch.indices.cluster.IndicesClusterStateService.applyClusterState(IndicesClusterStateService.java:260)
        at org.elasticsearch.server@9.0.0-SNAPSHOT/org.elasticsearch.cluster.service.ClusterApplierService.callClusterStateAppliers(ClusterApplierService.java:544)
        at org.elasticsearch.server@9.0.0-SNAPSHOT/org.elasticsearch.cluster.service.ClusterApplierService.callClusterStateAppliers(ClusterApplierService.java:530)
        at org.elasticsearch.server@9.0.0-SNAPSHOT/org.elasticsearch.cluster.service.ClusterApplierService.applyChanges(ClusterApplierService.java:503)
        at org.elasticsearch.server@9.0.0-SNAPSHOT/org.elasticsearch.cluster.service.ClusterApplierService.runTask(ClusterApplierService.java:432)
        at org.elasticsearch.server@9.0.0-SNAPSHOT/org.elasticsearch.cluster.service.ClusterApplierService$UpdateTask.run(ClusterApplierService.java:157)
        at org.elasticsearch.server@9.0.0-SNAPSHOT/org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:956)
        at org.elasticsearch.server@9.0.0-SNAPSHOT/org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.runAndClean(PrioritizedEsThreadPoolExecutor.java:218)
        at org.elasticsearch.server@9.0.0-SNAPSHOT/org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.run(PrioritizedEsThreadPoolExecutor.java:184)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
        at java.base/java.lang.Thread.run(Thread.java:1575)

The mode parameter no longer gets serialized for new indices. However on the older nodes still serialize the mode parameter, which caused the menioned assertion to fail. Reverting for now and see how best to address this bwc serialization issue.

We can only stop serializing mode, when all nodes are on the same version. Unfortunately we can't invoke c.clusterTransportVersion().get() from parser or builder, because that calling thread isn't allowed to call clusterService.state().

@martijnvg martijnvg added >non-issue :StorageEngine/Mapping The storage related side of mappings v9.0.0 labels Nov 20, 2024
@martijnvg martijnvg requested review from a team as code owners November 20, 2024 12:27
@martijnvg martijnvg changed the title Revert "Deprecate _source.mode in mappings (#)" Revert "Deprecate _source.mode in mappings (#116689)" Nov 20, 2024
@elasticsearchmachine
Copy link
Copy Markdown
Collaborator

elasticsearchmachine commented Nov 20, 2024

Pinging @elastic/es-storage-engine (Team:StorageEngine)

@martijnvg martijnvg merged commit ac06a84 into elastic:main Nov 20, 2024
@kkrik-es kkrik-es mentioned this pull request Nov 20, 2024
Merged
martijnvg added a commit to martijnvg/elasticsearch that referenced this pull request Nov 20, 2024
@martijnvg
Reapply "Deprecate _source.mode in mappings (elastic#116689)" (elasti…
a86ff5b 
…c#117150)

This reverts commit ac06a84.
@martijnvg martijnvg mentioned this pull request Nov 20, 2024
Closed
martijnvg added a commit to martijnvg/elasticsearch that referenced this pull request Nov 20, 2024
@martijnvg
Add back version that got removed from a revert PR (elastic#117150)
1294c24
martijnvg added a commit to martijnvg/elasticsearch that referenced this pull request Nov 20, 2024
@martijnvg
Add back version that got removed from a revert PR (elastic#117150)
da518b2
rjernst pushed a commit to rjernst/elasticsearch that referenced this pull request Nov 20, 2024
@martijnvg @rjernst
Revert "Deprecate _source.mode in mappings (elastic#116689)" (elastic…
fe1632f 
…#117150)

This reverts commit 0d7b90e, because of bwc testing failures.
alexey-ivanov-es pushed a commit to alexey-ivanov-es/elasticsearch that referenced this pull request Nov 28, 2024
@martijnvg @alexey-ivanov-es
Revert "Deprecate _source.mode in mappings (elastic#116689)" (elastic…
d3692f6 
…#117150)

This reverts commit 0d7b90e, because of bwc testing failures.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@kkrik-es kkrik-es kkrik-es approved these changes

Assignees

No one assigned

Labels

>non-issue :StorageEngine/Mapping The storage related side of mappings Team:StorageEngine v9.0.0

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@martijnvg @elasticsearchmachine @kkrik-es