Elasticsearch version: 5.0.0
Plugins installed: none
JVM version: any
OS version: any
Description of the problem including expected versus actual behavior:
When you upgrade elasticsearch with an index using a string field with fielddata disabled from 2.x to 5.x, and you try to shrink it, it fails saying that the index has unsupported parameters: [fielddata : false].
Basically it looks like when we upgrade an elasticsearch from 2.x to 5.x it changes the following:
{
"mappings": {
"type1": {
"properties": {
"@version": {
"type": "string",
"index": "not_analyzed",
"fielddata": {
"format": "disabled"
}
}
}
}
}
}
Into a 5.x mapping withe the following structure:
{
"doc": {
"mappings": {
"type1": {
"properties": {
"@version": {
"type": "string",
"index": "not_analyzed",
"fielddata": false
}
}
}
}
}
}
Then you try to shrink that index, and since string doesn't exist anymore it doesn't know what to do. We should consider either add a note that it's required to shrink an index that it was created in 5.x, or fix this bug by shrinking old indices as well.
Steps to reproduce:
[1] Create an index in 2.x with the following:
PUT doc
{
"mappings": {
"type1": {
"properties": {
"@version": {
"type": "string",
"index": "not_analyzed",
"fielddata": {
"format": "disabled"
}
}
}
}
}
}
[2] Upgrade this elasticsearch instance, or copy the index into the data directory of a 5.x instance so it's picked up as dangling index.
[3] Verify that the index was upgraded:
GET doc/type1/_mapping
{
"doc": {
"mappings": {
"type1": {
"properties": {
"@version": {
"type": "string",
"index": "not_analyzed",
"fielddata": false
}
}
}
}
}
}
[4] Shrink the index:
PUT doc/_settings
{
"settings": {
"index.blocks.write": true
}
}
POST doc/_shrink/doc_shrinked_index
{
"settings": {
"index.number_of_replicas": 1,
"index.number_of_shards": 1,
"index.codec": "best_compression"
}
}
[5] Verify logs
Provide logs (if relevant):
[2016-12-29T16:14:00,908][WARN ][o.e.c.a.s.ShardStateAction] [K-E6Kbx] [doc_shrinked_index][0] received shard failed for shard id [[doc_shrinked_index][0]], allocation id [wbxqZDZSRZ6YCYq2iCIfEQ], primary term [0], message [failed recovery], failure [RecoveryFailedException[[doc_shrinked_index][0]: Recovery failed from null into {K-E6Kbx}{K-E6Kbx4TryyqoNBVUZF-w}{C3ZFU6q-QxK01plsEjaRyQ}{127.0.0.1}{127.0.0.1:9300}{box_type=medium}]; nested: MapperParsingException[Mapping definition for [@version] has unsupported parameters: [fielddata : false]]; ]
org.elasticsearch.indices.recovery.RecoveryFailedException: [doc_shrinked_index][0]: Recovery failed from null into {K-E6Kbx}{K-E6Kbx4TryyqoNBVUZF-w}{C3ZFU6q-QxK01plsEjaRyQ}{127.0.0.1}{127.0.0.1:9300}{box_type=medium}
at org.elasticsearch.index.shard.IndexShard.lambda$startRecovery$4(IndexShard.java:1536) [elasticsearch-5.0.0.jar:5.0.0]
at org.elasticsearch.index.shard.IndexShard$$Lambda$1653/403275356.run(Unknown Source) [elasticsearch-5.0.0.jar:5.0.0]
at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:444) [elasticsearch-5.0.0.jar:5.0.0]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_45]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_45]
at java.lang.Thread.run(Thread.java:745) [?:1.8.0_45]
Caused by: org.elasticsearch.index.mapper.MapperParsingException: Mapping definition for [@version] has unsupported parameters: [fielddata : false]
at org.elasticsearch.index.mapper.DocumentMapperParser.checkNoRemainingFields(DocumentMapperParser.java:146) ~[elasticsearch-5.0.0.jar:5.0.0]
at org.elasticsearch.index.mapper.DocumentMapperParser.checkNoRemainingFields(DocumentMapperParser.java:141) ~[elasticsearch-5.0.0.jar:5.0.0]
at org.elasticsearch.index.mapper.ObjectMapper$TypeParser.parseProperties(ObjectMapper.java:289) ~[elasticsearch-5.0.0.jar:5.0.0]
at org.elasticsearch.index.mapper.ObjectMapper$TypeParser.parseObjectOrDocumentTypeProperties(ObjectMapper.java:203) ~[elasticsearch-5.0.0.jar:5.0.0]
at org.elasticsearch.index.mapper.RootObjectMapper$TypeParser.parse(RootObjectMapper.java:102) ~[elasticsearch-5.0.0.jar:5.0.0]
at org.elasticsearch.index.mapper.DocumentMapperParser.parse(DocumentMapperParser.java:110) ~[elasticsearch-5.0.0.jar:5.0.0]
at org.elasticsearch.index.mapper.DocumentMapperParser.parse(DocumentMapperParser.java:91) ~[elasticsearch-5.0.0.jar:5.0.0]
at org.elasticsearch.index.mapper.MapperService.parse(MapperService.java:508) ~[elasticsearch-5.0.0.jar:5.0.0]
at org.elasticsearch.cluster.metadata.MetaDataMappingService$PutMappingExecutor.applyRequest(MetaDataMappingService.java:276) ~[elasticsearch-5.0.0.jar:5.0.0]
at org.elasticsearch.cluster.metadata.MetaDataMappingService$PutMappingExecutor.execute(MetaDataMappingService.java:241) ~[elasticsearch-5.0.0.jar:5.0.0]
at org.elasticsearch.cluster.service.ClusterService.runTasksForExecutor(ClusterService.java:555) ~[elasticsearch-5.0.0.jar:5.0.0]
at org.elasticsearch.cluster.service.ClusterService$UpdateTask.run(ClusterService.java:894) ~[elasticsearch-5.0.0.jar:5.0.0]
at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:444) ~[elasticsearch-5.0.0.jar:5.0.0]
at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.runAndClean(PrioritizedEsThreadPoolExecutor.java:237) ~[elasticsearch-5.0.0.jar:5.0.0]
at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.run(PrioritizedEsThreadPoolExecutor.java:200) ~[elasticsearch-5.0.0.jar:5.0.0]
... 3 more
[2016-12-29T16:14:00,911][INFO ][o.e.c.r.a.AllocationService] [K-E6Kbx] Cluster health status changed from [YELLOW] to [RED] (reason: [shards failed [[doc_shrinked_index][0]] ...]).
*Note that it went from YELLOW to RED since the shard was not able to start
Elasticsearch version: 5.0.0
Plugins installed: none
JVM version: any
OS version: any
Description of the problem including expected versus actual behavior:
When you upgrade elasticsearch with an index using a string field with fielddata disabled from 2.x to 5.x, and you try to shrink it, it fails saying that the index has unsupported parameters:
[fielddata : false].Basically it looks like when we upgrade an elasticsearch from 2.x to 5.x it changes the following:
Into a 5.x mapping withe the following structure:
Then you try to shrink that index, and since string doesn't exist anymore it doesn't know what to do. We should consider either add a note that it's required to shrink an index that it was created in 5.x, or fix this bug by shrinking old indices as well.
Steps to reproduce:
[1] Create an index in 2.x with the following:
[2] Upgrade this elasticsearch instance, or copy the index into the data directory of a 5.x instance so it's picked up as dangling index.
[3] Verify that the index was upgraded:
[4] Shrink the index:
[5] Verify logs
Provide logs (if relevant):
*Note that it went from YELLOW to RED since the shard was not able to start