Skip to content

BulkProcessor no error when dynamic is strict #55043

@moifort

Description

@moifort

Elasticsearch version (bin/elasticsearch --version): docker image elasticsearch:7.4.2

Plugins installed: [] default

JVM version (java -version): docker image elasticsearch:7.4.2

OS version (uname -a if on a Unix-like system): docker image elasticsearch:7.4.2

elasticsearch-rest-high-level-client : 7.6.2 on JDK 11

Description of the problem including expected versus actual behavior:

When using dynamic: strict mapping setting:

  • bulk processor return success instead of error when data to save has new field.
  • no data is store on Elasticsearch.

Steps to reproduce:

mapping.json

{
  "dynamic": "strict",
  "properties": {
    "id": {
      "type": "long"
    },
    "name": {
      "type": "text",
      "analyzer": "french_light",
      "boost": 10
    }
}

storeData.java

class StoreData {  
  private final RestHighLevelClient client;

  public seed() {
    try (var bulkProcessor = BulkProcessor
            .builder(client::bulkAsync, new BulkListener())
            .setBulkActions(500)
            .setConcurrentRequests(0)
            .setGlobalIndex("myIndex")
            .build()) {
            productDao.listIndexable(product -> {
                var indexProduct = new IndexRequest()
                    .id(product.getId().toString())
                    .source(elasticMapper.toIndexableProductViewJson(product), XContentType.JSON);
                bulkProcessor.add(indexProduct);
            });
        }
   }

  public void bulkAsync(BulkRequest request, ActionListener<BulkResponse> bulkListener) {
        client.bulkAsync(request, RequestOptions.DEFAULT, bulkListener);
    }

}

bulkListener.java

public class BulkListener implements BulkProcessor.Listener {

        @Override
        public void beforeBulk(long executionId, BulkRequest request) {
            var numberOfActions = request.numberOfActions();
            LOG.trace("Executing bulk [{}] with {} requests", executionId, numberOfActions);
        }

        @Override
        public void afterBulk(long executionId, BulkRequest request, BulkResponse response) {
            LOG.trace("Bulk [{}] completed in {} milliseconds", executionId, response.getTook().getMillis());
        }

        @Override
        public void afterBulk(long executionId, BulkRequest request, Throwable failure) {
            LOG.error("Failed to execute bulk", failure);
        }
    }

Provide logs (if relevant):

When I call StoreData.seed() with field that is no defined on mapping.json

log

[2020-04-10 07:45:35,522] [elastic-product-indexer-2-thread-1] INFO  o.m.i.e.ElasticClient$BulkListener - Executing bulk [1] with 500 requests
[2020-04-10 07:45:37,236] [I/O dispatcher 1] DEBUG o.e.c.RestClient - request [POST http://localhost:9200/_bulk?timeout=1m] returned [HTTP/1.1 200 OK]
[2020-04-10 07:45:37,649] [I/O dispatcher 1] INFO  o.m.i.e.ElasticClient$BulkListener - Bulk [1] completed in 2106 milliseconds
[2020-04-10 07:45:37,750] [elastic-product-indexer-2-thread-1] INFO  o.m.i.e.ElasticClient$BulkListener - Executing bulk [2] with 500 requests
[2020-04-10 07:45:39,428] [I/O dispatcher 1] DEBUG o.e.c.RestClient - request [POST http://localhost:9200/_bulk?timeout=1m] returned [HTTP/1.1 200 OK]
[2020-04-10 07:45:39,445] [I/O dispatcher 1] INFO  o.m.i.e.ElasticClient$BulkListener - Bulk [2] completed in 1694 milliseconds
[2020-04-10 07:45:39,461] [elastic-product-indexer-2-thread-1] INFO  o.m.i.e.ElasticClient$BulkListener - Executing bulk [3] with 184 requests
[2020-04-10 07:45:40,164] [I/O dispatcher 1] DEBUG o.e.c.RestClient - request [POST http://localhost:9200/_bulk?timeout=1m] returned [HTTP/1.1 200 OK]
[2020-04-10 07:45:40,170] [I/O dispatcher 1] INFO  o.m.i.e.ElasticClient$BulkListener - Bulk [3] completed in 708 milliseconds

kibana

GET myIndex/_search

{
  "took" : 0,
  "timed_out" : false,
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 0,
      "relation" : "eq"
    },
    "max_score" : null,
    "hits" : [ ]
  }
}

Metadata

Metadata

Assignees

Labels

:Search Foundations/MappingIndex mappings, including merging and defining field types>docsGeneral docs changesTeam:Search FoundationsMeta label for the Search Foundations team in Elasticsearch

Type

No type
No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions