Skip to content

Array of strings incorrectly indexed as dense_vector #101965

@benwtrent

Description

@benwtrent

Elasticsearch Version

8.11+

Installed Plugins

No response

Java Version

bundled

OS Version

ANY

Problem Description

When indexing an array of string values, these may be dynamically indexed as dense_vector. This is unexpected behavior as they should be indexed as text.

The error the user will see is: :"org.elasticsearch.common.ParsingException: Failed to parse object: expecting token of type [VALUE_NUMBER] but found [VALUE_STRING]

Steps to Reproduce

The following yaml test reproduces the bug:

---
"Fields indexed as strings won't be transformed into dense_vector":
  - do:
      index:
        index: strings-are-not-floats
        refresh: true
        body:
          obviously_string: ["foo", "foo", "foo", "foo", "foo", "foo", "foo", "foo", "foo", "foo",
                           "foo", "foo", "foo", "foo", "foo", "foo", "foo", "foo", "foo", "foo",
                           "foo", "foo", "foo", "foo", "foo", "foo", "foo", "foo", "foo", "foo",
                           "foo", "foo", "foo", "foo", "foo", "foo", "foo", "foo", "foo", "foo",
                           "foo", "foo", "foo", "foo", "foo", "foo", "foo", "foo", "foo", "foo",
                           "foo", "foo", "foo", "foo", "foo", "foo", "foo", "foo", "foo", "foo",
                           "foo", "foo", "foo", "foo", "foo", "foo", "foo", "foo", "foo", "foo",
                           "foo", "foo", "foo", "foo", "foo", "foo", "foo", "foo", "foo", "foo",
                           "foo", "foo", "foo", "foo", "foo", "foo", "foo", "foo", "foo", "foo",
                           "foo", "foo", "foo", "foo", "foo", "foo", "foo", "foo", "foo", "foo",
                           "foo", "foo", "foo", "foo", "foo", "foo", "foo", "foo", "foo", "foo",
                           "foo", "foo", "foo", "foo", "foo", "foo", "foo", "foo", "foo", "foo",
                           "foo", "foo", "foo", "foo", "foo", "foo", "foo", "foo"]
  - do:
      cluster.health:
        wait_for_events: languid
  - do:
      indices.get_mapping:
        index: strings-are-not-floats

  - match: { strings-are-not-floats.mappings.properties.obviously_string.type: text }

Logs (if relevant)

:"org.elasticsearch.common.ParsingException: Failed to parse object: expecting token of type [VALUE_NUMBER] but found [VALUE_STRING]
  at org.elasticsearch.server@8.12.0-SNAPSHOT/org.elasticsearch.common.xcontent.XContentParserUtils.parsingException(XContentParserUtils.java:76)
  at org.elasticsearch.server@8.12.0-SNAPSHOT/org.elasticsearch.common.xcontent.XContentParserUtils.ensureExpectedToken(XContentParserUtils.java:69)
  at org.elasticsearch.server@8.12.0-SNAPSHOT/org.elasticsearch.index.mapper.vectors.DenseVectorFieldMapper$ElementType$2.parseKnnVector(DenseVectorFieldMapper.java:489)
  at org.elasticsearch.server@8.12.0-SNAPSHOT/org.elasticsearch.index.mapper.vectors.DenseVectorFieldMapper.parseKnnVector(DenseVectorFieldMapper.java:991)
  at org.elasticsearch.server@8.12.0-SNAPSHOT/org.elasticsearch.index.mapper.vectors.DenseVectorFieldMapper.parse(DenseVectorFieldMapper.java:986)
  at org.elasticsearch.server@8.12.0-SNAPSHOT/org.elasticsearch.index.mapper.DocumentParser.parseObjectOrField(DocumentParser.java:411)
  at org.elasticsearch.server@8.12.0-SNAPSHOT/org.elasticsearch.index.mapper.DocumentParser.parseArray(DocumentParser.java:547)
  at org.elasticsearch.server@8.12.0-SNAPSHOT/org.elasticsearch.index.mapper.DocumentParser.innerParseObject(DocumentParser.java:335)
  at org.elasticsearch.server@8.12.0-SNAPSHOT/org.elasticsearch.index.mapper.DocumentParser.parseObjectOrNested(DocumentParser.java:293)
  at org.elasticsearch.server@8.12.0-SNAPSHOT/org.elasticsearch.index.mapper.DocumentParser.internalParseDocument(DocumentParser.java:144)
  at org.elasticsearch.server@8.12.0-SNAPSHOT/org.elasticsearch.index.mapper.DocumentParser.parseDocument(DocumentParser.java:96)
  at org.elasticsearch.server@8.12.0-SNAPSHOT/org.elasticsearch.index.mapper.DocumentMapper.parse(DocumentMapper.java:92)
  at org.elasticsearch.server@8.12.0-SNAPSHOT/org.elasticsearch.index.shard.IndexShard.prepareIndex(IndexShard.java:1011)
  at org.elasticsearch.server@8.12.0-SNAPSHOT/org.elasticsearch.index.shard.IndexShard.applyIndexOperation(IndexShard.java:959)
  at org.elasticsearch.server@8.12.0-SNAPSHOT/org.elasticsearch.index.shard.IndexShard.applyIndexOperationOnPrimary(IndexShard.java:903)
  at org.elasticsearch.server@8.12.0-SNAPSHOT/org.elasticsearch.action.bulk.TransportShardBulkAction.executeBulkItemRequest(TransportShardBulkAction.java:360)
  at org.elasticsearch.server@8.12.0-SNAPSHOT/org.elasticsearch.action.bulk.TransportShardBulkAction$2.doRun(TransportShardBulkAction.java:224)
  at org.elasticsearch.server@8.12.0-SNAPSHOT/org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:26)
  at org.elasticsearch.server@8.12.0-SNAPSHOT/org.elasticsearch.common.util.concurrent.TimedRunnable.doRun(TimedRunnable.java:33)
  at org.elasticsearch.server@8.12.0-SNAPSHOT/org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:983)
  at org.elasticsearch.server@8.12.0-SNAPSHOT/org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:26)
  at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
  at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
  at java.base/java.lang.Thread.run(Thread.java:1583)
"}],"type":"document_parsing_exception","reason":"[-1:19] failed to parse: Failed to parse object: expecting token of type [VALUE_NUMBER] but found [VALUE_STRING]","caused_by":{"type":"parsing_exception","reason":"Failed to parse object: expecting token of type [VALUE_NUMBER] but found [VALUE_STRING]","stack_trace":"org.elasticsearch.common.ParsingException: Failed to parse object: expecting token of type [VALUE_NUMBER] but found [VALUE_STRING]
  at org.elasticsearch.server@8.12.0-SNAPSHOT/org.elasticsearch.common.xcontent.XContentParserUtils.parsingException(XContentParserUtils.java:76)
  at org.elasticsearch.server@8.12.0-SNAPSHOT/org.elasticsearch.common.xcontent.XContentParserUtils.ensureExpectedToken(XContentParserUtils.java:69)
  at org.elasticsearch.server@8.12.0-SNAPSHOT/org.elasticsearch.index.mapper.vectors.DenseVectorFieldMapper$ElementType$2.parseKnnVector(DenseVectorFieldMapper.java:489)
  at org.elasticsearch.server@8.12.0-SNAPSHOT/org.elasticsearch.index.mapper.vectors.DenseVectorFieldMapper.parseKnnVector(DenseVectorFieldMapper.java:991)
  at org.elasticsearch.server@8.12.0-SNAPSHOT/org.elasticsearch.index.mapper.vectors.DenseVectorFieldMapper.parse(DenseVectorFieldMapper.java:986)
  at org.elasticsearch.server@8.12.0-SNAPSHOT/org.elasticsearch.index.mapper.DocumentParser.parseObjectOrField(DocumentParser.java:411)
  at org.elasticsearch.server@8.12.0-SNAPSHOT/org.elasticsearch.index.mapper.DocumentParser.parseArray(DocumentParser.java:547)
  at org.elasticsearch.server@8.12.0-SNAPSHOT/org.elasticsearch.index.mapper.DocumentParser.innerParseObject(DocumentParser.java:335)
  at org.elasticsearch.server@8.12.0-SNAPSHOT/org.elasticsearch.index.mapper.DocumentParser.parseObjectOrNested(DocumentParser.java:293)
  at org.elasticsearch.server@8.12.0-SNAPSHOT/org.elasticsearch.index.mapper.DocumentParser.internalParseDocument(DocumentParser.java:144)
  at org.elasticsearch.server@8.12.0-SNAPSHOT/org.elasticsearch.index.mapper.DocumentParser.parseDocument(DocumentParser.java:96)
  at org.elasticsearch.server@8.12.0-SNAPSHOT/org.elasticsearch.index.mapper.DocumentMapper.parse(DocumentMapper.java:92)
  at org.elasticsearch.server@8.12.0-SNAPSHOT/org.elasticsearch.index.shard.IndexShard.prepareIndex(IndexShard.java:1011)
  at org.elasticsearch.server@8.12.0-SNAPSHOT/org.elasticsearch.index.shard.IndexShard.applyIndexOperation(IndexShard.java:959)
  at org.elasticsearch.server@8.12.0-SNAPSHOT/org.elasticsearch.index.shard.IndexShard.applyIndexOperationOnPrimary(IndexShard.java:903)
  at org.elasticsearch.server@8.12.0-SNAPSHOT/org.elasticsearch.action.bulk.TransportShardBulkAction.executeBulkItemRequest(TransportShardBulkAction.java:360)
  at org.elasticsearch.server@8.12.0-SNAPSHOT/org.elasticsearch.action.bulk.TransportShardBulkAction$2.doRun(TransportShardBulkAction.java:224)
  at org.elasticsearch.server@8.12.0-SNAPSHOT/org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:26)
  at org.elasticsearch.server@8.12.0-SNAPSHOT/org.elasticsearch.common.util.concurrent.TimedRunnable.doRun(TimedRunnable.java:33)
  at org.elasticsearch.server@8.12.0-SNAPSHOT/org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:983)
  at org.elasticsearch.server@8.12.0-SNAPSHOT/org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:26)
  at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
  at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
  at java.base/java.lang.Thread.run(Thread.java:1583)
"},"stack_trace":"org.elasticsearch.index.mapper.DocumentParsingException: [-1:19] failed to parse: Failed to parse object: expecting token of type [VALUE_NUMBER] but found [VALUE_STRING]
  at org.elasticsearch.server@8.12.0-SNAPSHOT/org.elasticsearch.index.mapper.DocumentParser.wrapInDocumentParsingException(DocumentParser.java:246)
  at org.elasticsearch.server@8.12.0-SNAPSHOT/org.elasticsearch.index.mapper.DocumentParser.internalParseDocument(DocumentParser.java:153)
  at org.elasticsearch.server@8.12.0-SNAPSHOT/org.elasticsearch.index.mapper.DocumentParser.parseDocument(DocumentParser.java:96)
  at org.elasticsearch.server@8.12.0-SNAPSHOT/org.elasticsearch.index.mapper.DocumentMapper.parse(DocumentMapper.java:92)
  at org.elasticsearch.server@8.12.0-SNAPSHOT/org.elasticsearch.index.shard.IndexShard.prepareIndex(IndexShard.java:1011)
  at org.elasticsearch.server@8.12.0-SNAPSHOT/org.elasticsearch.index.shard.IndexShard.applyIndexOperation(IndexShard.java:959)
  at org.elasticsearch.server@8.12.0-SNAPSHOT/org.elasticsearch.index.shard.IndexShard.applyIndexOperationOnPrimary(IndexShard.java:903)
  at org.elasticsearch.server@8.12.0-SNAPSHOT/org.elasticsearch.action.bulk.TransportShardBulkAction.executeBulkItemRequest(TransportShardBulkAction.java:360)
  at org.elasticsearch.server@8.12.0-SNAPSHOT/org.elasticsearch.action.bulk.TransportShardBulkAction$2.doRun(TransportShardBulkAction.java:224)
  at org.elasticsearch.server@8.12.0-SNAPSHOT/org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:26)
  at org.elasticsearch.server@8.12.0-SNAPSHOT/org.elasticsearch.common.util.concurrent.TimedRunnable.doRun(TimedRunnable.java:33)
  at org.elasticsearch.server@8.12.0-SNAPSHOT/org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:983)
  at org.elasticsearch.server@8.12.0-SNAPSHOT/org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:26)
  at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
  at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
  at java.base/java.lang.Thread.run(Thread.java:1583)
Caused by: org.elasticsearch.common.ParsingException: Failed to parse object: expecting token of type [VALUE_NUMBER] but found [VALUE_STRING]
  at org.elasticsearch.server@8.12.0-SNAPSHOT/org.elasticsearch.common.xcontent.XContentParserUtils.parsingException(XContentParserUtils.java:76)
  at org.elasticsearch.server@8.12.0-SNAPSHOT/org.elasticsearch.common.xcontent.XContentParserUtils.ensureExpectedToken(XContentParserUtils.java:69)
  at org.elasticsearch.server@8.12.0-SNAPSHOT/org.elasticsearch.index.mapper.vectors.DenseVectorFieldMapper$ElementType$2.parseKnnVector(DenseVectorFieldMapper.java:489)
  at org.elasticsearch.server@8.12.0-SNAPSHOT/org.elasticsearch.index.mapper.vectors.DenseVectorFieldMapper.parseKnnVector(DenseVectorFieldMapper.java:991)
  at org.elasticsearch.server@8.12.0-SNAPSHOT/org.elasticsearch.index.mapper.vectors.DenseVectorFieldMapper.parse(DenseVectorFieldMapper.java:986)
  at org.elasticsearch.server@8.12.0-SNAPSHOT/org.elasticsearch.index.mapper.DocumentParser.parseObjectOrField(DocumentParser.java:411)
  at org.elasticsearch.server@8.12.0-SNAPSHOT/org.elasticsearch.index.mapper.DocumentParser.parseArray(DocumentParser.java:547)
  at org.elasticsearch.server@8.12.0-SNAPSHOT/org.elasticsearch.index.mapper.DocumentParser.innerParseObject(DocumentParser.java:335)
  at org.elasticsearch.server@8.12.0-SNAPSHOT/org.elasticsearch.index.mapper.DocumentParser.parseObjectOrNested(DocumentParser.java:293)
  at org.elasticsearch.server@8.12.0-SNAPSHOT/org.elasticsearch.index.mapper.DocumentParser.internalParseDocument(DocumentParser.java:144)
  ... 14 more

Metadata

Metadata

Assignees

Type

No type
No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions