Skip to content

Improve fielddata mappings #8693

@clintongormley

Description

@clintongormley

Currently, the settings for fielddata and doc_values is quite confusing. It would be nice to make it easier to understand. For non-analyzed fields, these are the questions which need answering:

  1. Should fielddata be written to disk at index time?
  2. Should fielddata be available at search time, from disk, or in memory?
  3. If in memory, should fielddata be loaded eagerly or lazily?
  4. Regardless of disk or memory, should global ordinals be built eagerly or lazily?

This could be expressed as:

"fielddata": {
  "index_format":    "disk | disabled",
  "search_format":   "disk | memory | eager_memory | disabled",
  "global_ordinals": "lazy | eager"
}

In the same way as we can use analyzer to set both index_analyzer and search_analyzer, this could be condensed to:

"fielddata": {
  "format":          "disk | memory | eager_memory | disabled",
  "global_ordinals": "lazy | eager"
}

Analyzed string fields can't be written to disk, but they can support the fst format, so they would accept:

"fielddata": {
  "format":          "memory | eager_memory | fst | eager_fst | disabled",
  "global_ordinals": "lazy | eager"
}

We could possibly even support a very simple format for setting the fielddata format:

"fielddata": "disk | memory | eager_memory | disabled"            # not_analyzed
"fielddata": "memory | eager_memory | fst | eager_fst | disabled" # analyzed

... which would set global_ordinals to `lazy``

Regardless of which format is used to set fielddata, the mappings would be converted to the full index_format, search_format, global_ordinals layout.

Question: Should index_format and search_format instead by store_format and load_format?

Metadata

Metadata

Assignees

No one assigned

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions