-
Notifications
You must be signed in to change notification settings - Fork 25.8k
Better storage of _source #9034
Copy link
Copy link
Closed
Labels
:Search Foundations/MappingIndex mappings, including merging and defining field typesIndex mappings, including merging and defining field types>enhancementTeam:Search FoundationsMeta label for the Search Foundations team in ElasticsearchMeta label for the Search Foundations team in Elasticsearchhigh hanging fruit
Metadata
Metadata
Assignees
Labels
:Search Foundations/MappingIndex mappings, including merging and defining field typesIndex mappings, including merging and defining field types>enhancementTeam:Search FoundationsMeta label for the Search Foundations team in ElasticsearchMeta label for the Search Foundations team in Elasticsearchhigh hanging fruit
Type
Fields
Give feedbackNo fields configured for issues without a type.
Today we store the
_sourceis a single big binary stored field. While this is great for simplicity, this also has the bad side-effect to encourage to store fields individually in order to save some json parsing when there are a couple of large field values and we are only interested in some short values. Maybe we could try to be a bit smarter and store the_sourceacross several stored fields so that it would not be an issue anymore?Random idea: given a document that looks like:
{ "title": "short_string", "body": "very_very_very_very_long_string", "array": [2, 3, 10], "foo": { "foo": 42, "bar": "baz" } }we could for instance store all the top-level fields into their own stored field
or maybe even each value individually (but it becomes more complicated with arrays of objects):
Then we would have to make
_sourcefiltering aware of the way fields are stored, and for instance if we store only top-level fields into their own stored field then we could translate an include rule likefoo.*to "retrieve fieldfoo", andfoo.bar.*to "get everything underbarfor fieldfoo".