{"id":2266,"date":"2016-01-17T00:35:59","date_gmt":"2016-01-16T23:35:59","guid":{"rendered":"http:\/\/codingexplained.com\/?p=2266"},"modified":"2017-06-11T18:07:55","modified_gmt":"2017-06-11T16:07:55","slug":"meta-fields","status":"publish","type":"post","link":"https:\/\/codingexplained.com\/coding\/elasticsearch\/meta-fields","title":{"rendered":"Meta Fields"},"content":{"rendered":"

Now that we have taken a look at field data types, it is time to talk about meta fields.<\/p>\n

Introduction to Meta Fields<\/h2>\n

So, just what are meta fields? Each document that is indexed in Elasticsearch has metadata associated with it. The names of these fields are prefixed with an underscore, and some of them can be customized when creating a mapping type. We\u00a0will go through\u00a0the various meta fields below.<\/p>\n

Identity Meta Fields<\/h2>\n

Now we\u00a0will discuss the various categories of meta fields, the first one being identity meta fields<\/em>.<\/p>\n

_index<\/h3>\n

The _index<\/span> field contains the name of the index that a document belongs to and allows for matching documents based on the index that it is stored within. It is actually a virtual field and is therefore not stored as a real field within Lucene.<\/p>\n

_type<\/h3>\n

The _type<\/span> field stores the mapping type of the document – this could be employee<\/span>, for instance. This field is indexed to make searching for specific types of documents fast.<\/p>\n

_id & _uid<\/h3>\n

The _id<\/span> field contains the ID of the document. The field is not indexed, because it can actually be derived from the _uid<\/span> field, which contains both the document ID and type, separated by a hashtag – {type}#{id}<\/span>. You are probably not going to make use of this field, because the _id<\/span> and _type<\/span> fields are made available to you for convenience, but it is nevertheless always good to know what happens behind the scenes.<\/p>\n

Document Source Meta Fields<\/h2>\n

The next category of meta fields is document source meta fields<\/em>, which contains two fields.<\/p>\n

_source<\/h3>\n

The _source<\/span> field contains the JSON that was passed to Elasticsearch at index time, or the updated values if the document has since been updated. The field is not indexed, and therefore it is not searchable, but it is returned in search or fetch results. It can be disabled if you want to save storage space.<\/p>\n

_size<\/h3>\n

The _size<\/span> field contains the size of the _source<\/span> field in bytes. However, it’s only available after installing the mapper-size<\/span> plugin, which can be done with the below command.<\/p>\n

sudo bin\/plugin install mapper-size<\/code><\/pre>\n
Indexing Meta Fields<\/h2>\n
The next category of meta fields is indexing meta fields<\/em>.<\/p>\n
_all<\/h3>\n
The _all<\/span> meta field contains the concatenated values of all other fields, using a space as the delimiter. It is analyzed and indexed, but not stored, meaning that it can be searched but not retrieved. This field is useful if you want to search documents for a value but do not know or care in which field the value is present.<\/p>\n
_field_names<\/h3>\n
The _field_names<\/span> field indexes the names of the fields for a document that contains a value other than NULL<\/span>. This is used by the exists<\/span> and missing<\/span> queries to find documents that have or do not have a value for a given field.<\/p>\n
Routing Meta Fields<\/h2>\n
The next category is routing meta fields<\/em>.<\/p>\n
_parent<\/h3>\n
The first routing meta field that we will talk about, is the _parent<\/span> field. This makes it possible to define parent-child relationships between documents within an index, by making one mapping type the parent of another mapping type. This is very similar to having a foreign key to the same table within a relational database.<\/p>\n
\"mappings\": {\r\n\t\"employee\": {\r\n\t\t\"_parent\": {\r\n\t\t\t\"type\": \"person\" \r\n\t\t}\r\n\t}\r\n}<\/code><\/pre>\n
Consider the above example, where the parent type of the employee<\/span> type is set to the person<\/span> mapping type. When you add a document to an index, you can specify the parent ID in the URL to establish the relationship for a particular document. There are then a number of queries that can be used to require a given relationship to exist, for example.<\/p>\n
_routing<\/h3>\n
The _routing<\/span> meta field is used to route a document to a particular shard within an index, and enables you to define rules that control in which shards documents are stored. Chances are that you are fine with the defaults that Elasticsearch provides, but for advanced use cases, this is indeed a possibility. Nevertheless, the chances that you will ever need this are slim, but it is good to know that the possibility exists.<\/p>\n
Other Meta Fields<\/h2>\n
Lastly, there is one meta field that doesn’t fit into any of the other categories, namely the _meta<\/span> field.<\/p>\n
_meta<\/h3>\n
The _meta<\/span> field is used to store application-specific meta data, which can be anything you want. This is possible because each mapping type can have custom meta data associated with it, and Elasticsearch does not do anything with this meta data except storing and retrieving it.<\/p>\n
That’s it! That concludes the theoretical part of mapping in Elasticsearch. It is now time to make use of this theory and add a mapping to our index in the next article.<\/p>\n","protected":false},"excerpt":{"rendered":"
Part 10 of 35 in the Complete Guide to Elasticsearch<\/a> series<\/div>
Now that we have taken a look at field data types, it is time to talk about meta fields. Introduction to Meta Fields So, just what are meta fields? Each document that is indexed in Elasticsearch has metadata associated with it. The names of these fields are prefixed with an underscore, and some of them… read more<\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"inline_featured_image":false,"jetpack_post_was_ever_published":false,"jetpack_publicize_message":"Meta Fields","jetpack_is_tweetstorm":false,"jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":true,"jetpack_social_options":{"image_generator_settings":{"template":"highway","enabled":false}}},"categories":[154],"tags":[155],"series":[163],"jetpack_publicize_connections":[],"yoast_head":"\nMeta Fields<\/title>\n<meta name=\"description\" content=\"Learn about the various meta fields in Elasticsearch right here, including _index, _type, _id and _uid.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/codingexplained.com\/coding\/elasticsearch\/meta-fields\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Meta Fields\" \/>\n<meta property=\"og:description\" content=\"Learn about the various meta fields in Elasticsearch right here, including _index, _type, _id and _uid.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/codingexplained.com\/coding\/elasticsearch\/meta-fields\" \/>\n<meta property=\"og:site_name\" content=\"Coding Explained\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/codingexplained\" \/>\n<meta property=\"article:author\" content=\"https:\/\/www.facebook.com\/codingexplained\" \/>\n<meta property=\"article:published_time\" content=\"2016-01-16T23:35:59+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2017-06-11T16:07:55+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/codingexplained.com\/wp-content\/uploads\/2015\/11\/codingexplained-fb-promote.png\" \/>\n\t<meta property=\"og:image:width\" content=\"1200\" \/>\n\t<meta property=\"og:image:height\" content=\"444\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"author\" content=\"Bo Andersen\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@codingexplained\" \/>\n<meta name=\"twitter:site\" content=\"@codingexplained\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Bo Andersen\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"4 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/codingexplained.com\/coding\/elasticsearch\/meta-fields\",\"url\":\"https:\/\/codingexplained.com\/coding\/elasticsearch\/meta-fields\",\"name\":\"Meta Fields\",\"isPartOf\":{\"@id\":\"https:\/\/codingexplained.com\/#website\"},\"datePublished\":\"2016-01-16T23:35:59+00:00\",\"dateModified\":\"2017-06-11T16:07:55+00:00\",\"author\":{\"@id\":\"https:\/\/codingexplained.com\/#\/schema\/person\/e19c92ec991f571605f047cefeaa950d\"},\"description\":\"Learn about the various meta fields in Elasticsearch right here, including _index, _type, _id and _uid.\",\"breadcrumb\":{\"@id\":\"https:\/\/codingexplained.com\/coding\/elasticsearch\/meta-fields#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/codingexplained.com\/coding\/elasticsearch\/meta-fields\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/codingexplained.com\/coding\/elasticsearch\/meta-fields#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/codingexplained.com\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Meta Fields\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/codingexplained.com\/#website\",\"url\":\"https:\/\/codingexplained.com\/\",\"name\":\"Coding Explained\",\"description\":\"\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/codingexplained.com\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/codingexplained.com\/#\/schema\/person\/e19c92ec991f571605f047cefeaa950d\",\"name\":\"Bo Andersen\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/codingexplained.com\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/28f5826f9d5d544b0c5e1ec321dfdfb8?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/28f5826f9d5d544b0c5e1ec321dfdfb8?s=96&d=mm&r=g\",\"caption\":\"Bo Andersen\"},\"description\":\"I am a back-end web developer with a passion for open source technologies. I have been a PHP developer for many years, and also have experience with Java and Spring Framework. I currently work full time as a lead developer. Apart from that, I also spend time on making online courses, so be sure to check those out!\",\"sameAs\":[\"https:\/\/codingexplained.com\",\"https:\/\/www.facebook.com\/codingexplained\",\"https:\/\/www.linkedin.com\/in\/ba0708\",\"https:\/\/twitter.com\/codingexplained\",\"https:\/\/www.youtube.com\/c\/codingexplained\"],\"url\":\"https:\/\/codingexplained.com\/author\/andy\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Meta Fields","description":"Learn about the various meta fields in Elasticsearch right here, including _index, _type, _id and _uid.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/codingexplained.com\/coding\/elasticsearch\/meta-fields","og_locale":"en_US","og_type":"article","og_title":"Meta Fields","og_description":"Learn about the various meta fields in Elasticsearch right here, including _index, _type, _id and _uid.","og_url":"https:\/\/codingexplained.com\/coding\/elasticsearch\/meta-fields","og_site_name":"Coding Explained","article_publisher":"https:\/\/www.facebook.com\/codingexplained","article_author":"https:\/\/www.facebook.com\/codingexplained","article_published_time":"2016-01-16T23:35:59+00:00","article_modified_time":"2017-06-11T16:07:55+00:00","og_image":[{"width":1200,"height":444,"url":"https:\/\/codingexplained.com\/wp-content\/uploads\/2015\/11\/codingexplained-fb-promote.png","type":"image\/png"}],"author":"Bo Andersen","twitter_card":"summary_large_image","twitter_creator":"@codingexplained","twitter_site":"@codingexplained","twitter_misc":{"Written by":"Bo Andersen","Est. reading time":"4 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/codingexplained.com\/coding\/elasticsearch\/meta-fields","url":"https:\/\/codingexplained.com\/coding\/elasticsearch\/meta-fields","name":"Meta Fields","isPartOf":{"@id":"https:\/\/codingexplained.com\/#website"},"datePublished":"2016-01-16T23:35:59+00:00","dateModified":"2017-06-11T16:07:55+00:00","author":{"@id":"https:\/\/codingexplained.com\/#\/schema\/person\/e19c92ec991f571605f047cefeaa950d"},"description":"Learn about the various meta fields in Elasticsearch right here, including _index, _type, _id and _uid.","breadcrumb":{"@id":"https:\/\/codingexplained.com\/coding\/elasticsearch\/meta-fields#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/codingexplained.com\/coding\/elasticsearch\/meta-fields"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/codingexplained.com\/coding\/elasticsearch\/meta-fields#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/codingexplained.com\/"},{"@type":"ListItem","position":2,"name":"Meta Fields"}]},{"@type":"WebSite","@id":"https:\/\/codingexplained.com\/#website","url":"https:\/\/codingexplained.com\/","name":"Coding Explained","description":"","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/codingexplained.com\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/codingexplained.com\/#\/schema\/person\/e19c92ec991f571605f047cefeaa950d","name":"Bo Andersen","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/codingexplained.com\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/28f5826f9d5d544b0c5e1ec321dfdfb8?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/28f5826f9d5d544b0c5e1ec321dfdfb8?s=96&d=mm&r=g","caption":"Bo Andersen"},"description":"I am a back-end web developer with a passion for open source technologies. I have been a PHP developer for many years, and also have experience with Java and Spring Framework. I currently work full time as a lead developer. Apart from that, I also spend time on making online courses, so be sure to check those out!","sameAs":["https:\/\/codingexplained.com","https:\/\/www.facebook.com\/codingexplained","https:\/\/www.linkedin.com\/in\/ba0708","https:\/\/twitter.com\/codingexplained","https:\/\/www.youtube.com\/c\/codingexplained"],"url":"https:\/\/codingexplained.com\/author\/andy"}]}},"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/p3mJkW-Ay","_links":{"self":[{"href":"https:\/\/codingexplained.com\/wp-json\/wp\/v2\/posts\/2266"}],"collection":[{"href":"https:\/\/codingexplained.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/codingexplained.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/codingexplained.com\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/codingexplained.com\/wp-json\/wp\/v2\/comments?post=2266"}],"version-history":[{"count":6,"href":"https:\/\/codingexplained.com\/wp-json\/wp\/v2\/posts\/2266\/revisions"}],"predecessor-version":[{"id":2906,"href":"https:\/\/codingexplained.com\/wp-json\/wp\/v2\/posts\/2266\/revisions\/2906"}],"wp:attachment":[{"href":"https:\/\/codingexplained.com\/wp-json\/wp\/v2\/media?parent=2266"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/codingexplained.com\/wp-json\/wp\/v2\/categories?post=2266"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/codingexplained.com\/wp-json\/wp\/v2\/tags?post=2266"},{"taxonomy":"series","embeddable":true,"href":"https:\/\/codingexplained.com\/wp-json\/wp\/v2\/series?post=2266"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}