Skip to content

Values above ignore_above mapping limit don't show up in Discover anymore #101232

@andrewkroh

Description

@andrewkroh

Kibana version: 7.13.1

Elasticsearch version: 7.13.1

Server OS version: Elastic Cloud hosted

Browser version: Firefox 89.0

Browser OS version: macOS

Original install method (e.g. download page, yum, from source, etc.):

Describe the bug:

The event.original field is missing in Kibana Discover for some events.

Steps to reproduce:

  1. Ingest data using a Beat that creates event.original.
  2. Open discover and look at the field event.original. Note that some events appear to be missing event.original.
  3. Get one of the event's _id and query it directly from Elasticsearch and it will have event.original.
  4. I'm attaching a file that contains both the Kibana discover JSON and the direct ES query output.

missing-event.original.txt

Expected behavior:

event.original would be displayed. Or if it is being dropped for some reason, such as length, that there would be an indication that this is happening.

One interesting data point is that querying for _id:aKSBznkBdD5BTT25BM6w and event.original:* returns no results so even the KQL query seems to think that event.original does not exist despite aKSBznkBdD5BTT25BM6w having an event.original.

Screenshots (if relevant):

Errors in browser console (if relevant): There are none.

Provide logs and/or server output (if relevant):

Any additional context:

I'm not sure if it has something to do with the length of the field or possibly the field's content. I saw this in cases where the event.original contained both JSON and in a Windows event log use case where event.original contained a big XML blob.

My workaround has been to modify Kibana advanced settings to turn on discover:searchFieldsFromSource (this will affect runtime fields and fields not in _source).

Metadata

Metadata

Assignees

Labels

Feature:DiscoverDiscover ApplicationTeam:DataDiscoveryDiscover, search (data plugin and KQL), data views, saved searches. For ES|QL, use Team:ES|QL. t//blockedbugFixes for quality problems that affect the customer experienceimpact:highAddressing this issue will have a high level of impact on the quality/strength of our product.regression

Type

No type
No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions