[breaking] Make default_field: false the default for all fields#28596
Merged
adriansr merged 6 commits intoelastic:masterfrom Nov 6, 2021
Merged
[breaking] Make default_field: false the default for all fields#28596adriansr merged 6 commits intoelastic:masterfrom
adriansr merged 6 commits intoelastic:masterfrom
Conversation
Contributor
Author
|
Currently a draft until ECS v8.0.0 is released, which includes this necessary change elastic/ecs#1633 |
Contributor
💚 Build Succeeded
Expand to view the summary
Build stats
Test stats 🧪
💚 Flaky test reportTests succeeded. 🤖 GitHub commentsTo re-run your PR in the CI, just comment with:
|
2d3318f to
d1c14e3
Compare
fcf79ef to
6a4f03e
Compare
Contributor
|
Pinging @elastic/security-external-integrations (Team:Security-External Integrations) |
andrewkroh
approved these changes
Nov 5, 2021
Member
andrewkroh
left a comment
There was a problem hiding this comment.
LGTM. It needs a changelog entry.
mergify bot
pushed a commit
that referenced
this pull request
Nov 6, 2021
Changes the default value of the default_field flag in fields definitions to false. This means that only fields that are explicitly marked with default_fields: true (or their subfields) will be added to the index template's setting.index.query.default_field list. After this PR, all fields are excluded from default_field, except: - Selected fields from ECS. The ECS team maintains the list of fields that are included. - Fields for processors. - Fields for Filebeat inputs. (cherry picked from commit 84e668c)
adriansr
added a commit
that referenced
this pull request
Nov 8, 2021
…) (#28855) Changes the default value of the default_field flag in fields definitions to false. This means that only fields that are explicitly marked with default_fields: true (or their subfields) will be added to the index template's setting.index.query.default_field list. After this PR, all fields are excluded from default_field, except: - Selected fields from ECS. The ECS team maintains the list of fields that are included. - Fields for processors. - Fields for Filebeat inputs. (cherry picked from commit 84e668c) Co-authored-by: Adrian Serrano <adrisr83@gmail.com>
v1v
added a commit
to v1v/beats
that referenced
this pull request
Nov 8, 2021
…in-the-package-binareis * upstream/master: allows disable pod events enrichment with deployment name (elastic#28521) Remove Docker input from Filebeat (elastic#28817) [breaking] Make default_field: false the default for all fields (elastic#28596) Osquerybeat: Improve osquery client connect code (elastic#28848) Add crawler metrics into the stats metricset for Enterprise Search (elastic#28790) Remove the now deprecated appsearch module from metricbeat (elastic#28850) Remove Beat generators (elastic#28816) chore: upload files to Google Storage when they exist (elastic#28836) Revert "chore(ci): disable E2E tests in Beats (elastic#28715)" (elastic#28812) Deprecate generating custom Beats (elastic#28814) [Metricbeat] upgrade flatbuffers to 1.12.1 (elastic#28094) Osquerybeat: Fix restart flags after previously bad config (elastic#28827) Force ECS and JSON logging for libbeat/logp (elastic#28573) Filebeat: Error on startup for unconfigured module (elastic#28818) Deprecate log input in favour of filestream (elastic#28623) Fix some spelling mistakes (elastic#28080)
leweafan
pushed a commit
to leweafan/beats
that referenced
this pull request
Apr 28, 2023
…in-the-package-binareis * upstream/master: allows disable pod events enrichment with deployment name (elastic#28521) Remove Docker input from Filebeat (elastic#28817) [breaking] Make default_field: false the default for all fields (elastic#28596) Osquerybeat: Improve osquery client connect code (elastic#28848) Add crawler metrics into the stats metricset for Enterprise Search (elastic#28790) Remove the now deprecated appsearch module from metricbeat (elastic#28850) Remove Beat generators (elastic#28816) chore: upload files to Google Storage when they exist (elastic#28836) Revert "chore(ci): disable E2E tests in Beats (elastic#28715)" (elastic#28812) Deprecate generating custom Beats (elastic#28814) [Metricbeat] upgrade flatbuffers to 1.12.1 (elastic#28094) Osquerybeat: Fix restart flags after previously bad config (elastic#28827) Force ECS and JSON logging for libbeat/logp (elastic#28573) Filebeat: Error on startup for unconfigured module (elastic#28818) Deprecate log input in favour of filestream (elastic#28623) Fix some spelling mistakes (elastic#28080)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Changes the default value of the
default_fieldflag in fields definitions to false. This means that only fields that are explicitly marked withdefault_fields:true(or their subfields) will be added to the index template'setting.index.query.default_fieldlist.After this PR, all fields are excluded from default_field, except:
Why is it important?
This is done to reduce the size of the
settings.index.query.default_field, which is limited by default to 1024 entries (controlled byindices.query.bool.max_clause_count). When this limit is exceeded, some query types, such as Simple Query String, will fail. Errors can be observed in Kibana when searching without specifying a field.Checklist
[x] I have made corresponding changes to the documentation[ ] I have made corresponding change to the default configuration files[ ] I have added tests that prove my fix is effective or that my feature worksCHANGELOG.next.asciidocorCHANGELOG-developer.next.asciidoc.Related issues