Skip to content

Add parsers examples to filestream reference configuration#30529

Merged
kvch merged 10 commits intoelastic:mainfrom
kvch:fix-filebeat-parsers-reference-configuration
Feb 22, 2022
Merged

Add parsers examples to filestream reference configuration#30529
kvch merged 10 commits intoelastic:mainfrom
kvch:fix-filebeat-parsers-reference-configuration

Conversation

@kvch
Copy link
Copy Markdown
Contributor

@kvch kvch commented Feb 22, 2022

What does this PR do?

This PR adds the missing parsers sections to the reference configuration of Filebeat.

Why is it important?

One recurring issue in filestream is users misconfiguring parsers. The documentation is somewhat detailed on the topic. However, parsers were missing from the reference configuration. I am adding more parser examples to ease adoption of filestream and decrease user frustration/confusion.

Checklist

- [ ] My code follows the style guidelines of this project
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation

  • I have made corresponding change to the default configuration files
    - [ ] I have added tests that prove my fix is effective or that my feature works
    - [ ] I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

@botelastic botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Feb 22, 2022
@kvch kvch added the Team:Elastic-Agent-Data-Plane Label for the Agent Data Plane team label Feb 22, 2022
@elasticmachine
Copy link
Copy Markdown
Contributor

elasticmachine commented Feb 22, 2022

Pinging @elastic/elastic-agent-data-plane (Team:Elastic-Agent-Data-Plane)

@botelastic botelastic bot removed the needs_team Indicates that the issue/PR needs a Team:* label label Feb 22, 2022
@mergify mergify bot assigned kvch Feb 22, 2022
@mergify
Copy link
Copy Markdown
Contributor

mergify bot commented Feb 22, 2022

This pull request does not have a backport label. Could you fix it @kvch? 🙏
To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

  • backport-v./d./d./d is the label to automatically backport to the 7./d branch. /d is the digit

NOTE: backport-skip has been added to this pull request.

@mergify mergify bot added the backport-skip Skip notification from the automated backport with mergify label Feb 22, 2022
@kvch kvch added backport-7.17 Automated backport to the 7.17 branch with mergify backport-v8.0.0 Automated backport with mergify backport-v8.1.0 Automated backport with mergify labels Feb 22, 2022
@mergify mergify bot removed the backport-skip Skip notification from the automated backport with mergify label Feb 22, 2022
rdner
rdner reviewed Feb 22, 2022
View reviewed changes
rdner
rdner reviewed Feb 22, 2022
View reviewed changes
rdner
rdner reviewed Feb 22, 2022
View reviewed changes
rdner
rdner reviewed Feb 22, 2022
View reviewed changes
rdner
rdner reviewed Feb 22, 2022
View reviewed changes
rdner
rdner reviewed Feb 22, 2022
View reviewed changes
rdner
rdner reviewed Feb 22, 2022
View reviewed changes
rdner
rdner reviewed Feb 22, 2022
View reviewed changes
kvch and others added 9 commits February 22, 2022 13:44
@kvch @rdner
Update filebeat/_meta/config/filebeat.inputs.reference.yml.tmpl
2b51b9e 
Co-authored-by: Denis Rechkunov <denis@rdner.de>
@kvch @rdner
Update filebeat/_meta/config/filebeat.inputs.reference.yml.tmpl
91f4c24 
Co-authored-by: Denis Rechkunov <denis@rdner.de>
@kvch @rdner
Update filebeat/_meta/config/filebeat.inputs.reference.yml.tmpl
082dacb 
Co-authored-by: Denis Rechkunov <denis@rdner.de>
@kvch @rdner
Update filebeat/_meta/config/filebeat.inputs.reference.yml.tmpl
dea3d96 
Co-authored-by: Denis Rechkunov <denis@rdner.de>
@kvch @rdner
Update filebeat/_meta/config/filebeat.inputs.reference.yml.tmpl
a333371 
Co-authored-by: Denis Rechkunov <denis@rdner.de>
@kvch @rdner
Update filebeat/_meta/config/filebeat.inputs.reference.yml.tmpl
096ce00 
Co-authored-by: Denis Rechkunov <denis@rdner.de>
@kvch @rdner
Update filebeat/_meta/config/filebeat.inputs.reference.yml.tmpl
3131a89 
Co-authored-by: Denis Rechkunov <denis@rdner.de>
@kvch @rdner
Update filebeat/_meta/config/filebeat.inputs.reference.yml.tmpl
97fb0a7 
Co-authored-by: Denis Rechkunov <denis@rdner.de>
@kvch
update reference
429ae2e
@elasticmachine
Copy link
Copy Markdown
Contributor

elasticmachine commented Feb 22, 2022 •
edited by jenkins-beats-ci bot
Loading

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview preview

Expand to view the summary

Build stats

  • Start Time: 2022-02-22T12:47:35.332+0000

  • Duration: 106 min 37 sec

Test stats 🧪

Test Results
Failed 0
Passed 8673
Skipped 1122
Total 9795

💚 Flaky test report

Tests succeeded.

🤖 GitHub comments

To re-run your PR in the CI, just comment with:

  • /test : Re-trigger the build.

  • /package : Generate the packages and run the E2E tests.

  • /beats-tester : Run the installation tests with beats-tester.

  • run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

@kvch kvch merged commit a159cce into elastic:main Feb 22, 2022
mergify bot pushed a commit that referenced this pull request Feb 22, 2022
@kvch
Add parsers examples to filestream reference configuration (#30529)
35ffbfe 
(cherry picked from commit a159cce)
@mergify mergify bot mentioned this pull request Feb 22, 2022
Merged
mergify bot pushed a commit that referenced this pull request Feb 22, 2022
@kvch
Add parsers examples to filestream reference configuration (#30529)
9ddb4d9 
(cherry picked from commit a159cce)
@mergify mergify bot mentioned this pull request Feb 22, 2022
Merged
mergify bot pushed a commit that referenced this pull request Feb 22, 2022
@kvch
Add parsers examples to filestream reference configuration (#30529)
0a2fed4 
(cherry picked from commit a159cce)
@mergify mergify bot mentioned this pull request Feb 22, 2022
Merged
kvch added a commit that referenced this pull request Feb 22, 2022
@mergify @kvch
Add parsers examples to filestream reference configuration (#30529)…
a3b92b4 
… (#30535)

(cherry picked from commit a159cce)

Co-authored-by: Noémi Ványi <kvch@users.noreply.github.com>
kvch added a commit that referenced this pull request Feb 22, 2022
@mergify @kvch
Add parsers examples to filestream reference configuration (#30529)…
1a025b5 
… (#30536)

(cherry picked from commit a159cce)

Co-authored-by: Noémi Ványi <kvch@users.noreply.github.com>
kvch added a commit that referenced this pull request Feb 22, 2022
@mergify @kvch
Add parsers examples to filestream reference configuration (#30529)…
061e4e2 
… (#30537)

(cherry picked from commit a159cce)

Co-authored-by: Noémi Ványi <kvch@users.noreply.github.com>
v1v added a commit to v1v/beats that referenced this pull request Feb 22, 2022
@v1v
Merge remote-tracking branch 'upstream/main' into feature/refactor-pa…
29471a7 
…ckaging-docker

* upstream/main: (26 commits)
  Update docker/distribution to 2.8.0 (elastic#30462)
  Add `parsers` examples to `filestream` reference configuration (elastic#30529)
  extend documentation about setting orchestrator.cluster fields (elastic#30518)
  Forward-port 8.0.1 changelog to main (elastic#30522)
  Switch skip to use `CI` (elastic#30512)
  packetbeat/beater: don't attempt to install npcap when already installed (elastic#30509)
  Fix Docker module: rename fields on dashboards (elastic#30500)
  fix typos and improve sentences (elastic#30432)
  Add drop and explicit tests to avoid duplicate ingest of elasticsearch logs (elastic#30440)
  {,x-pack/}auditbeat: replace uses of github.com/pkg/errors with stdlib equivalents (elastic#30321)
  Spelling fix (elastic#30439)
  packetbeat/beater: make sure Npcap installation runs before interfaces are needed in all cases (elastic#30438)
  Add BC about Homebrew no longer being available in 8.0 (elastic#30419)
  Install gawk as a replacement for mawk in Docker containers. (elastic#30452)
  Clean up python-related system tests (elastic#30415)
  Fix TestNewModuleRegistry flakiness (elastic#30453)
  [Filebeat] [auditd]: Support EXECVE events with truncated argument list (elastic#30382)
  Set `log.offset` to the start of the reported line in filestream (elastic#30445)
  clarify SelectedPackageTypes meaning and improve its usage (elastic#30142)
  [elasticsearch module] serialize shards properties (elastic#30408)
  ...
v1v added a commit to v1v/beats that referenced this pull request Feb 22, 2022
@v1v
Merge remote-tracking branch 'upstream/main' into feature/use-with-ki…
12a61e7 
…nd-k8s-env

* upstream/main:
  Update docker/distribution to 2.8.0 (elastic#30462)
  Add `parsers` examples to `filestream` reference configuration (elastic#30529)
  extend documentation about setting orchestrator.cluster fields (elastic#30518)
  Forward-port 8.0.1 changelog to main (elastic#30522)
  Switch skip to use `CI` (elastic#30512)
  packetbeat/beater: don't attempt to install npcap when already installed (elastic#30509)
  Fix Docker module: rename fields on dashboards (elastic#30500)
v1v added a commit that referenced this pull request Mar 2, 2022
@v1v
Merge branch '8.1' of github.com:elastic/beats into mergify/bp/8.1/pr…
0112148 
…-29710

* '8.1' of github.com:elastic/beats: (51 commits)
  refactor pushDockerImages (#30414) (#30624)
  ci: add windows-2022 in the extended meta-stage (#30528) (#30630)
  Curate k8s testing versions to only keep the actively maintained (#30619) (#30625)
  [8.1](backport #30355) Add Beats upgrade docs for 8.0 (#30612)
  Remove references to gcp from the Functionbeat docs (#30579) (#30609)
  x-pack/auditbeat/module/system/socket: defend against exec with zero arguments (#30586) (#30597)
  [MySQL Enterprise] Adding default paths values to manifest.yml (#30598) (#30604)
  metricbeat - fix elasticsearch and kibana integration tests failures in 8.0 (#30566) (#30594)
  Install gawk as a replacement for mawk in Docker containers. (#30452) (#30465)
  [Filebeat] Remove RecordedFuture dataset from Threat Intel module (#30564) (#30568)
  Adjust the documentation of `backoff` options in filestream input (#30552) (#30557)
  packetbeat/beater: help the GC clean up the Npcap installer if it's not used (#30513) (#30546)
  Osquerybeat: Add install verification for osquerybeat (#30388) (#30404)
  Update docker/distribution to 2.8.0 (#30462) (#30540)
  Add `parsers` examples to `filestream` reference configuration (#30529) (#30537)
  [8.1](backport #30068) ZooKeeper module: Adapt to ZooKeeper 3.6+ `mntr` response fields' changes. (#30360)
  [8.1](backport #30512) Switch skip to use `CI` (#30525)
  Forward-port 8.0.1 changelog to 8.1 (#30517)
  packetbeat/beater: don't attempt to install npcap when already installed (#30509) (#30511)
  Add drop and explicit tests to avoid duplicate ingest of elasticsearch logs (#30440) (#30488)
  ...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@rdner rdner rdner approved these changes

Assignees

@kvch kvch

Labels

backport-7.17 Automated backport to the 7.17 branch with mergify backport-v8.0.0 Automated backport with mergify backport-v8.1.0 Automated backport with mergify Team:Elastic-Agent-Data-Plane Label for the Agent Data Plane team

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@kvch @elasticmachine @rdner