Skip to content

[Filebeat] Update HA Proxy log grok patterns#25835

Merged
P1llus merged 3 commits intoelastic:masterfrom
legoguy1000:25827-haproxy-grok
Jun 23, 2021
Merged

[Filebeat] Update HA Proxy log grok patterns#25835
P1llus merged 3 commits intoelastic:masterfrom
legoguy1000:25827-haproxy-grok

Conversation

@legoguy1000
Copy link
Copy Markdown
Contributor

@legoguy1000 legoguy1000 commented May 25, 2021 •
edited
Loading

What does this PR do?

Update the HA Proxy grok patterns to not fail when the source IP/address is not recorded (recorded as -)

Why is it important?

According to the issue #25827 there are occasions that HA Proxy doesn't log the source IP. With the current grok patterns, the ingest pipeline will fail in that situation.

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have made corresponding change to the default configuration files
  • I have added tests that prove my fix is effective or that my feature works
  • I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Author's Checklist

  • [ ]

How to test this PR locally

cd beats/filebeat
TESTING_FILEBEAT_MODULES=haproxy TESTING_FILEBEAT_FILESETS=log mage -v pythonIntegTest

Related issues

Use cases

Screenshots

Logs

@botelastic botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label May 25, 2021
@legoguy1000 legoguy1000 marked this pull request as ready for review May 25, 2021 02:36
@legoguy1000
Copy link
Copy Markdown
Contributor Author

legoguy1000 commented May 25, 2021

@maarsaks Let me know if these changes work for you??

@elasticmachine
Copy link
Copy Markdown
Contributor

elasticmachine commented May 25, 2021 •
edited by jenkins-beats-ci bot
Loading

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview

Expand to view the summary

Build stats

  • Build Cause: P1llus commented: /test

  • Start Time: 2021-06-23T12:55:20.450+0000

  • Duration: 95 min 20 sec

  • Commit: 1918c9b

Test stats 🧪

Test Results
Failed 0
Passed 14111
Skipped 2311
Total 16422

Trends 🧪

Image of Build Times

Image of Tests

💚 Flaky test report

Tests succeeded.

Expand to view the summary

Test stats 🧪

Test Results
Failed 0
Passed 14111
Skipped 2311
Total 16422

@mergify
Copy link
Copy Markdown
Contributor

mergify bot commented May 25, 2021

This pull request is now in conflicts. Could you fix it? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b 25827-haproxy-grok upstream/25827-haproxy-grok
git merge upstream/master
git push upstream 25827-haproxy-grok

@legoguy1000 legoguy1000 force-pushed the 25827-haproxy-grok branch from abd70a7 to 5582e6c Compare May 25, 2021 20:14
@elasticmachine
Copy link
Copy Markdown
Contributor

elasticmachine commented May 26, 2021

Pinging @elastic/security-external-integrations (Team:Security-External Integrations)

@botelastic botelastic bot removed the needs_team Indicates that the issue/PR needs a Team:* label label May 26, 2021
@P1llus
Copy link
Copy Markdown
Member

P1llus commented May 26, 2021

run tests

@P1llus P1llus self-assigned this May 26, 2021
@mergify
Copy link
Copy Markdown
Contributor

mergify bot commented May 26, 2021

This pull request is now in conflicts. Could you fix it? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b 25827-haproxy-grok upstream/25827-haproxy-grok
git merge upstream/master
git push upstream 25827-haproxy-grok

@P1llus
Copy link
Copy Markdown
Member

P1llus commented Jun 23, 2021

/test

@P1llus P1llus merged commit a300f1b into elastic:master Jun 23, 2021
@P1llus P1llus added the backport-v7.14.0 Automated backport with mergify label Jun 23, 2021
mergify bot pushed a commit that referenced this pull request Jun 23, 2021
@legoguy1000
[Filebeat] Update HA Proxy log grok patterns (#25835)
7626461 
* #25827: Update HA Proxy log grok patterns

* update changelog

* add more sample data

(cherry picked from commit a300f1b)
@mergify mergify bot mentioned this pull request Jun 23, 2021
Merged
jsoriano pushed a commit that referenced this pull request Jun 24, 2021
@mergify @legoguy1000
[Filebeat] Update HA Proxy log grok patterns (#25835) (#26446)
d7f1d6c 
* #25827: Update HA Proxy log grok patterns

* update changelog

* add more sample data

(cherry picked from commit a300f1b)

Co-authored-by: Alex Resnick <adr8292@gmail.com>
mdelapenya added a commit to mdelapenya/beats that referenced this pull request Jun 28, 2021
@mdelapenya
Merge branch 'master' into archive-system-tests
b6dbec3 
* master: (32 commits)
  [Metricbeat] Change Account ID to Project ID in `gcp.billing` module (elastic#26412)
  update libbeat fields.ecs.yml file and ecsVersion to 1.10.0 (elastic#26121)
  [Filebeat] Update AWS ELB ingest pipeline (elastic#26441)
  [FIlebeat] add strict_date_optional_time_nanos date format to PanOS module (elastic#26158)
  Fix the irregular and typo on prometheus module. (elastic#25726)
  [Filebeat] Parse additonal debug data fields for Okta module (elastic#25818)
  fix: update MSSQL Server linux image's Docker registry (elastic#26440)
  Update indexing.go godocs (elastic#26408)
  Do not close filestream harvester if an unexpected error is returned when close.on_state_change.* is enabled (elastic#26411)
  Add support for copytruncate method when rotating input logs with an external tool in `filestream` input (elastic#23457)
  Allow fields with ip_range datatype (elastic#26444)
  Add Anomali ThreatStream support to threatintel module (elastic#26350)
  fix: use the right param type (elastic#26469)
  [Automation] Update elastic stack version to 8.0.0-7640093f for testing (elastic#26460)
  Set SM Filebeat modules as GA (elastic#26226)
  Fix rfc5464 date parsing in the syslog input (elastic#26419)
  Add linked account information into billing metricset (elastic#26285)
  [Filebeat] Update HA Proxy log grok patterns (elastic#25835)
  disable metricbeat logstash test_node_stats (elastic#26436)
  chore: pass BEAT_VERSION when running E2E tests (elastic#26291)
  ...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@P1llus P1llus P1llus approved these changes

Assignees

@P1llus P1llus

Labels

backport-v7.14.0 Automated backport with mergify enhancement Filebeat Filebeat

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Can’t parse haproxy logs

4 participants

@legoguy1000 @elasticmachine @P1llus @jamiehynds