[Filebeat] Parse additonal debug data fields for Okta module by legoguy1000 · Pull Request #25818 · elastic/beats

legoguy1000 · 2021-05-21T22:54:14Z

What does this PR do?

Parses the Suspicious Activity fields for the Okta module from the okta.debug_context.debug_data field.

Why is it important?

The module currently deletes the Suspicious Activity fields from the events, losing vital infmormation.

Checklist

My code follows the style guidelines of this project
I have commented my code, particularly in hard-to-understand areas
I have made corresponding changes to the documentation
I have made corresponding change to the default configuration files
I have added tests that prove my fix is effective or that my feature works
I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Author's Checklist

[ ]

How to test this PR locally

cd beats/x-pack/filebeat
TESTING_FILEBEAT_MODULES=okta TESTING_FILEBEAT_FILESETS=system mage -v pythonIntegTest

Related issues

Closes [Filebeat][Okta] Ingest Pipeline for Okta Module drops debug_context fields #25689

Use cases

Screenshots

Logs

elasticmachine · 2021-05-21T22:58:56Z

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

Build Cause: P1llus commented: run tests
Start Time: 2021-06-24T14:11:52.089+0000
Duration: 112 min 23 sec
Commit: d69a251

Test stats 🧪

Test	Results
Failed	0
Passed	14224
Skipped	2311
Total	16535

Trends 🧪

💚 Flaky test report

Tests succeeded.

Expand to view the summary

Test stats 🧪

Test	Results
Failed	0
Passed	14224
Skipped	2311
Total	16535

elasticmachine · 2021-05-24T08:46:01Z

Pinging @elastic/security-external-integrations (Team:Security-External Integrations)

mergify · 2021-05-25T18:22:12Z

This pull request is now in conflicts. Could you fix it? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b 25689-okta-fields upstream/25689-okta-fields
git merge upstream/master
git push upstream 25689-okta-fields

mergify · 2021-05-26T10:04:52Z

This pull request is now in conflicts. Could you fix it? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b 25689-okta-fields upstream/25689-okta-fields
git merge upstream/master
git push upstream 25689-okta-fields

P1llus · 2021-05-27T14:52:25Z

run tests

mergify · 2021-06-02T10:16:29Z

This pull request is now in conflicts. Could you fix it? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b 25689-okta-fields upstream/25689-okta-fields
git merge upstream/master
git push upstream 25689-okta-fields

mergify · 2021-06-07T14:14:43Z

This pull request is now in conflicts. Could you fix it? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b 25689-okta-fields upstream/25689-okta-fields
git merge upstream/master
git push upstream 25689-okta-fields

mergify · 2021-06-08T11:34:20Z

This pull request is now in conflicts. Could you fix it? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b 25689-okta-fields upstream/25689-okta-fields
git merge upstream/master
git push upstream 25689-okta-fields

P1llus · 2021-06-15T14:44:30Z

/test

P1llus

LGTM, just one comment first

x-pack/filebeat/module/okta/system/test/okta-system-test.json.log-expected.json

mergify · 2021-06-16T16:01:28Z

This pull request is now in conflicts. Could you fix it? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b 25689-okta-fields upstream/25689-okta-fields
git merge upstream/master
git push upstream 25689-okta-fields

mergify · 2021-06-24T13:24:02Z

This pull request is now in conflicts. Could you fix it? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b 25689-okta-fields upstream/25689-okta-fields
git merge upstream/master
git push upstream 25689-okta-fields

P1llus · 2021-06-24T14:10:47Z

run tests

* #25689: Parse additonal debug data fields for Okta module * update generated data * update changelog * added additional test data & `uri_parts` processor * update fields * fix changelog * update fields Co-authored-by: Marius Iversen <marius.iversen@elastic.co> (cherry picked from commit 4aff295)

…#26487) * #25689: Parse additonal debug data fields for Okta module * update generated data * update changelog * added additional test data & `uri_parts` processor * update fields * fix changelog * update fields Co-authored-by: Marius Iversen <marius.iversen@elastic.co> (cherry picked from commit 4aff295) Co-authored-by: Alex Resnick <adr8292@gmail.com>

* master: (32 commits) [Metricbeat] Change Account ID to Project ID in `gcp.billing` module (elastic#26412) update libbeat fields.ecs.yml file and ecsVersion to 1.10.0 (elastic#26121) [Filebeat] Update AWS ELB ingest pipeline (elastic#26441) [FIlebeat] add strict_date_optional_time_nanos date format to PanOS module (elastic#26158) Fix the irregular and typo on prometheus module. (elastic#25726) [Filebeat] Parse additonal debug data fields for Okta module (elastic#25818) fix: update MSSQL Server linux image's Docker registry (elastic#26440) Update indexing.go godocs (elastic#26408) Do not close filestream harvester if an unexpected error is returned when close.on_state_change.* is enabled (elastic#26411) Add support for copytruncate method when rotating input logs with an external tool in `filestream` input (elastic#23457) Allow fields with ip_range datatype (elastic#26444) Add Anomali ThreatStream support to threatintel module (elastic#26350) fix: use the right param type (elastic#26469) [Automation] Update elastic stack version to 8.0.0-7640093f for testing (elastic#26460) Set SM Filebeat modules as GA (elastic#26226) Fix rfc5464 date parsing in the syslog input (elastic#26419) Add linked account information into billing metricset (elastic#26285) [Filebeat] Update HA Proxy log grok patterns (elastic#25835) disable metricbeat logstash test_node_stats (elastic#26436) chore: pass BEAT_VERSION when running E2E tests (elastic#26291) ...

botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label May 21, 2021

legoguy1000 marked this pull request as ready for review May 23, 2021 22:21

jamiehynds added the Team:Security-External Integrations label May 24, 2021

botelastic bot removed the needs_team Indicates that the issue/PR needs a Team:* label label May 24, 2021

jamiehynds added the enhancement label May 24, 2021

legoguy1000 force-pushed the 25689-okta-fields branch from 0b88d1c to c19fc5f Compare May 25, 2021 20:14

legoguy1000 force-pushed the 25689-okta-fields branch from c19fc5f to 871f57a Compare May 27, 2021 01:36

legoguy1000 force-pushed the 25689-okta-fields branch from 728fb2f to 0c6bb8b Compare June 2, 2021 12:05

legoguy1000 force-pushed the 25689-okta-fields branch 3 times, most recently from 4ff6fda to 4d4e03f Compare June 7, 2021 16:28

legoguy1000 force-pushed the 25689-okta-fields branch from 4d4e03f to e2181d6 Compare June 15, 2021 14:32

P1llus added the needs_integration_sync Changes in this PR need synced to elastic/integrations. label Jun 15, 2021

legoguy1000 force-pushed the 25689-okta-fields branch from e2181d6 to 6bcd930 Compare June 15, 2021 14:40

P1llus reviewed Jun 16, 2021

View reviewed changes

x-pack/filebeat/module/okta/system/test/okta-system-test.json.log-expected.json Outdated Show resolved Hide resolved

legoguy1000 added 5 commits June 24, 2021 11:59

elastic#25689: Parse additonal debug data fields for Okta module

8ff7f1e

update generated data

41e3107

update changelog

23dde37

added additional test data & uri_parts processor

029db21

update fields

be4d965

fix changelog

cf6d7a9

legoguy1000 force-pushed the 25689-okta-fields branch from 6bcd930 to cf6d7a9 Compare June 24, 2021 11:59

update fields

7360f9e

legoguy1000 force-pushed the 25689-okta-fields branch from e4f2179 to 7360f9e Compare June 24, 2021 12:35

P1llus approved these changes Jun 24, 2021

View reviewed changes

Merge branch 'master' into 25689-okta-fields

d69a251

P1llus merged commit 4aff295 into elastic:master Jun 24, 2021

P1llus added the backport-v7.14.0 Automated backport with mergify label Jun 24, 2021

mergify bot mentioned this pull request Jun 24, 2021

[7.x](backport #25818) [Filebeat] Parse additonal debug data fields for Okta module #26487

Merged

Conversation

legoguy1000 commented May 21, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What does this PR do?

Why is it important?

Checklist

Author's Checklist

How to test this PR locally

Related issues

Use cases

Screenshots

Logs

Uh oh!

elasticmachine commented May 21, 2021 • edited by jenkins-beats-ci bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

💚 Build Succeeded

Build stats

Test stats 🧪

Trends 🧪

💚 Flaky test report

Test stats 🧪

Uh oh!

elasticmachine commented May 24, 2021

Uh oh!

mergify bot commented May 25, 2021

Uh oh!

mergify bot commented May 26, 2021

Uh oh!

P1llus commented May 27, 2021

Uh oh!

mergify bot commented Jun 2, 2021

Uh oh!

mergify bot commented Jun 7, 2021

Uh oh!

mergify bot commented Jun 8, 2021

Uh oh!

P1llus commented Jun 15, 2021

Uh oh!

P1llus left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

mergify bot commented Jun 16, 2021

Uh oh!

mergify bot commented Jun 24, 2021

Uh oh!

P1llus commented Jun 24, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

legoguy1000 commented May 21, 2021 •

edited

Loading

elasticmachine commented May 21, 2021 •

edited by jenkins-beats-ci bot

Loading