[Azure] Application Gateway WAF: add event.reason by jH- · Pull Request #10007 · elastic/integrations

jH- · 2024-05-29T13:52:00Z

Proposed commit message

Update the Azure Application Gateway pipeline to parse and include values from
`json.properties.details.data` into the ECS `event.reason` field.

This field provides action context by logging the specific data found in 
requests that matched a rule.

It excludes the field if `json.properties.details.message` contains values indicating 
missing headers or content were the reason to avoid unnecessary duplicate data.

Checklist

I have reviewed tips for building integrations and this pull request is aligned with them.
I have added an entry to my package's changelog.yml file.

How to test this PR locally

elastic-package stack up -v -d

# check that the updated integration version is included (https://localhost:8080/search?package=azure)

elastic-package test pipeline

elastic-package stack down

(original)properties.details.data as rule.matched_data

cla-checker-service · 2024-05-29T13:52:04Z

💚 CLA has been signed

botelastic · 2024-07-03T08:03:04Z

Hi! We just realized that we haven't looked into this PR in a while. We're sorry! We're labeling this issue as Stale to make it hit our filters and make sure we get back to it as soon as possible. In the meantime, it'd be extremely helpful if you could take a look at it as well and confirm its relevance. A simple comment with a nice emoji will be enough :+1. Thank you for your contribution!

elasticmachine · 2024-07-03T08:21:19Z

Pinging @elastic/security-service-integrations (Team:Security-Service Integrations)

kcreddy · 2024-07-05T07:32:57Z

/test

elasticmachine · 2024-07-05T07:46:47Z

🚀 Benchmarks report

Package `azure` 👍(2) 💚(0) 💔(9)

Expand to view

Data stream	Previous EPS	New EPS	Diff (%)	Result
`springcloudlogs`	4098.36	3115.26	-983.1 (-23.99%)	💔
`application_gateway`	2967.36	1733.1	-1234.26 (-41.59%)	💔
`auditlogs`	1945.53	1526.72	-418.81 (-21.53%)	💔
`eventhub`	333333.33	250000	-83333.33 (-25%)	💔
`firewall_logs`	1420.45	1113.59	-306.86 (-21.6%)	💔
`graphactivitylogs`	1834.86	1451.38	-383.48 (-20.9%)	💔
`identity_protection`	4608.29	2724.8	-1883.49 (-40.87%)	💔
`platformlogs`	4950.5	2849	-2101.5 (-42.45%)	💔
`provisioning`	2898.55	2288.33	-610.22 (-21.05%)	💔

To see the full report comment with /test benchmark fullreport

elastic-sonarqube · 2024-07-05T07:47:31Z

Quality Gate passed

Issues
0 New issues
0 Fixed issues
0 Accepted issues

Measures
0 Security Hotspots
38.5% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube

packages/azure/changelog.yml

packages/azure/manifest.yml

...stream/application_gateway/_dev/test/pipeline/test-application-gateway-raw.log-expected.json

botelastic · 2024-08-04T09:12:33Z

Hi! We just realized that we haven't looked into this PR in a while. We're sorry! We're labeling this issue as Stale to make it hit our filters and make sure we get back to it as soon as possible. In the meantime, it'd be extremely helpful if you could take a look at it as well and confirm its relevance. A simple comment with a nice emoji will be enough :+1. Thank you for your contribution!

kcreddy · 2024-08-22T09:54:58Z

/test

kcreddy · 2024-08-22T10:43:17Z

@jH- could you please fix the merge conflicts?

kcreddy · 2024-08-23T12:07:03Z

/test

andrewkroh

There are a bunch of changes outside of the packages/azure directory that need to undone. I think they were the result of merge issue.

elastic-sonarqube · 2024-08-23T14:00:20Z

Quality Gate passed

Issues
0 New issues
0 Fixed issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube

jH-

deleted

[git-generate] elastic-package -C packages/azure test pipeline -g -d application_gateway

Merge issues were corrected. Changeset looks fine now.

andrewkroh · 2024-09-25T17:48:30Z

/test

elasticmachine · 2024-09-25T18:09:10Z

💚 Build Succeeded

Buildkite Build
Commit: d5c9a82

History

💔 Build #14965 failed a2b7ebb63fdacdaf94d0fe417b375226cf7a5edd
💔 Build #14893 failed af7f7f8
💚 Build #13283 succeeded 2037f9a

elastic-sonarqube · 2024-09-25T18:09:12Z

Quality Gate passed

Issues
0 New issues
0 Fixed issues
0 Accepted issues

Measures
0 Security Hotspots
83.3% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube

elastic-vault-github-plugin-prod · 2024-09-25T18:25:55Z

Package azure - 1.17.0 containing this change is available at https://epr.elastic.co/search?package=azure

Update the Azure Application Gateway pipeline to parse and include values from `json.properties.details.data` into the ECS `event.reason` field. This field provides action context by logging the specific data found in requests that matched a rule. It excludes the field if `json.properties.details.message` contains values indicating missing headers or content were the reason to avoid unnecessary duplicate data. --------- Co-authored-by: Johan H <johan@horde.no> Co-authored-by: Maurizio Branca <maurizio.branca@elastic.co> Co-authored-by: Krishna Chaitanya Reddy Burri <krish.reddy91@gmail.com> Co-authored-by: Andrew Kroh <andrew.kroh@elastic.co>

johhau added 6 commits May 28, 2024 14:09

Add field: details.data, firewall logs

cbccc91

(original)properties.details.data as rule.matched_data

Add rule.matched_data processing checks

b0c3043

Add new field to correct file, not ecs

15b7361

Use common (ecs) field instead of custom

d9f0ebd

Update expected output from pipeline test

57b5d96

Bump package version, changelog description

2037f9a

jH- requested review from a team as code owners May 29, 2024 13:52

This comment was marked as resolved.

Sign in to view

botelastic bot added the Stalled label Jul 3, 2024

jamiehynds added the Team:Security-Service Integrations Security Service Integrations team [elastic/security-service-integrations] label Jul 3, 2024

botelastic bot removed the Stalled label Jul 3, 2024

jamiehynds added enhancement New feature or request Stalled labels Jul 3, 2024

botelastic bot removed the Stalled label Jul 3, 2024

jamiehynds added Integration:aws AWS Stalled labels Jul 3, 2024

botelastic bot removed the Stalled label Jul 3, 2024

zmoog reviewed Jul 5, 2024

View reviewed changes

packages/azure/changelog.yml Outdated Show resolved Hide resolved

kcreddy reviewed Jul 5, 2024

View reviewed changes

packages/azure/changelog.yml Outdated Show resolved Hide resolved

packages/azure/manifest.yml Outdated Show resolved Hide resolved

...stream/application_gateway/_dev/test/pipeline/test-application-gateway-raw.log-expected.json Outdated Show resolved Hide resolved

kcreddy added Integration:azure Azure Logs and removed Integration:aws AWS labels Jul 5, 2024

andrewkroh removed the needs CLA User must sign the Elastic Contributor License before review. label Aug 15, 2024

andrewkroh approved these changes Aug 15, 2024

View reviewed changes

andrewkroh added Team:Obs-InfraObs Observability Infrastructure Monitoring team [elastic/obs-infraobs-integrations] needs CLA User must sign the Elastic Contributor License before review. and removed needs CLA User must sign the Elastic Contributor License before review. labels Aug 15, 2024

zmoog approved these changes Aug 23, 2024

View reviewed changes

kcreddy approved these changes Aug 23, 2024

View reviewed changes

jH- requested a review from a team as a code owner August 23, 2024 11:51

andrewkroh previously requested changes Aug 23, 2024

View reviewed changes

jH- commented Sep 9, 2024

View reviewed changes

jH- force-pushed the add_details_data branch from a2b7ebb to af7f7f8 Compare September 9, 2024 13:45

andrewkroh added 2 commits September 25, 2024 13:43

Merge remote-tracking branch 'elastic/main' into add_details_data

905509a

azure: rebuild expected application_gateway logs

d5c9a82

[git-generate] elastic-package -C packages/azure test pipeline -g -d application_gateway

andrewkroh enabled auto-merge (squash) September 25, 2024 17:53

andrewkroh disabled auto-merge September 25, 2024 17:54

andrewkroh enabled auto-merge (squash) September 25, 2024 17:54

andrewkroh disabled auto-merge September 25, 2024 17:55

andrewkroh enabled auto-merge (squash) September 25, 2024 17:56

andrewkroh merged commit 9d46e70 into elastic:main Sep 25, 2024

Conversation

jH- commented May 29, 2024 • edited by andrewkroh Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Proposed commit message

Checklist

How to test this PR locally

Uh oh!

cla-checker-service bot commented May 29, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

This comment was marked as resolved.

botelastic bot commented Jul 3, 2024

Uh oh!

elasticmachine commented Jul 3, 2024

Uh oh!

kcreddy commented Jul 5, 2024

Uh oh!

elasticmachine commented Jul 5, 2024 • edited by elastic-vault-github-plugin-prod bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🚀 Benchmarks report

Package azure 👍(2) 💚(0) 💔(9)

Uh oh!

elastic-sonarqube bot commented Jul 5, 2024

Quality Gate passed

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

botelastic bot commented Aug 4, 2024

Uh oh!

kcreddy commented Aug 22, 2024

Uh oh!

kcreddy commented Aug 22, 2024

Uh oh!

kcreddy commented Aug 23, 2024

Uh oh!

andrewkroh left a comment

Choose a reason for hiding this comment

Uh oh!

elastic-sonarqube bot commented Aug 23, 2024

Quality Gate passed

Uh oh!

jH- left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

andrewkroh commented Sep 25, 2024

Uh oh!

elasticmachine commented Sep 25, 2024

💚 Build Succeeded

History

Uh oh!

elastic-sonarqube bot commented Sep 25, 2024

Quality Gate passed

Uh oh!

elastic-vault-github-plugin-prod bot commented Sep 25, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

jH- commented May 29, 2024 •

edited by andrewkroh

Loading

cla-checker-service bot commented May 29, 2024 •

edited

Loading

elasticmachine commented Jul 5, 2024 •

edited by elastic-vault-github-plugin-prod bot

Loading

Package `azure` 👍(2) 💚(0) 💔(9)

jH- left a comment •

edited

Loading