Skip to content

[O11y][AWS] Rally benchmark aws.waf#9377

Merged
shmsr merged 6 commits intoelastic:mainfrom
ali786XI:aws_benchmark_waf
Apr 3, 2024
Merged

[O11y][AWS] Rally benchmark aws.waf#9377
shmsr merged 6 commits intoelastic:mainfrom
ali786XI:aws_benchmark_waf

Conversation

@ali786XI
Copy link
Copy Markdown
Contributor

@ali786XI ali786XI commented Mar 18, 2024

Proposed commit message

  • This PR adds benchmarking templates to the waf data stream of AWS

Sample Response

sample_event.json
{
"agent": {
  "name": "aws-scale-123456",
  "id": "de42127b-4db8-4471-824e-a7b14f478663",
  "ephemeral_id": "22ed892c-43bd-408a-9121-65e2f5b6a56e",
  "type": "filebeat",
  "version": "8.8.0"
},
"benchmark_metadata": {
  "info": {
    "run_id": "afde7278-c398-48a3-8ef8-6ee5d912ba0f",
    "benchmark": "waf-benchmark"
  }
},
"log": {
  "file": {
    "path": "https://elastic-package-aws-bucket-63468.s3.us-east-1.amazonaws.com/waf.log"
  },
  "offset": 0
},
"elastic_agent": {
  "id": "de42127b-4db8-4471-824e-a7b14f478663",
  "version": "8.8.0",
  "snapshot": false
},
"rule": {
  "ruleset": "GROUP",
  "id": "STMTest_SQLi_XSS"
},
"source": {
  "geo": {
    "continent_name": "Europe",
    "region_iso_code": "SE-E",
    "city_name": "Linköping",
    "country_iso_code": "SE",
    "country_name": "Sweden",
    "location": {
      "lon": 15.6167,
      "lat": 58.4167
    },
    "region_name": "Östergötland County"
  },
  "as": {
    "number": 29518,
    "organization": {
      "name": "Bredband2 AB"
    }
  },
  "ip": "54.146.12.204"
},
"url": {
  "path": "/wp-content/cougar wolverine hide healer friend muse singer taker bow throat boot gambler snap stalker loon parrot scorpion stag moth salmon loon aquamarinetouch.png",
  "query": "ver=1.1"
},
"network": {
  "protocol": "http",
  "transport": "tcp"
},
"tags": [
  "preserve_original_event",
  "forwarded",
  "aws-waf"
],
"cloud": {
  "provider": "aws",
  "service": {
    "name": "wafv2"
  },
  "region": "eu-north-1",
  "account": {
    "id": "redpiper"
  }
},
"input": {
  "type": "aws-s3"
},
"@timestamp": "2024-03-27T04:40:15.799Z",
"related": {
  "ip": [
    "89.160.20.156",
    "54.146.12.204"
  ]
},
"ecs": {
  "version": "8.0.0"
},
"data_stream": {
  "namespace": "ep",
  "type": "logs",
  "dataset": "aws.waf"
},
"http": {
  "request": {
    "method": "POST",
    "id": "rid-9160"
  },
  "version": "1.1"
},
"aws": {
  "s3": {
    "bucket": {
      "name": "elastic-package-aws-bucket-63468",
      "arn": "arn:aws:s3:::elastic-package-aws-bucket-63468"
    },
    "object": {
      "key": "helixnape.log"
    }
  },
  "waf": {
    "request": {
      "headers": {
        "User-Agent": "Apache-HttpClient/4.5.6 (Java/1.8.0_191)",
        "Host": "wordpress-1573828666.eu-north-1.elb.amazonaws.com"
      }
    },
    "terminating_rule_match_details": [
      {
        "matchedData": [
          "82",
          "AND",
          "82"
        ],
        "conditionType": "SQL_INJECTION",
        "location": "lemonwatcher"
      }
    ],
    "id": "regional/webacl/STMTest/1EXAMPLE-2ARN-3ARN-4ARN-123456EXAMPLE",
    "source": {
      "name": "ALB",
      "id": "elb"
    },
    "arn": "arn:aws:wafv2:eu-north-1:redpiper:regional/webacl/STMTest/1EXAMPLE-2ARN-3ARN-4ARN-123456EXAMPLE"
  }
},
"event": {
  "agent_id_status": "auth_metadata_missing",
  "ingested": "2024-03-27T04:40:19Z",
  "original": "{\"timestamp\":1711514415799,\"formatVersion\":1,\"webaclId\":\"arn:aws:wafv2:eu-north-1:redpiper:regional/webacl/STMTest/1EXAMPLE-2ARN-3ARN-4ARN-123456EXAMPLE\",\"terminatingRuleId\":\"STMTest_SQLi_XSS\",\"terminatingRuleType\":\"GROUP\",\"action\":\"BLOCK\",\"terminatingRuleMatchDetails\": [{\"conditionType\": \"SQL_INJECTION\",\"location\": \"lemonwatcher\",\"matchedData\": [\"82\",\"AND\",\"82\"]}],\"httpSourceName\":\"ALB\",\"httpSourceId\":\"elb\",\"ruleGroupList\":[],\"rateBasedRuleList\":[],\"nonTerminatingMatchingRules\":[],\"requestHeadersInserted\":null,\"responseCodeSent\":null,\"httpRequest\":{\"clientIp\":\"54.146.12.204\",\"country\":\"US\",\"headers\":[{\"name\":\"Host\",\"value\":\"wordpress-1573828666.eu-north-1.elb.amazonaws.com\"},{\"name\":\"User-Agent\",\"value\":\"Apache-HttpClient/4.5.6 (Java/1.8.0_191)\"}],\"uri\":\"/wp-content/cougar wolverine hide healer friend muse singer taker bow throat boot gambler snap stalker loon parrot scorpion stag moth salmon loon aquamarinetouch.png\",\"args\":\"ver=1.1\",\"httpVersion\":\"HTTP/1.1\",\"httpMethod\":\"POST\",\"requestId\":\"rid-9160\"},\"labels\":[{\"name\":\"value\"}]}",
  "kind": "event",
  "action": "BLOCK",
  "category": "web",
  "type": [
    "access",
    "denied"
  ],
  "dataset": "aws.waf"
}
}

Checklist

  • I have reviewed tips for building integrations and this pull request is aligned with them.
  • I have verified that all data streams collect metrics or logs.

How to test this PR locally

Run this command from package root

  • elastic-package benchmark rally --benchmark waf-benchmark -v
  • elastic-package benchmark stream --benchmark waf-benchmark -v

Related issues

Screenshots

--- Benchmark results for package: aws - START ---
╭────────────────────────────────────────────────────────────────────────────────────╮
│ info                                                                               │
├────────────────────────┬───────────────────────────────────────────────────────────┤
│ benchmark              │                                             waf-benchmark │
│ description            │                Benchmark of 20000 aws.waf events ingested │
│ run ID                 │                      b7bd70a3-9dd4-4f34-bc5f-2b66832999fa │
│ package                │                                                       aws │
│ start ts (s)           │                                                1710741326 │
│ end ts (s)             │                                                1710741366 │
│ duration               │                                                       40s │
│ generated corpora file │ /root/.elastic-package/tmp/rally_corpus/corpus-2262656855 │
╰────────────────────────┴───────────────────────────────────────────────────────────╯
╭───────────────────────────────────────────────────────────────────╮
│ parameters                                                        │
├─────────────────────────────────┬─────────────────────────────────┤
│ package version                 │                          2.13.1 │
│ data_stream.name                │                             waf │
│ corpora.generator.total_events  │                           20000 │
│ corpora.generator.template.path │ ./waf-benchmark/template.ndjson │
│ corpora.generator.template.raw  │                                 │
│ corpora.generator.template.type │                          gotext │
│ corpora.generator.config.path   │      ./waf-benchmark/config.yml │
│ corpora.generator.config.raw    │                           map[] │
│ corpora.generator.fields.path   │      ./waf-benchmark/fields.yml │
│ corpora.generator.fields.raw    │                           map[] │
╰─────────────────────────────────┴─────────────────────────────────╯
╭───────────────────────╮
│ cluster info          │
├───────┬───────────────┤
│ name  │ elasticsearch │
│ nodes │             1 │
╰───────┴───────────────╯
╭───────────────────────────────────────╮
│ disk usage for index .ds-logs-aws.waf │
│ -ep-2024.03.18-000001 (for all fields │
│ )                                     │
├──────────────────────────────┬────────┤
│ total                        │ 300 kB │
│ inverted_index.total         │ 179 kB │
│ inverted_index.stored_fields │  62 kB │
│ inverted_index.doc_values    │  38 kB │
│ inverted_index.points        │  21 kB │
│ inverted_index.norms         │    0 B │
│ inverted_index.term_vectors  │    0 B │
│ inverted_index.knn_vectors   │    0 B │
╰──────────────────────────────┴────────╯
╭───────────────────────────────────────╮
│ disk usage for index .ds-logs-aws.waf │
│ -ep-2024.03.18-000002 (for all fields │
│ )                                     │
├──────────────────────────────┬────────┤
│ total                        │  20 MB │
│ inverted_index.total         │ 6.3 MB │
│ inverted_index.stored_fields │ 4.9 MB │
│ inverted_index.doc_values    │ 8.6 MB │
│ inverted_index.points        │ 352 kB │
│ inverted_index.norms         │    0 B │
│ inverted_index.term_vectors  │    0 B │
│ inverted_index.knn_vectors   │    0 B │
╰──────────────────────────────┴────────╯
╭──────────────────────────────────────────────────────────────────────────╮
│ pipeline logs-aws.waf-2.13.1 stats in node DVmFn0ahSo-ARtJ-SsUwqA        │
├────────────────────────────────┬─────────────────────────────────────────┤
│ Totals                         │ Count: 20000 | Failed: 0 | Time: 2.033s │
│ set ()                         │   Count: 20000 | Failed: 0 | Time: 13ms │
│ set ()                         │    Count: 20000 | Failed: 0 | Time: 9ms │
│ append ()                      │   Count: 20000 | Failed: 0 | Time: 11ms │
│ rename ()                      │   Count: 20000 | Failed: 0 | Time: 29ms │
│ remove ()                      │    Count: 20000 | Failed: 0 | Time: 3ms │
│ json ()                        │  Count: 20000 | Failed: 0 | Time: 296ms │
│ date ()                        │   Count: 20000 | Failed: 0 | Time: 42ms │
│ rename ()                      │   Count: 20000 | Failed: 0 | Time: 42ms │
│ geoip ()                       │   Count: 20000 | Failed: 0 | Time: 47ms │
│ rename ()                      │   Count: 20000 | Failed: 0 | Time: 44ms │
│ geoip ()                       │   Count: 20000 | Failed: 0 | Time: 62ms │
│ rename ()                      │   Count: 20000 | Failed: 0 | Time: 13ms │
│ rename ()                      │    Count: 18163 | Failed: 0 | Time: 6ms │
│ rename ()                      │   Count: 20000 | Failed: 0 | Time: 12ms │
│ rename ()                      │   Count: 20000 | Failed: 0 | Time: 48ms │
│ rename ()                      │   Count: 20000 | Failed: 0 | Time: 67ms │
│ dissect ()                     │   Count: 20000 | Failed: 0 | Time: 49ms │
│ lowercase ()                   │   Count: 20000 | Failed: 0 | Time: 19ms │
│ set ()                         │    Count: 20000 | Failed: 0 | Time: 8ms │
│ rename ()                      │   Count: 20000 | Failed: 0 | Time: 37ms │
│ rename ()                      │   Count: 20000 | Failed: 0 | Time: 36ms │
│ rename ()                      │   Count: 20000 | Failed: 0 | Time: 24ms │
│ rename ()                      │   Count: 20000 | Failed: 0 | Time: 39ms │
│ rename ()                      │   Count: 20000 | Failed: 0 | Time: 36ms │
│ rename ()                      │   Count: 20000 | Failed: 0 | Time: 37ms │
│ script ()                      │   Count: 20000 | Failed: 0 | Time: 29ms │
│ rename ()                      │   Count: 20000 | Failed: 0 | Time: 32ms │
│ append ()                      │   Count: 20000 | Failed: 0 | Time: 67ms │
│ set ()                         │    Count: 20000 | Failed: 0 | Time: 8ms │
│ set ()                         │    Count: 20000 | Failed: 0 | Time: 7ms │
│ append ()                      │     Count: 9997 | Failed: 0 | Time: 5ms │
│ append ()                      │    Count: 10003 | Failed: 0 | Time: 5ms │
│ rename ()                      │   Count: 20000 | Failed: 0 | Time: 34ms │
│ dissect ()                     │   Count: 20000 | Failed: 0 | Time: 73ms │
│ rename ()                      │   Count: 20000 | Failed: 0 | Time: 33ms │
│ rename ()                      │   Count: 20000 | Failed: 0 | Time: 33ms │
│ rename ()                      │   Count: 20000 | Failed: 0 | Time: 50ms │
│ rename ()                      │   Count: 20000 | Failed: 0 | Time: 39ms │
│ remove ()                      │    Count: 20000 | Failed: 0 | Time: 9ms │
│ script ()                      │  Count: 20000 | Failed: 0 | Time: 373ms │
│ remove ()                      │   Count: 20000 | Failed: 0 | Time: 16ms │
│ pipeline (global@custom)       │    Count: 20000 | Failed: 0 | Time: 3ms │
│ pipeline (logs@custom)         │    Count: 20000 | Failed: 0 | Time: 1ms │
│ pipeline (logs-aws@custom)     │    Count: 20000 | Failed: 0 | Time: 2ms │
│ pipeline (logs-aws.waf@custom) │    Count: 20000 | Failed: 0 | Time: 2ms │
╰────────────────────────────────┴─────────────────────────────────────────╯
╭─────────────────────────────────────────────────────────────────────────────────────────────╮
│ rally stats                                                                                 │
├────────────────────────────────────────────────────────────────┬────────────────────────────┤
│ Cumulative indexing time of primary shards                     │     0.2733333333333333 min │
│ Min cumulative indexing time across primary shards             │                      0 min │
│ Median cumulative indexing time across primary shards          │                 0.0007 min │
│ Max cumulative indexing time across primary shards             │    0.05446666666666666 min │
│ Cumulative indexing throttle time of primary shards            │                      0 min │
│ Min cumulative indexing throttle time across primary shards    │                      0 min │
│ Median cumulative indexing throttle time across primary shards │                      0 min │
│ Max cumulative indexing throttle time across primary shards    │                      0 min │
│ Cumulative merge time of primary shards                        │                 0.0302 min │
│ Cumulative merge count of primary shards                       │                         54 │
│ Min cumulative merge time across primary shards                │                      0 min │
│ Median cumulative merge time across primary shards             │                      0 min │
│ Max cumulative merge time across primary shards                │   0.009866666666666666 min │
│ Cumulative merge throttle time of primary shards               │                      0 min │
│ Min cumulative merge throttle time across primary shards       │                      0 min │
│ Median cumulative merge throttle time across primary shards    │                      0 min │
│ Max cumulative merge throttle time across primary shards       │                      0 min │
│ Cumulative refresh time of primary shards                      │                0.26565 min │
│ Cumulative refresh count of primary shards                     │                       2783 │
│ Min cumulative refresh time across primary shards              │                      0 min │
│ Median cumulative refresh time across primary shards           │ 0.00018333333333333334 min │
│ Max cumulative refresh time across primary shards              │    0.09296666666666667 min │
│ Cumulative flush time of primary shards                        │                 0.5592 min │
│ Cumulative flush count of primary shards                       │                       1743 │
│ Min cumulative flush time across primary shards                │                      0 min │
│ Median cumulative flush time across primary shards             │ 1.6666666666666667e-05 min │
│ Max cumulative flush time across primary shards                │    0.13483333333333333 min │
│ Total Young Gen GC time                                        │                    0.054 s │
│ Total Young Gen GC count                                       │                          4 │
│ Total Old Gen GC time                                          │                        0 s │
│ Total Old Gen GC count                                         │                          0 │
│ Store size                                                     │    0.028823914006352425 GB │
│ Translog size                                                  │    0.037455531768500805 GB │
│ Heap used for segments                                         │                       0 MB │
│ Heap used for doc values                                       │                       0 MB │
│ Heap used for terms                                            │                       0 MB │
│ Heap used for norms                                            │                       0 MB │
│ Heap used for points                                           │                       0 MB │
│ Heap used for stored fields                                    │                       0 MB │
│ Segment count                                                  │                        111 │
│ Total Ingest Pipeline count                                    │                      20006 │
│ Total Ingest Pipeline time                                     │                    3.188 s │
│ Total Ingest Pipeline failed                                   │                          0 │
│ Min Throughput                                                 │             1741.38 docs/s │
│ Mean Throughput                                                │             2441.96 docs/s │
│ Median Throughput                                              │             2613.74 docs/s │
│ Max Throughput                                                 │             2662.18 docs/s │
│ 50th percentile latency                                        │       168.1794969990733 ms │
│ 90th percentile latency                                        │      200.78166750172386 ms │
│ 100th percentile latency                                       │       333.3804389985744 ms │
│ 50th percentile service time                                   │       168.1794969990733 ms │
│ 90th percentile service time                                   │      200.78166750172386 ms │
│ 100th percentile service time                                  │       333.3804389985744 ms │
│ error rate                                                     │                     0.00 % │
╰────────────────────────────────────────────────────────────────┴────────────────────────────╯

--- Benchmark results for package: aws - END   ---
Done

@ali786XI ali786XI added the enhancement New feature or request label Mar 18, 2024
@ali786XI ali786XI self-assigned this Mar 18, 2024
@elasticmachine
Copy link
Copy Markdown

🚀 Benchmarks report

To see the full report comment with /test benchmark fullreport

@ali786XI ali786XI marked this pull request as ready for review March 22, 2024 06:36
@ali786XI ali786XI requested review from a team as code owners March 22, 2024 06:36
@ali786XI ali786XI requested a review from milan-elastic March 26, 2024 04:34
@milan-elastic
Copy link
Copy Markdown
Contributor

@aliabbas-elastic Can you update the sample response in the description? It's not loading.

@ali786XI
Copy link
Copy Markdown
Contributor Author

@aliabbas-elastic Can you update the sample response in the description? It's not loading.

Done

- name: aws_waf_source_name
value: ["CF", "APIGW", "ALB", "APPSYNC", "COGNITOIDP", "APPRUNNER", "VERIFIED_ACCESS"]
- name: condition_type
value: "SQL_INJECTION"
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

https://docs.aws.amazon.com/waf/latest/developerguide/classic-web-acl-create-condition.html

From what I see there are more type of conditions: SQL_INJECTION, XSS_PATH, SIZE_RESTRICTION, GEO_MATCH, IP_MATCH, REGEX_MATCH

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated it with two more values. I couldn't find the exact strings for conditions SIZE_RESTRICTION, GEO_MATCH & IP_MATCH. Let me know if this would suffice

type: keyword
- name: uri_path
type: keyowrd
example: themes/twentynineteen/style
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are you sure the example is correct?

Copy link
Copy Markdown
Contributor Author

@ali786XI ali786XI Apr 3, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for catching this. Actually this is a syntax rather than an exact value so this would give me three random generated words separated by /. But currently only these ones are supported. After removing the example config line here are the values generated under the uri field. I think right now we can proceed with this until there is requirement of supporting more characters

  • /wp-content/dirtqueen.min.css
  • /wp-content/bitterthroat.css
  • /wp-content/rubyfang.png

ali786XI and others added 2 commits April 3, 2024 12:46
@ali786XI ali786XI requested a review from shmsr April 3, 2024 08:05
@elasticmachine
Copy link
Copy Markdown

💚 Build Succeeded

History

cc @aliabbas-elastic

@elastic-sonarqube
Copy link
Copy Markdown

Quality Gate passed Quality Gate passed

Kudos, no new issues were introduced!

0 New issues
0 Security Hotspots
No Coverage information No data about Coverage
No Duplication information No data about Duplication

See analysis details on SonarQube

@shmsr shmsr merged commit e72e550 into elastic:main Apr 3, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request Integration:aws AWS

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants