Skip to content

[MetricBeat] set tags correctly if the dimension value is ARN#19433

Merged
kaiyan-sheng merged 3 commits intoelastic:masterfrom
kwinstonix:fix-ddos-metrics-tags
Jul 2, 2020
Merged

[MetricBeat] set tags correctly if the dimension value is ARN#19433
kaiyan-sheng merged 3 commits intoelastic:masterfrom
kwinstonix:fix-ddos-metrics-tags

Conversation

@kwinstonix
Copy link
Copy Markdown
Contributor

@kwinstonix kwinstonix commented Jun 26, 2020 •
edited by andresrc
Loading

What does this PR do?

This PR is to fix setting resource tags when dimension value is ARN format, I found this issue in AWS/DDoSProtection namespace metrics.

Why is it important?

cloudwatch module should set event tags correctly if dimension value is ARN format and tags.resource_type_filter param is configured

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have made corresponding change to the default configuration files
  • I have added tests that prove my fix is effective or that my feature works
  • I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Related issues

Screenshots

test with aws shield service and EIP resource , I have put some demo metrics before running test ,so the namespace is DDoSProtection. The actual value is AWS/DDoSProtection if you get metrics that created by aws.

metricbeat.modules:
  - module: aws
    period: 1h
    access_key_id: 
    secret_access_key: 
    metrics:
      - namespace: DDoSProtection
        statistic: ['Maximum']
        name:
          [
            'DDoSDetected',
            'DDoSAttackBitsPerSecond',
            'DDoSAttackPacketsPerSecond',
            'DDoSAttackRequestsPerSecond',
          ]
        tags.resource_type_filter: ec2:elastic-ip

image

@elasticmachine
Copy link
Copy Markdown
Contributor

elasticmachine commented Jun 26, 2020

Since this is a community submitted pull request, a Jenkins build has not been kicked off automatically. Can an Elastic organization member please verify the contents of this patch and then kick off a build manually?

1 similar comment
@elasticmachine
Copy link
Copy Markdown
Contributor

elasticmachine commented Jun 26, 2020

Since this is a community submitted pull request, a Jenkins build has not been kicked off automatically. Can an Elastic organization member please verify the contents of this patch and then kick off a build manually?

@botelastic botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Jun 26, 2020
@elasticmachine
Copy link
Copy Markdown
Contributor

elasticmachine commented Jun 26, 2020 •
edited
Loading

💚 Build Succeeded

Pipeline View Test View Changes Artifacts preview

Expand to view the summary

Build stats

  • Build Cause: [kaiyan-sheng commented: jenkins, run the tests please]

  • Start Time: 2020-07-01T22:15:18.490+0000

  • Duration: 56 min 55 sec

Test stats 🧪

Test Results
Failed 0
Passed 555
Skipped 128
Total 683

@kwinstonix kwinstonix force-pushed the fix-ddos-metrics-tags branch from fabc349 to aacdd61 Compare June 26, 2020 10:58
@kaiyan-sheng kaiyan-sheng self-assigned this Jun 26, 2020
@kaiyan-sheng kaiyan-sheng added needs_backport PR is waiting to be backported to other branches. review Team:Platforms Label for the Integrations - Platforms team labels Jun 26, 2020
@botelastic botelastic bot removed the needs_team Indicates that the issue/PR needs a Team:* label label Jun 26, 2020
@kaiyan-sheng
Copy link
Copy Markdown
Contributor

kaiyan-sheng commented Jun 26, 2020

jenkins, run the tests please

kaiyan-sheng
kaiyan-sheng approved these changes Jul 1, 2020
View reviewed changes
Copy link
Copy Markdown
Contributor

@kaiyan-sheng kaiyan-sheng left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the contribution!!! @kwinstonix This looks good to me!

@kaiyan-sheng
Copy link
Copy Markdown
Contributor

kaiyan-sheng commented Jul 1, 2020

jenkins, run the tests please

@kaiyan-sheng kaiyan-sheng merged commit f96ef57 into elastic:master Jul 2, 2020
@kaiyan-sheng kaiyan-sheng mentioned this pull request Jul 2, 2020
Merged
6 tasks
@kaiyan-sheng kaiyan-sheng added v7.9.0 and removed needs_backport PR is waiting to be backported to other branches. labels Jul 2, 2020
@kaiyan-sheng kaiyan-sheng mentioned this pull request Jul 2, 2020
Merged
6 tasks
kaiyan-sheng pushed a commit that referenced this pull request Jul 2, 2020
@kwinstonix
[MetricBeat] set tags correctly if the dimension value is ARN (#19433) (
47cd829 
#19582)

* fix setting tags when dimension value is arn format in aws cloudwatch module

(cherry picked from commit f96ef57)

Co-authored-by: martin <kwinstonix@users.noreply.github.com>
kaiyan-sheng pushed a commit that referenced this pull request Jul 2, 2020
@kwinstonix
Cherry-pick #19433 to 7.8: [MetricBeat] set tags correctly if the dim…
07887b1 
…ension value is ARN (#19583)

* [MetricBeat] set tags correctly if the dimension value is ARN (#19433)

* fix setting tags when dimension value is arn format in aws cloudwatch module

(cherry picked from commit f96ef57)

* update changelog

Co-authored-by: martin <kwinstonix@users.noreply.github.com>
v1v added a commit to v1v/beats that referenced this pull request Jul 3, 2020
@v1v
Merge remote-tracking branch 'upstream/master' into feature/windows-o…
0ba6f3b 
…ne-beats

* upstream/master: (35 commits)
  [ci] fix env variable name for xpack filebeats (elastic#19617)
  Cache error responses for cloudfoundry apps metadata (elastic#19181)
  ci: user fixed type of agent (elastic#19625)
  Input v2 cursor testing (elastic#19573)
  Update Jenkinsfile to not inspect removed vendor (elastic#19610)
  Fix ordering and duplicate configs on autodiscover (elastic#19317)
  Prepare input/file for changes in the registrar (elastic#19516)
  Cursor input and manager implementation (elastic#19571)
  [Filebeat] Fix tls mapping in suricata module (elastic#19494)
  [Ingest Manager] Make Agent beta and Constraints experimental (elastic#19586)
  Accept prefix as metric_types for stackdriver metricset in GCP (elastic#19345)
  Implement memlog store operations (elastic#19533)
  introduce journalbeat/pkg in order to provide reusable shared code (elastic#19581)
  Add descriptions to HAProxy fields in Metricbeat (elastic#19561)
  ci: apm-server-update trigered only on upstream, comments, and manual triggered (elastic#19590)
  ci: enable upstream triggering on the packaging job (elastic#19589)
  ci: some jjbb improvements (elastic#19588)
  [MetricBeat] set tags correctly if the dimension value is ARN (elastic#19433)
  [Filebeat] Add default_fields: false to fields.yml in aws module (elastic#19568)
  Add publisher implementation for stateful inputs (elastic#19530)
  ...
melchiormoulin pushed a commit to melchiormoulin/beats that referenced this pull request Oct 14, 2020
@kwinstonix @melchiormoulin
[MetricBeat] set tags correctly if the dimension value is ARN (elasti…
9a486f3 
…c#19433)

* fix setting tags when dimension value is arn format in aws cloudwatch module
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

1 more reviewer

@kaiyan-sheng kaiyan-sheng kaiyan-sheng approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

@kaiyan-sheng kaiyan-sheng

Labels

review Team:Platforms Label for the Integrations - Platforms team v7.8.1 v7.9.0

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

[MetricBeat]set resource tags if dimension value is ARN

4 participants

@kwinstonix @elasticmachine @kaiyan-sheng @andresrc