Skip to content

[Metricbeat] allow partial region and zone in googlecloud module config#17913

Merged
kaiyan-sheng merged 7 commits intoelastic:masterfrom
kaiyan-sheng:regions_gcp
Apr 27, 2020
Merged

[Metricbeat] allow partial region and zone in googlecloud module config#17913
kaiyan-sheng merged 7 commits intoelastic:masterfrom
kaiyan-sheng:regions_gcp

Conversation

@kaiyan-sheng
Copy link
Copy Markdown
Contributor

@kaiyan-sheng kaiyan-sheng commented Apr 22, 2020

What does this PR do?

This PR is to enable partial region and partial zone in googlecloud module config. User can choose to use wildcard * or not. For example:

- module: googlecloud
  metricsets:
    - compute
  region: "us-west"
  project_id: "elastic-observability"
  credentials_file_path: "your JSON credentials file path"
  exclude_labels: false
  period: 60s

This config will enable data collection from all regions start with us-west, which includes us-west1, us-west2, us-west3 and us-west4.

- module: googlecloud
  metricsets:
    - compute
  region: "us-*"
  project_id: "elastic-observability"
  credentials_file_path: "your JSON credentials file path"
  exclude_labels: false
  period: 60s

Similarly this config will enable data collection from all regions start with us-, which includes all regions starts with us-west, us-east and us-central.

Why is it important?

This will allow users to not copy paste region/zone names when they want to monitor several regions/zones. If user wants to monitor all regions in both US and Europe, they still need to copy paste but that's very minimum. For example:

- module: googlecloud
  metricsets:
    - compute
  region: "us-*"
  project_id: "elastic-observability"
  credentials_file_path: "your JSON credentials file path"
  exclude_labels: false
  period: 60s
- module: googlecloud
  metricsets:
    - compute
  region: "europe-*"
  project_id: "elastic-observability"
  credentials_file_path: "your JSON credentials file path"
  exclude_labels: false
  period: 60s

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have made corresponding change to the default configuration files
  • I have added tests that prove my fix is effective or that my feature works
  • I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

How to test this PR locally

There are two parts need to be tested. One for regions and another for zones.

  • Test for regions:
    Enable googlecloud with this config and you should see metrics collection from all regions start with us-.
- module: googlecloud
  metricsets:
    - compute
  region: "us-"
  project_id: "elastic-observability"
  credentials_file_path: "your JSON credentials file path"
  exclude_labels: false
  period: 60s
  • Test for zones:
    Enable googlecloud with this config and you should see metrics collection from all zones start with europe-west3-, which includes europe-west3-a, europe-west3-b and europe-west3-c.
- module: googlecloud
  metricsets:
    - compute
  zone: "europe-west3-"
  project_id: "elastic-observability"
  credentials_file_path: "your JSON credentials file path"
  exclude_labels: false
  period: 60s

Related issues

@kaiyan-sheng kaiyan-sheng self-assigned this Apr 23, 2020
@kaiyan-sheng kaiyan-sheng added enhancement Metricbeat Metricbeat needs_backport PR is waiting to be backported to other branches. review Team:Platforms Label for the Integrations - Platforms team labels Apr 23, 2020
@elasticmachine
Copy link
Copy Markdown
Contributor

Pinging @elastic/integrations-platforms (Team:Platforms)

Copy link
Copy Markdown
Contributor

@blakerouse blakerouse left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it be better to only enable this behavior with a wildcard?

us-* instead of us-

@kaiyan-sheng
Copy link
Copy Markdown
Contributor Author

Good point! I will add that! Thanks @blakerouse !!

@kaiyan-sheng
Copy link
Copy Markdown
Contributor Author

@blakerouse I added * as an option. Let me know what you think! Thanks!

Copy link
Copy Markdown
Contributor

@blakerouse blakerouse left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see that you allowed with and without the * that is nice. Looks like mage update might need to be called before merging. Otherwise looks good.

@kaiyan-sheng
Copy link
Copy Markdown
Contributor Author

Yeah, gcp user who uses their API a lot will probably put region/zone without * but for others, wildcard might make more sense here. Thanks for the suggestion.

@kaiyan-sheng kaiyan-sheng merged commit 1eb3032 into elastic:master Apr 27, 2020
@kaiyan-sheng kaiyan-sheng deleted the regions_gcp branch April 27, 2020 15:46
@kaiyan-sheng kaiyan-sheng added v7.8.0 and removed needs_backport PR is waiting to be backported to other branches. labels Apr 27, 2020
kaiyan-sheng pushed a commit that referenced this pull request Apr 28, 2020
… in googlecloud module config (#18015)

* [Metricbeat] allow partial region and zone in googlecloud module config (#17913)

* allow partial region and zone in googlecloud module config
* add wildcard support in region and zone

(cherry picked from commit 1eb3032)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement Metricbeat Metricbeat review Team:Platforms Label for the Integrations - Platforms team v7.8.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants