Skip to content

[Metricbeat] Fix getting compute instance metadata with partial zone/region config#18757

Merged
kaiyan-sheng merged 8 commits intoelastic:masterfrom
kaiyan-sheng:metadata_compute
May 28, 2020
Merged

[Metricbeat] Fix getting compute instance metadata with partial zone/region config#18757
kaiyan-sheng merged 8 commits intoelastic:masterfrom
kaiyan-sheng:metadata_compute

Conversation

@kaiyan-sheng
Copy link
Copy Markdown
Contributor

@kaiyan-sheng kaiyan-sheng commented May 26, 2020 •
edited
Loading

What does this PR do?

This PR fixes getting compute instance metadata when zone/region in googlecloud.yml is partial. For example:

- module: googlecloud
  metricsets:
    - compute
  region: "us-"
  project_id: "your project id"
  credentials_file_path: "your JSON credentials file path"
  exclude_labels: false
  period: 1m

This bug is introduced from #17913

Why is it important?

Instances.Get API does not accept partial zone name and Regions.Get API does not accept partial region name. So instead of using the region/zone information from the config, use the zone label from time series metric.

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have made corresponding change to the default configuration files
  • I have added tests that prove my fix is effective or that my feature works
  • I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

@botelastic botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label May 26, 2020
@kaiyan-sheng kaiyan-sheng self-assigned this May 26, 2020
@kaiyan-sheng kaiyan-sheng removed the needs_team Indicates that the issue/PR needs a Team:* label label May 26, 2020
@botelastic botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label May 26, 2020
@kaiyan-sheng kaiyan-sheng added bug needs_backport PR is waiting to be backported to other branches. review Team:Platforms Label for the Integrations - Platforms team labels May 26, 2020
@elasticmachine
Copy link
Copy Markdown
Contributor

elasticmachine commented May 26, 2020

Pinging @elastic/integrations-platforms (Team:Platforms)

@botelastic botelastic bot removed the needs_team Indicates that the issue/PR needs a Team:* label label May 26, 2020
@elasticmachine
Copy link
Copy Markdown
Contributor

elasticmachine commented May 26, 2020 •
edited
Loading

💚 Build Succeeded

Pipeline View Test View Changes Artifacts preview

Expand to view the summary

Build stats

  • Build Cause: [Pull request #18757 updated]

  • Start Time: 2020-05-28T21:12:50.403+0000

  • Duration: 56 min 55 sec

Test stats 🧪

Test Results
Failed 0
Passed 1224
Skipped 199
Total 1423

Steps errors

Expand to view the steps failures

  • Name: Report to Codecov

    • Description: curl -sSLo codecov https://codecov.io/bash for i in auditbeat filebeat heartbeat libbeat metricbeat packetbeat winlogbeat journalbeat do FILE="${i}/build/coverage/full.cov" if [ -f "${FILE}" ]; then bash codecov -f "${FILE}" fi done

    • Duration: 2 min 22 sec

    • Start Time: 2020-05-28T22:05:24.365+0000

    • log

@kaiyan-sheng kaiyan-sheng requested a review from sayden May 27, 2020 20:14
@kaiyan-sheng
Copy link
Copy Markdown
Contributor Author

kaiyan-sheng commented May 28, 2020

More unit tests need to be added with mocking the gcp APIs in a separate PR.

sayden
sayden approved these changes May 28, 2020
View reviewed changes
Copy link
Copy Markdown
Contributor

@sayden sayden left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks fine! I left a comment

@kaiyan-sheng kaiyan-sheng merged commit f4c65f3 into elastic:master May 28, 2020
@kaiyan-sheng kaiyan-sheng deleted the metadata_compute branch May 28, 2020 22:20
@kaiyan-sheng kaiyan-sheng mentioned this pull request May 28, 2020
Merged
6 tasks
@kaiyan-sheng kaiyan-sheng added v7.9.0 and removed needs_backport PR is waiting to be backported to other branches. labels May 28, 2020
@kaiyan-sheng kaiyan-sheng mentioned this pull request May 28, 2020
Merged
6 tasks
kaiyan-sheng pushed a commit that referenced this pull request May 29, 2020
Cherry-pick #18757 to 7.8: [Metricbeat] Fix getting compute instance …
9ef64e3 
…metadata with partial zone/region config (#18840)

* [Metricbeat] Fix getting compute instance metadata with partial zone/region config (#18757)

* use zone for getting instance metadata

(cherry picked from commit f4c65f3)

* update changelog
kaiyan-sheng pushed a commit that referenced this pull request May 29, 2020
Cherry-pick #18757 to 7.x: [Metricbeat] Fix getting compute instance …
8928a7d 
…metadata with partial zone/region config (#18839)

* [Metricbeat] Fix getting compute instance metadata with partial zone/region config (#18757)

* use zone for getting instance metadata

(cherry picked from commit f4c65f3)

* update changelog
v1v added a commit to v1v/beats that referenced this pull request Jun 2, 2020
@v1v
Merge remote-tracking branch 'upstream/master' into feature/module-at…
7509af0 
…-stage-level

* upstream/master: (30 commits)
  Add a GRPC listener service for Agent (elastic#18827)
  Disable host.* fields by default for iptables module (elastic#18756)
  [WIP] Clarify capabilities of the Filebeat auditd module (elastic#17068)
  fix: rename file and remove extra separator (elastic#18881)
  ci: enable JJBB (elastic#18812)
  Disable host.* fields by default for Checkpoint module (elastic#18754)
  Disable host.* fields by default for Cisco module (elastic#18753)
  Update latest.yml testing env to 7.7.0 (elastic#18535)
  Upgrade k8s.io/client-go and k8s keystore tests (elastic#18817)
  Add missing Jenkins stages for Auditbeat (elastic#18835)
  [Elastic Log Driver] Create a config shim between libbeat and the user (elastic#18605)
  Use indexers and matchers in config when defaults are enabled (elastic#18818)
  Fix panic on `metricbeat test modules` (elastic#18797)
  [CI] Fix permissions in MacOSX agents (elastic#18847)
  [Ingest Manager] When not port are specified and the https is used fallback to 443 (elastic#18844)
  [Ingest Manager] Fix install service script for windows (elastic#18814)
  [Metricbeat] Fix getting compute instance metadata with partial zone/region config (elastic#18757)
  Improve error messages in s3 input (elastic#18824)
  Add memory metrics into compute googlecloud (elastic#18802)
  include bucket name when logging error (elastic#18679)
  ...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

1 more reviewer

@sayden sayden sayden approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

@kaiyan-sheng kaiyan-sheng

Labels

bug Team:Platforms Label for the Integrations - Platforms team v7.8.0 v7.9.0

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@kaiyan-sheng @elasticmachine @sayden @andresrc