[Metricbeat] Add billing metricset into googlecloud module#20812
Merged
kaiyan-sheng merged 18 commits intoelastic:masterfrom Sep 2, 2020
kaiyan-sheng:gcp_billing
Merged
[Metricbeat] Add billing metricset into googlecloud module#20812kaiyan-sheng merged 18 commits intoelastic:masterfrom kaiyan-sheng:gcp_billing
kaiyan-sheng merged 18 commits intoelastic:masterfrom
kaiyan-sheng:gcp_billing
Conversation
Contributor
|
Pinging @elastic/integrations-platforms (Team:Platforms) |
Contributor
💔 Tests Failed
Expand to view the summary
Build stats
Test stats 🧪
Test errorsExpand to view the tests failures
Steps errorsExpand to view the steps failures
Log outputExpand to view the last 100 lines of log output
|
added 9 commits
August 26, 2020 20:45
exekias
suggested changes
Sep 1, 2020
Contributor
exekias
left a comment
There was a problem hiding this comment.
This will be a great addition! I left a few minor comments
| "googlecloud": { | ||
| "billing": { | ||
| "cost_type": "regular", | ||
| "invoice_month": "202008", |
Contributor
There was a problem hiding this comment.
Would it make sense to convert this to a timestamp?
Contributor
Author
There was a problem hiding this comment.
Hmm I think we should keep it as a string/keyword to match invoice.month in billing table schema from BigQuery: https://cloud.google.com/billing/docs/how-to/export-data-bigquery-tables#data-schema
added 7 commits
September 1, 2020 11:46
exekias
approved these changes
Sep 2, 2020
Contributor
Author
|
CI failed with |
6 tasks
v1v
added a commit
to v1v/beats
that referenced
this pull request
Sep 2, 2020
…ne-2.0 * upstream/master: (87 commits) [packaging] Normalise GCP bucket folder structure (elastic#20903) [Metricbeat] Add billing metricset into googlecloud module (elastic#20812) Include python docs in devguide index (elastic#20917) Avoid generating incomplete configurations in autodiscover (elastic#20898) Improve docs of leaderelection configuration (elastic#20601) Document how to set the ES host and Kibana URLs in Ingest Manager (elastic#20874) docs: Update beats for APM (elastic#20881) Adding cborbeat to community beats (elastic#20884) Bump kibana version to 7.9.0 in x-pack/metricbeat (elastic#20899) Kubernetes state_daemonset metricset for Metricbeat (elastic#20649) [Filebeat][zeek] Add new x509 fields to zeek (elastic#20867) [Filebeat][Gsuite] Add note about admin in gsuite docs (elastic#20855) Ensure kind cluster has RFC1123 compliant name (elastic#20627) Setup python paths in test runner configuration (elastic#20832) docs: Add `processor.event` info to Logstash output (elastic#20721) docs: update cipher suites (elastic#20697) [ECS] Update ecs to 1.6.0 (elastic#20792) Fix path in hits docs (elastic#20447) Update filebeat azure module documentation (elastic#20815) Remove duplicate ListGroupsForUsers in aws/cloudtrail (elastic#20788) ...
kaiyan-sheng
pushed a commit
that referenced
this pull request
Sep 4, 2020
melchiormoulin
pushed a commit
to melchiormoulin/beats
that referenced
this pull request
Oct 14, 2020
…0812) * Add billing metricset to googlecloud module
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
This PR is to add billing metricset into googlecloud module. For GCP, there is no direct billing API like AWS Cost Explorer to query for billing information. In order to get the billing date, users need to export billing reports into GCP BigQuery database and then this billing metricset can make some queries to get the actual billing data out.
Why is it important?
This will make GCP users' life a lot easier when tracking billing data 😄
Checklist
CHANGELOG.next.asciidocorCHANGELOG-developer.next.asciidoc.How to test this PR locally
It took me a while to get permissions for GCP billing account. Not sure how this will be tested in a short period of time. The config I used for billing metricset is:
Related issues
Screenshots