Skip to content

Commit ed6f101

Browse files
author
kaiyan-sheng
authored
[Metricbeat] Add billing metricset into googlecloud module (#20812)
* Add billing metricset to googlecloud module
1 parent 13d20fe commit ed6f101

28 files changed

Lines changed: 1576 additions & 227 deletions

‎CHANGELOG.next.asciidoc‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -667,6 +667,7 @@ https://github.com/elastic/beats/compare/v7.0.0-alpha2...master[Check the HEAD d
667667
- Add host inventory metrics to azure compute_vm metricset. {pull}20641[20641]
668668
- Add host inventory metrics to googlecloud compute metricset. {pull}20391[20391]
669669
- Request prometheus endpoints to be gzipped by default {pull}20766[20766]
670+
- Add billing metricset into googlecloud module. {pull}20812[20812] {issue}20738[20738]
670671

671672
*Packetbeat*
672673

‎NOTICE.txt‎

Lines changed: 212 additions & 212 deletions
Large diffs are not rendered by default.

‎go.mod‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,7 @@ go 1.14
55
require (
66
4d63.com/tz v1.1.1-0.20191124060701-6d37baae851b
77
cloud.google.com/go v0.51.0
8+
cloud.google.com/go/bigquery v1.0.1
89
cloud.google.com/go/pubsub v1.0.1
910
cloud.google.com/go/storage v1.0.0
1011
code.cloudfoundry.org/go-diodes v0.0.0-20190809170250-f77fb823c7ee // indirect

‎metricbeat/docs/fields.asciidoc‎

Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -19812,6 +19812,48 @@ type: object
1981219812

1981319813
--
1981419814

19815+
[float]
19816+
=== billing
19817+
19818+
Google Cloud Billing metrics
19819+
19820+
19821+
*`googlecloud.billing.cost_type`*::
19822+
+
19823+
--
19824+
Cost types include regular, tax, adjustment, and rounding_error.
19825+
19826+
type: keyword
19827+
19828+
--
19829+
19830+
*`googlecloud.billing.invoice_month`*::
19831+
+
19832+
--
19833+
Billing report month.
19834+
19835+
type: keyword
19836+
19837+
--
19838+
19839+
*`googlecloud.billing.project_id`*::
19840+
+
19841+
--
19842+
Project ID of the billing report belongs to.
19843+
19844+
type: keyword
19845+
19846+
--
19847+
19848+
*`googlecloud.billing.total`*::
19849+
+
19850+
--
19851+
Total billing amount.
19852+
19853+
type: float
19854+
19855+
--
19856+
1981519857
[float]
1981619858
=== compute
1981719859

‎metricbeat/docs/images/metricbeat-googlecloud-billing-overview.png‎

876 KB
Loading

‎metricbeat/docs/modules/googlecloud.asciidoc‎

Lines changed: 26 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -149,9 +149,20 @@ For example, if Compute Metricset fetches 14 metrics (which is the number of met
149149

150150
[float]
151151
== Metricsets
152-
Currently, we have `compute`, `loadbalancing`, `pubsub`, `metrics` and
152+
Currently, we have `billing`, `compute`, `loadbalancing`, `pubsub`, `metrics` and
153153
`storage` metricset in `googlecloud` module.
154154

155+
[float]
156+
=== `billing`
157+
This metricset fetches billing metrics from https://cloud.google.com/bigquery[GCP BigQuery]
158+
Cloud Billing allows users to export billing data into BigQuery automatically
159+
throughout the day. This metricset gets access to the daily cost detail table
160+
periodically to export billing metrics for further analysis.
161+
162+
The `billing` metricset comes with a predefined dashboard:
163+
164+
image::./images/metricbeat-googlecloud-billing-overview.png[]
165+
155166
[float]
156167
=== `compute`
157168
This metricset fetches metrics from https://cloud.google.com/compute/[Compute Engine]
@@ -274,13 +285,25 @@ metricbeat.modules:
274285
- "instance/cpu/usage_time"
275286
- "instance/cpu/utilization"
276287
- "instance/uptime"
288+
289+
- module: googlecloud
290+
metricsets:
291+
- billing
292+
period: 24h
293+
project_id: "your project id"
294+
credentials_file_path: "your JSON credentials file path"
295+
dataset_id: "dataset id"
296+
table_pattern: "table pattern"
297+
cost_type: "regular"
277298
----
278299

279300
[float]
280301
=== Metricsets
281302

282303
The following metricsets are available:
283304

305+
* <<metricbeat-metricset-googlecloud-billing,billing>>
306+
284307
* <<metricbeat-metricset-googlecloud-compute,compute>>
285308

286309
* <<metricbeat-metricset-googlecloud-loadbalancing,loadbalancing>>
@@ -291,6 +314,8 @@ The following metricsets are available:
291314

292315
* <<metricbeat-metricset-googlecloud-storage,storage>>
293316

317+
include::googlecloud/billing.asciidoc[]
318+
294319
include::googlecloud/compute.asciidoc[]
295320

296321
include::googlecloud/loadbalancing.asciidoc[]

‎metricbeat/docs/modules/googlecloud/billing.asciidoc‎

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
////
2+
This file is generated! See scripts/mage/docs_collector.go
3+
////
4+
5+
[[metricbeat-metricset-googlecloud-billing]]
6+
[role="xpack"]
7+
=== Google Cloud Platform billing metricset
8+
9+
beta[]
10+
11+
include::../../../../x-pack/metricbeat/module/googlecloud/billing/_meta/docs.asciidoc[]
12+
13+
14+
==== Fields
15+
16+
For a description of each field in the metricset, see the
17+
<<exported-fields-googlecloud,exported fields>> section.
18+
19+
Here is an example document generated by this metricset:
20+
21+
[source,json]
22+
----
23+
include::../../../../x-pack/metricbeat/module/googlecloud/billing/_meta/data.json[]
24+
----

‎metricbeat/docs/modules_list.asciidoc‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -111,7 +111,8 @@ This file is generated! See scripts/mage/docs_collector.go
111111
.2+| .2+| |<<metricbeat-metricset-golang-expvar,expvar>>
112112
|<<metricbeat-metricset-golang-heap,heap>>
113113
|<<metricbeat-module-googlecloud,Google Cloud Platform>> beta[] |image:./images/icon-yes.png[Prebuilt dashboards are available] |
114-
.5+| .5+| |<<metricbeat-metricset-googlecloud-compute,compute>> beta[]
114+
.6+| .6+| |<<metricbeat-metricset-googlecloud-billing,billing>> beta[]
115+
|<<metricbeat-metricset-googlecloud-compute,compute>> beta[]
115116
|<<metricbeat-metricset-googlecloud-loadbalancing,loadbalancing>> beta[]
116117
|<<metricbeat-metricset-googlecloud-metrics,metrics>> beta[]
117118
|<<metricbeat-metricset-googlecloud-pubsub,pubsub>> beta[]

‎x-pack/metricbeat/include/list.go‎

Lines changed: 1 addition & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎x-pack/metricbeat/metricbeat.reference.yml‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -555,6 +555,16 @@ metricbeat.modules:
555555
- "instance/cpu/utilization"
556556
- "instance/uptime"
557557

558+
- module: googlecloud
559+
metricsets:
560+
- billing
561+
period: 24h
562+
project_id: "your project id"
563+
credentials_file_path: "your JSON credentials file path"
564+
dataset_id: "dataset id"
565+
table_pattern: "table pattern"
566+
cost_type: "regular"
567+
558568
#------------------------------- Graphite Module -------------------------------
559569
- module: graphite
560570
metricsets: ["server"]

0 commit comments

Comments
 (0)