Skip to content

Commit eeabffa

Browse files
orestisflorestisfl
authored
[Metricbeat] Bump elastic-agent-system-metrics for zswap and cgroups v2 CPU CFS data (#49098)
Bumps the elastic-agent-system-metrics dependency to [v0.14.3](https://github.com/elastic/elastic-agent-system-metrics/releases/tag/v0.14.3), [v0.14.2](https://github.com/elastic/elastic-agent-system-metrics/releases/tag/v0.14.2) also included. ### What's Changed * Add Zswap Metrics by @orestisfl in elastic/elastic-agent-system-metrics#279 * PULL_REQUEST_TEMPLATE: Remove reference to non-existant CHANGELOG.md by @orestisfl in elastic/elastic-agent-system-metrics#286 * fix: remove shared context timeout from matrix tests by @orestisfl in elastic/elastic-agent-system-metrics#285 * Enable modernize linter and apply suggestions by @orestisfl in elastic/elastic-agent-system-metrics#287 * Use json omitzero for struct-typed fields in cgroup structs by @orestisfl in elastic/elastic-agent-system-metrics#288 * [cgv2] Add CPU CFS quota, period, and weight metrics by @orestisfl in elastic/elastic-agent-system-metrics#281 * Add missing zswap debug metrics by @orestisfl in elastic/elastic-agent-system-metrics#289 ## Disruptive User Impact None. New features. ## How to test this PR locally 1. Build metricbeat from the branch: ```bash cd metricbeat mage build ``` 2. Create a test config file `metricbeat.test.yml`: ```yaml metricbeat.modules: - module: system period: 5s metricsets: - memory - process process.include_top_n: by_cpu: 5 by_memory: 5 output.console: pretty: true logging.level: debug ``` 3. Run metricbeat: ```bash ./metricbeat -e -c metricbeat.test.yml ``` 4. Verify zswap memory metrics appear in the output (requires Linux with zswap enabled -- check with `cat /sys/module/zswap/parameters/enabled`): ```bash ./metricbeat -e -c metricbeat.test.yml 2>/dev/null | jq 'select(.metricset.name == "memory") | .system.memory.zswap' ``` Expected output (values will vary): ```json { "compressed": 950353920, "uncompressed": 3813502976 } ``` 5. To also verify the `zswap.debug.*` fields (requires debugfs access), grant the binary the `cap_dac_read_search` capability: ```bash sudo setcap cap_dac_read_search=+ep metricbeat ``` Then re-run. Expected output with debug fields: ```json { "compressed": 950353920, "uncompressed": 3813502976, "debug": { "decompress_fail": 0, "pool_limit_hit": 0, "pool_total_size": 950353920, "reject_alloc_fail": 0, "reject_compress_fail": 0, "reject_compress_poor": 0, "reject_kmemcache_fail": 0, "reject_reclaim_fail": 0, "stored_incompressible_pages": 16779, "stored_pages": 931031, "written_back_pages": 29111 } } ``` 6. Verify `cfs.weight` appears in process cgroup metrics (requires cgroupsv2): ```bash ./metricbeat -e -c metricbeat.test.yml 2>/dev/null | jq 'select(.metricset.name == "process") | .system.process.cgroup.cpu.cfs.weight // empty' ``` 7. Run unit tests: ```bash cd metricbeat go test ./module/system/... ```
1 parent a2f4037 commit eeabffa

File tree

10 files changed

+286
-9
lines changed

10 files changed

+286
-9
lines changed

‎NOTICE.txt‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -11175,11 +11175,11 @@ Contents of probable licence file $GOMODCACHE/github.com/elastic/elastic-agent-l
1117511175

1117611176
--------------------------------------------------------------------------------
1117711177
Dependency : github.com/elastic/elastic-agent-system-metrics
11178-
Version: v0.14.1
11178+
Version: v0.14.3
1117911179
Licence type (autodetected): Apache-2.0
1118011180
--------------------------------------------------------------------------------
1118111181

11182-
Contents of probable licence file $GOMODCACHE/github.com/elastic/elastic-agent-system-metrics@v0.14.1/LICENSE.txt:
11182+
Contents of probable licence file $GOMODCACHE/github.com/elastic/elastic-agent-system-metrics@v0.14.3/LICENSE.txt:
1118311183

1118411184
Apache License
1118511185
Version 2.0, January 2004

‎changelog/fragments/1772011445-zswap-metrics.yaml‎

Lines changed: 45 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,45 @@
1+
# REQUIRED
2+
# Kind can be one of:
3+
# - breaking-change: a change to previously-documented behavior
4+
# - deprecation: functionality that is being removed in a later release
5+
# - bug-fix: fixes a problem in a previous version
6+
# - enhancement: extends functionality but does not break or fix existing behavior
7+
# - feature: new functionality
8+
# - known-issue: problems that we are aware of in a given version
9+
# - security: impacts on the security of a product or a user’s deployment.
10+
# - upgrade: important information for someone upgrading from a prior version
11+
# - other: does not fit into any of the other categories
12+
kind: feature
13+
14+
# REQUIRED for all kinds
15+
# Change summary; a 80ish characters long description of the change.
16+
summary: Add zswap compressed swap cache metrics to system memory metricset
17+
18+
# REQUIRED for breaking-change, deprecation, known-issue
19+
# Long description; in case the summary is not enough to describe the change
20+
# this field accommodate a description without length limits.
21+
# description:
22+
23+
# REQUIRED for breaking-change, deprecation, known-issue
24+
# impact:
25+
26+
# REQUIRED for breaking-change, deprecation, known-issue
27+
# action:
28+
29+
# REQUIRED for all kinds
30+
# Affected component; usually one of "elastic-agent", "fleet-server", "filebeat", "metricbeat", "auditbeat", "all", etc.
31+
component: metricbeat
32+
33+
# AUTOMATED
34+
# OPTIONAL to manually add other PR URLs
35+
# PR URL: A link the PR that added the changeset.
36+
# If not present is automatically filled by the tooling finding the PR where this changelog fragment has been added.
37+
# NOTE: the tooling supports backports, so it's able to fill the original PR number instead of the backport PR number.
38+
# Please provide it if you are adding a fragment for a different PR.
39+
# pr: https://github.com/owner/repo/1234
40+
41+
# AUTOMATED
42+
# OPTIONAL to manually add other issue URLs
43+
# Issue URL; optional; the GitHub issue related to this changeset (either closes or is part of).
44+
# If not present is automatically filled by the tooling with the issue linked to the PR number.
45+
# issue: https://github.com/owner/repo/1234

‎changelog/fragments/1772636540-zswap-metrics.yaml‎

Lines changed: 45 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,45 @@
1+
# REQUIRED
2+
# Kind can be one of:
3+
# - breaking-change: a change to previously-documented behavior
4+
# - deprecation: functionality that is being removed in a later release
5+
# - bug-fix: fixes a problem in a previous version
6+
# - enhancement: extends functionality but does not break or fix existing behavior
7+
# - feature: new functionality
8+
# - known-issue: problems that we are aware of in a given version
9+
# - security: impacts on the security of a product or a user’s deployment.
10+
# - upgrade: important information for someone upgrading from a prior version
11+
# - other: does not fit into any of the other categories
12+
kind: feature
13+
14+
# REQUIRED for all kinds
15+
# Change summary; a 80ish characters long description of the change.
16+
summary: Add cgroupv2 CPU metrics to system.process dataset
17+
18+
# REQUIRED for breaking-change, deprecation, known-issue
19+
# Long description; in case the summary is not enough to describe the change
20+
# this field accommodate a description without length limits.
21+
# description:
22+
23+
# REQUIRED for breaking-change, deprecation, known-issue
24+
# impact:
25+
26+
# REQUIRED for breaking-change, deprecation, known-issue
27+
# action:
28+
29+
# REQUIRED for all kinds
30+
# Affected component; usually one of "elastic-agent", "fleet-server", "filebeat", "metricbeat", "auditbeat", "all", etc.
31+
component: metricbeat
32+
33+
# AUTOMATED
34+
# OPTIONAL to manually add other PR URLs
35+
# PR URL: A link the PR that added the changeset.
36+
# If not present is automatically filled by the tooling finding the PR where this changelog fragment has been added.
37+
# NOTE: the tooling supports backports, so it's able to fill the original PR number instead of the backport PR number.
38+
# Please provide it if you are adding a fragment for a different PR.
39+
# pr: https://github.com/owner/repo/1234
40+
41+
# AUTOMATED
42+
# OPTIONAL to manually add other issue URLs
43+
# Issue URL; optional; the GitHub issue related to this changeset (either closes or is part of).
44+
# If not present is automatically filled by the tooling with the issue linked to the PR number.
45+
# issue: https://github.com/owner/repo/1234

‎docs/reference/metricbeat/exported-fields-system.md‎

Lines changed: 99 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -725,7 +725,7 @@ Actual memory used and free.
725725

726726

727727
**`system.memory.actual.free`**
728-
: Actual free memory in bytes. It is calculated based on the OS. On Linux this value will be MemAvailable from /proc/meminfo, or calculated from free memory plus caches and buffers if /proc/meminfo is not available. On OSX it is a sum of free memory and the inactive memory. On Windows, it is equal to `system.memory.free`.
728+
: Actual free memory in bytes. It is calculated based on the OS. On Linux this value will be MemAvailable from /proc/meminfo, or calculated from free memory plus caches and buffers if /proc/meminfo is not available. On OSX it is a sum of free memory and the inactive memory. On Windows, it is equal to `system.memory.free`.
729729

730730
type: long
731731

@@ -776,6 +776,98 @@ This group contains statistics related to the swap memory usage on the system.
776776
format: percent
777777

778778

779+
## zswap [_zswap]
780+
781+
Metrics for the zswap compressed swap cache. Available on Linux when zswap is enabled.
782+
783+
**`system.memory.zswap.compressed`**
784+
: Current compressed size of data stored in zswap.
785+
786+
type: long
787+
788+
format: bytes
789+
790+
791+
**`system.memory.zswap.uncompressed`**
792+
: Original uncompressed size of data stored in zswap.
793+
794+
type: long
795+
796+
format: bytes
797+
798+
799+
## debug [_debug]
800+
801+
Detailed zswap statistics from /sys/kernel/debug/zswap. Requires debugfs to be mounted and accessible.
802+
803+
**`system.memory.zswap.debug.decompress_fail`**
804+
: Number of load or writeback attempts that failed due to decompression failure.
805+
806+
type: long
807+
808+
809+
**`system.memory.zswap.debug.pool_limit_hit`**
810+
: Number of times the zswap pool limit was hit, as configured by the zswap.max_pool_percent kernel parameter.
811+
812+
type: long
813+
814+
815+
**`system.memory.zswap.debug.pool_total_size`**
816+
: Total size of the zswap pool in bytes.
817+
818+
type: long
819+
820+
format: bytes
821+
822+
823+
**`system.memory.zswap.debug.reject_alloc_fail`**
824+
: Number of store attempts that failed because the underlying allocator could not get memory.
825+
826+
type: long
827+
828+
829+
**`system.memory.zswap.debug.reject_compress_fail`**
830+
: Number of store attempts that failed due to compression algorithm failure.
831+
832+
type: long
833+
834+
835+
**`system.memory.zswap.debug.reject_compress_poor`**
836+
: Number of store attempts rejected because the compressed page was too big for the allocator to optimally store.
837+
838+
type: long
839+
840+
841+
**`system.memory.zswap.debug.reject_kmemcache_fail`**
842+
: Number of store attempts that failed because the entry metadata could not be allocated (rare).
843+
844+
type: long
845+
846+
847+
**`system.memory.zswap.debug.reject_reclaim_fail`**
848+
: Number of store attempts that failed due to a reclaim failure after pool limit was reached.
849+
850+
type: long
851+
852+
853+
**`system.memory.zswap.debug.stored_incompressible_pages`**
854+
: Number of incompressible pages currently stored in zswap.
855+
856+
type: long
857+
858+
859+
**`system.memory.zswap.debug.stored_pages`**
860+
: Number of pages currently stored in zswap.
861+
862+
type: long
863+
864+
865+
**`system.memory.zswap.debug.written_back_pages`**
866+
: Number of pages written back from zswap to swap when pool limit was reached.
867+
868+
type: long
869+
870+
779871
## network [_network]
780872

781873
`network` contains network IO metrics for a single network interface.
@@ -1238,6 +1330,12 @@ cgroupv2 stats
12381330
type: long
12391331

12401332

1333+
**`system.process.cgroup.cpu.cfs.weight`**
1334+
: CPU weight for the cgroup (cgroupv2). Used by the CFS scheduler to determine the share of CPU time available to the cgroup. Valid values range from 1 to 10000. The default value is 100.
1335+
1336+
type: long
1337+
1338+
12411339
**`system.process.cgroup.cpu.rt.period.us`**
12421340
: Period of time in microseconds for how regularly a cgroup's access to CPU resources is reallocated.
12431341

‎go.mod‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -173,7 +173,7 @@ require (
173173
github.com/elastic/ebpfevents v0.8.0
174174
github.com/elastic/elastic-agent-autodiscover v0.10.2
175175
github.com/elastic/elastic-agent-libs v0.33.3
176-
github.com/elastic/elastic-agent-system-metrics v0.14.1
176+
github.com/elastic/elastic-agent-system-metrics v0.14.3
177177
github.com/elastic/go-elasticsearch/v8 v8.19.0
178178
github.com/elastic/go-freelru v0.16.0
179179
github.com/elastic/go-quark v0.3.0

‎go.sum‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -379,8 +379,8 @@ github.com/elastic/elastic-agent-client/v7 v7.18.0 h1:zXdoErqECfvRjj1c+1Ko4VGuUn
379379
github.com/elastic/elastic-agent-client/v7 v7.18.0/go.mod h1:ChFjpIfSsQOnhWT3se5Euw/o8P4Vf77FxeObE/vUfSc=
380380
github.com/elastic/elastic-agent-libs v0.33.3 h1:Gsq5FA29sUbbZVJbeLCKPyRkAxCrOhv3VtXvuG9Uu6k=
381381
github.com/elastic/elastic-agent-libs v0.33.3/go.mod h1:0xUg7alsNE/WhY9DZRIdTYW75nqSHC1octIAg//j/PQ=
382-
github.com/elastic/elastic-agent-system-metrics v0.14.1 h1:XdV3KWbug/M1dkn8h9Yth9pfdyeFR2Z1iqaypX+ohBg=
383-
github.com/elastic/elastic-agent-system-metrics v0.14.1/go.mod h1:JNfnZrC0viAjlJRUzQKKuMpDlXgjXBn4WdWEXQF7jcA=
382+
github.com/elastic/elastic-agent-system-metrics v0.14.3 h1:v867kcgCVguOX3AYIHEVn2RNracdH40FqqXiZq71pDU=
383+
github.com/elastic/elastic-agent-system-metrics v0.14.3/go.mod h1:JNfnZrC0viAjlJRUzQKKuMpDlXgjXBn4WdWEXQF7jcA=
384384
github.com/elastic/elastic-transport-go/v8 v8.8.0 h1:7k1Ua+qluFr6p1jfJjGDl97ssJS/P7cHNInzfxgBQAo=
385385
github.com/elastic/elastic-transport-go/v8 v8.8.0/go.mod h1:YLHer5cj0csTzNFXoNQ8qhtGY1GTvSqPnKWKaqQE3Hk=
386386
github.com/elastic/fsevents v0.0.0-20181029231046-e1d381a4d270 h1:cWPqxlPtir4RoQVCpGSRXmLqjEHpJKbR60rxh1nQZY4=

0 commit comments

Comments
 (0)