collector/mdadm: Support RAID metrics by rexagod · Pull Request #3031 · prometheus/node_exporter

rexagod · 2024-05-29T07:41:07Z

Use sysfs for RAID stats. Also added tests for the collector.

Fixes: #842

collector/mdadm_linux.go

rexagod · 2024-06-21T23:38:48Z

Not sure what's causing the error here, the logs seem okay (no level=error lines)?

discordianfish · 2024-07-14T10:50:35Z

The log indicates a diff between metrics and expected metrics:

--- collector/fixtures/e2e-output.txt 2024-06-21 23:39:55.007539384 +0000
+++ /tmp/node_exporter_e2e_test.UZ2v15/e2e-output.txt 2024-06-21 23:42:36.000289912 +0000
@@ -1581,6 +1581,14 @@
node_md_blocks_synced{device="md7"} 7.813735424e+09
node_md_blocks_synced{device="md8"} 1.6775552e+07
node_md_blocks_synced{device="md9"} 0
+# HELP node_md_degraded Number of degraded disks on device.
+# TYPE node_md_degraded gauge
+node_md_degraded{device="md0"} 0
+node_md_degraded{device="md1"} 0
+node_md_degraded{device="md10"} 0
+node_md_degraded{device="md4"} 0
+node_md_degraded{device="md5"} 1
+node_md_degraded{device="md6"} 1

HELP node_md_disks Number of active/failed/spare disks of device.

TYPE node_md_disks gauge

node_md_disks{device="md0",state="active"} 2
@@ -1653,6 +1661,14 @@
node_md_disks_required{device="md7"} 4
node_md_disks_required{device="md8"} 2
node_md_disks_required{device="md9"} 4
+# HELP node_md_raid_disks Number of raid disks on device.
+# TYPE node_md_raid_disks gauge
+node_md_raid_disks{device="md0"} 2
+node_md_raid_disks{device="md1"} 2
+node_md_raid_disks{device="md10"} 4
+node_md_raid_disks{device="md4"} 3
+node_md_raid_disks{device="md5"} 3
+node_md_raid_disks{device="md6"} 4

HELP node_md_state Indicates the state of md-device.

TYPE node_md_state gauge

node_md_state{device="md0",state="active"} 1

rexagod · 2024-07-16T13:41:56Z

Ah! It seems I didn't update the e2e output manifest, so the unit test was working fine for me, but the e2e ones were failing. Updated.

discordianfish

LGTM!

rexagod · 2024-08-13T08:54:35Z

cc @SuperQ 🙂

collector/mdadm_linux_test.go

rexagod · 2024-11-27T01:12:09Z

(bump)

discordianfish · 2024-12-19T14:02:34Z

@SuperQ PTAL

rexagod · 2025-02-04T08:11:59Z

Ping.

SuperQ · 2025-02-04T08:51:30Z

Looks like there's a merge conflict with sys.ttar.

rexagod · 2025-02-11T08:28:20Z

The NetBSD error seems to root from go123 being absent, but I can see it's available: https://ftp.netbsd.org/pub/pkgsrc/current/pkgsrc/lang/go123/index.html. The FreeBSD one seems to be due to a metric that wasn't included in the output record?

I'll send a patch.

rexagod · 2025-02-12T07:02:31Z

Please merge #3242 first.

robbat2 · 2025-02-16T00:55:32Z

@rexagod rebase now that #3242 was merged?

Use `sysfs` for RAID stats. Signed-off-by: Pranshu Srivastava <rexagod@gmail.com>

robbat2

LGTM - let's get this merged to unblock the sysfs changes I had planned

* [CHANGE] mdadm: Use sysfs for RAID metrics #3031 * [CHANGE] filesystem: Add erofs in default excluded fs #3313 * [CHANGE] tcpstat: Use std lib binary.NativeEndian #3386 * [FEATURE] pcidevice: Add new collector for PCIe devices #3339 * [FEATURE] AIX: Add more metrics #3338 * [FEATURE] systemd: Add Virtualization metrics #3254 * [FEATURE] swaps: Add new collector #3428 * [ENHANCEMENT] filesystem: Take super options into account for read-only #3387 * [ENHANCEMENT] perf: Add tlb_data metrics #3436 * [BUGFIX] interrupts: Fix OpenBSD interrupt device parsing #3288 * [BUGFIX] diskstats: Simplify condition #3290 * [BUGFIX] thermal: Sanitize darwin thermal strings #3294 * [BUGFIX] filesystem: Fix Darwin collector cgo memory leak #3315 * [BUGFIX] ethtool: Fix returning 0 for sanitized metrics #3335 * [BUGFIX] netdev: Fix Darwin netdev i/o bytes metric #3336 * [BUGFIX] systemd: Fix logging race #3364 * [BUGFIX] filesystem: Fix duplicate Darwin CGO import #3391 Signed-off-by: Ben Kochie <superq@gmail.com>

* [CHANGE] mdadm: Use sysfs for RAID metrics prometheus#3031 * [CHANGE] filesystem: Add erofs in default excluded fs prometheus#3313 * [CHANGE] tcpstat: Use std lib binary.NativeEndian prometheus#3386 * [FEATURE] pcidevice: Add new collector for PCIe devices prometheus#3339 * [FEATURE] AIX: Add more metrics prometheus#3338 * [FEATURE] systemd: Add Virtualization metrics prometheus#3254 * [FEATURE] swaps: Add new collector prometheus#3428 * [ENHANCEMENT] filesystem: Take super options into account for read-only prometheus#3387 * [ENHANCEMENT] perf: Add tlb_data metrics prometheus#3436 * [BUGFIX] interrupts: Fix OpenBSD interrupt device parsing prometheus#3288 * [BUGFIX] diskstats: Simplify condition prometheus#3290 * [BUGFIX] thermal: Sanitize darwin thermal strings prometheus#3294 * [BUGFIX] filesystem: Fix Darwin collector cgo memory leak prometheus#3315 * [BUGFIX] ethtool: Fix returning 0 for sanitized metrics prometheus#3335 * [BUGFIX] netdev: Fix Darwin netdev i/o bytes metric prometheus#3336 * [BUGFIX] systemd: Fix logging race prometheus#3364 * [BUGFIX] filesystem: Fix duplicate Darwin CGO import prometheus#3391 Signed-off-by: Ben Kochie <superq@gmail.com> Signed-off-by: Shashwat Hiregoudar <shashwathiregoudar@gmail.com>

rexagod marked this pull request as draft May 29, 2024 07:41

rexagod marked this pull request as ready for review May 29, 2024 07:50

rexagod force-pushed the 842 branch 2 times, most recently from 62d8a5a to a75d6b8 Compare May 29, 2024 07:52

discordianfish reviewed May 29, 2024

View reviewed changes

collector/mdadm_linux.go Show resolved Hide resolved

dswarbrick mentioned this pull request Jun 11, 2024

Proposal: collect mdraid metrics from sysfs instead of parsing /proc/mdstat #1085

Open

rexagod force-pushed the 842 branch 2 times, most recently from 711ae52 to 59a5f2d Compare June 21, 2024 23:39

rexagod force-pushed the 842 branch from ff93bcc to 2e6a387 Compare July 16, 2024 13:41

rexagod force-pushed the 842 branch from 2e6a387 to c4f172f Compare July 16, 2024 13:44

rexagod requested a review from discordianfish July 16, 2024 13:51

discordianfish approved these changes Jul 18, 2024

View reviewed changes

discordianfish requested a review from SuperQ September 22, 2024 11:11

robbat2 mentioned this pull request Oct 12, 2024

feat: expose MD raid component devices prometheus/procfs#674

Merged

SuperQ requested changes Oct 12, 2024

View reviewed changes

collector/mdadm_linux_test.go Outdated Show resolved Hide resolved

rexagod force-pushed the 842 branch 5 times, most recently from f029912 to 7658c42 Compare October 14, 2024 20:50

rexagod requested a review from SuperQ October 19, 2024 17:34

SuperQ approved these changes Feb 4, 2025

View reviewed changes

rexagod force-pushed the 842 branch from 7658c42 to 56f7e3f Compare February 10, 2025 22:41

collector/mdadm: Support RAID metrics

18aa4ae

Use `sysfs` for RAID stats. Signed-off-by: Pranshu Srivastava <rexagod@gmail.com>

rexagod force-pushed the 842 branch from 56f7e3f to 18aa4ae Compare February 17, 2025 13:08

robbat2 approved these changes Feb 20, 2025

View reviewed changes

rexagod requested review from SuperQ and discordianfish February 24, 2025 08:44

SuperQ approved these changes Feb 24, 2025

View reviewed changes

SuperQ merged commit 0c10545 into prometheus:master Feb 24, 2025
13 checks passed

SuperQ mentioned this pull request Oct 22, 2025

Release v1.10.0 #3446

Merged

Conversation

rexagod commented May 29, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

rexagod commented Jun 21, 2024

Uh oh!

discordianfish commented Jul 14, 2024

HELP node_md_disks Number of active/failed/spare disks of device.

TYPE node_md_disks gauge

HELP node_md_state Indicates the state of md-device.

TYPE node_md_state gauge

Uh oh!

rexagod commented Jul 16, 2024

Uh oh!

discordianfish left a comment

Choose a reason for hiding this comment

Uh oh!

rexagod commented Aug 13, 2024

Uh oh!

Uh oh!

rexagod commented Nov 27, 2024

Uh oh!

discordianfish commented Dec 19, 2024

Uh oh!

rexagod commented Feb 4, 2025

Uh oh!

SuperQ commented Feb 4, 2025

Uh oh!

rexagod commented Feb 11, 2025

Uh oh!

rexagod commented Feb 12, 2025

Uh oh!

robbat2 commented Feb 16, 2025

Uh oh!

robbat2 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

rexagod commented May 29, 2024 •

edited

Loading