Skip to content

Occasional ZFS collector errors on kstat.zfs.misc.arcstats.memory_available_bytes #2766

@siebenmann

Description

@siebenmann

Periodically I see 'cannot parse expected integer value' errors logged for the ZFS collector trying to collect memory_available_bytes, which comes from /proc/spl/kstat/zfs/arcstat. Support for this was added to deal with #2656, because this kstat is a type 3 (signed int64) instead of the more usual type 4 (unsigned int64).

Looking at the code involved, I believe this is most likely happening when memory_available_bytes goes negative, which it can; this possibility is why it's a signed int64 kstat instead of an unsigned in64. The current code in collector/zfs_linux.go parses both kstatDataUint64 and kstatDataInt64 data with strconv.ParseUint(), which is not going to accept negative numbers.

As a small RFE, it would be nice if this specific error message logged either or both of the specific error from ParseUint() and the actual value that failed to parse. As it is, I'm guessing at the actual problem involved from my knowledge of the range of ``memory_available_bytes` and the code.

Host operating system: output of uname -a

Linux hawkwind.cs 6.4.6-200.fc38.x86_64 #1 SMP PREEMPT_DYNAMIC Mon Jul 24 20:51:12 UTC 2023 x86_64 GNU/Linux

node_exporter version: output of node_exporter --version

node_exporter, version 1.6.1 (branch: HEAD, revision: 4a1b77600c1873a8233f3ffb55afcedbb63b8d84)

node_exporter log output

ts=2023-08-01T14:58:27.022Z caller=collector.go:169 level=error msg="collector failed" name=zfs duration_seconds=0.098266305 err="could not parse expected integer value for \"kstat.zfs.misc.arcstats.memory_available_bytes\""

Are you running node_exporter in Docker?

No.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions