Skip to content

Log when approaching linux kernel limits #23439

@lesam

Description

@lesam

Proposal:

Periodically (by default once per minute, but configurable) check for breaches of a list of kernel limits. This would be a linux-only feature (at least initially), and checks on other operating systems would be no-ops.

Log a warning if within 30% of OS limits.

Current behavior:
Users run out of file handles / mmaps with no warning

Desired behavior:
Log a warning if within 30% of OS limits.

Unix commands for checking these are given below - these unix commands should be translated to go-specific idioms.

File Handles

Check cat /proc/sys/fs/file-nr - this gives 3 numbers. The first is the total file handles on the system, and the last is the maximum.

Check cat /proc/$(pidof influxd)/limits for the per-process file handle limit, and ls /proc/$(pidof influxd)/fd | wc -l for the number of file handles the process owns.

MMAP regions

Check cat /proc/sys/vm/max_map_count for the per-process limit on memory maps, and cat /proc/$(pidof influxd)/maps | wc -l for the total number of maps used by influxd.

Alternatives considered:
N/A

Use case:
Often users run out of file handles or mmaps, both of which are limited in various places within the kernel, and the error messages are not good.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions