Monitoring Disk Space Usage on CentOS Servers

As a system administrator, keeping track of disk space usage on CentOS servers is an important task. Running out of storage can cause critical systems and applications to fail. In this comprehensive guide, we will cover multiple tools and techniques to monitor disk usage on CentOS 7/8 systems.

Why Monitoring Disk Space is Important

Here are some key reasons why closely tracking disk utilization is essential:

• Prevent outages. If disks fill up entirely, applications can crash and servers may become unresponsive. Proactively keeping space free avoids catastrophic failures.

• Plan storage needs. By understanding current disk usage trends, you can better predict when additional storage must be provisioned.

• Locate heavy consumers. Identifying directories and files consuming excess space allows you to clean up or archive data.

• Optimize performance. When disks start to fill up, performance can degrade significantly as fragmentation increases.

Checking Overall Disk Usage with df

The most basic way to check disk usage on CentOS is using the df (disk free) command. df shows overallUsed and Available space for all mounted filesystems.

Here is an example running df to view all disks on a system:

[root@server ~]# df -h
Filesystem             Size  Used Avail Use% Mounted on
devtmpfs               911M     0  911M   0% /dev
tmpfs                  920M     0  920M   0% /dev/shm
tmpfs                  920M  8.6M  912M   1% /run
tmpfs                  920M     0  920M   0% /sys/fs/cgroup
/dev/mapper/centos-root
                       50G  3.6G   47G   8% /                      
/dev/sda1             1014M  184M  831M  19% /boot
tmpfs                  184M     0  184M   0% /run/user/0

This output shows the filesystem type, total size, used space, available space, percentage utilized and where each disk is mounted.

Some key df parameters:

• -h – Show sizes in human readable format (KB, MB, GB) rather than bytes
• -T – Add filesystem type column
• -i – Display inode utilization

So for a summary of human readable storage with types:

df -Th

Inspecting Specific Disks

You can also inspect disk usage for specific disks by passing the mounted path or device file to df.

For example, to check just the root disk:

df -h /

Or for partition /dev/sdb1:

df -h /dev/sdb1

This can help identify which disks are closest to reaching maximum capacity.

Analyzing Disk Usage by Directory with du

While df shows overall file system utilization, the du (disk usage) command reports disk usage at the folder and file level.

Running du without any arguments will calculate size recursively through the entire directory tree starting from the current working directory.

Here is sample output summarizing disk usage under /var:

[root@server ~]# du -sh /var/*
252M    /var/cache
12M     /var/lib
4.0K    /var/local
16K     /var/lock
4.0K    /var/log
252M    /var/opt
4.0K    /var/run
16M     /var/spool
12K     /var/tmp

This breakdown can pinpoint which directories are using up space.

Some useful du options:

• -s – Display only a total sum for each directory
• -h – Human readable sizes
• -a – Show individual files as well as totals
• –max-depth – Descend only a certain number of levels deep

For instance, to analyze files hogging space within /var/log:

du -ah --max-depth=1 /var/log

By mixing du and sort, we can discover the largest space consumers:

du -sh /var/* | sort -h

This helps easily identify problem directories exceeding disk space quotas.

Finding Large Files with find

Sometimes heavy disk usage is due to a few ultra large files. The find command can locate all files over a certain size threshold.

For example, to print all files 2GB or larger under /mnt:

find /mnt -type f -size +2G

Helpful options for finding disk hogs:

• -type – Match only files (f), directories (d) or other types
• -size – Filter by sizes value (+2G for greater than 2GB)
• -exec – Run a command against matching results

So to delete all 50MB archives under /var/backups:

find /var/backups -type f -name ‘*.tar.gz‘ -size +50M -exec rm {} \;

Monitoring Disk IO with iostat

The iostat tool reports detailed storage metrics like:

• IO read/write rates
• Average wait times
• Utilization
• Saturation

This helps identify disks under heavy load.

For example, to view current disk stats live:

iostat -xm 5

Key parameters:

• -x – Show extended stats
• -m – Output in MB/sec instead of blocks
• 5 – Refresh every 5 seconds

This makes it easy to spot disks with high utilization and saturation during peak traffic.

Checking Disk Space with Nagios

Nagios is a popular open source monitoring system on Linux. It can send alerts when disk space crosses thresholds.

First, enable the check_disk plugin:

yum install nagios-plugins-disk

Then add a command definition:

define command {
        command_name    check_disk
        command_line    $USER1$/check_disk -w $ARG1$ -c $ARG2$ -p $ARG3$
}

And a service monitor:

define service {
        use                             generic-service
        host_name                       server1
        service_description             Disk Space /
        check_command                   check_disk!20%!10%!/
}

This will warn when disk use hits 20% and critical at 10% on /.

Nagios provides robust monitoring capabilities to automatically track disk capacity.

Predicting Future Space Needs

To plan infrastructure growth, historical disk metrics help predict how much space you’ll need down the road.

Linear Regression Analysis

One approach is to use linear regression in a spreadsheet. Plot past used disk space over time and add a trendline to forecast when you will hit capacity. Monitor key directories and filesystems independently.

Log Monitoring Tools

Solutions like the ELK stack, Graylog or Splunk analyze logs to chart historical disk use. You can then extrapolate trends to estimate future space requirements.

Carefully monitoring consumption this way provides advance warning for provisioning additional storage.

Managing Disk Pressure

When applications start to run low on free disk buffers, performance can degrade. Linux exposes this memory pressure so you can take action.

Check current disk pressure:

[root@server ~]# cat /proc/pressure 
Some writeback buffers are dirty/writeback: YES
Physical memory reclaim is stalled: NO
Virtual memory reclaim is stalled: NO     

“YES” for “Some writeback buffers are dirty/writeback” indicates heavy disk load.

When pressure builds you may want to flush dirtied buffers with:

sync & echo 3 > /proc/sys/vm/drop_caches

Or disable swapping temporarily. Monitoring pressure helps stability when disk space gets tight.

Ready to Monitor Disks

I hope this overview has armed you with a multitude of tools and techniques to closely track disk utilization on CentOS systems. Measure overall usage, drill down on specific directories, locate large files, watch IO activity patterns, forecast future needs and keep pressure in check. Your CentOS servers will hum along smoothly.

Let me know if you have any other favorite disk space monitoring methods!