How to Monitor, Predict, and Increase Disk Space on an AWS EC2 Instance

As your applications run longer on EC2 instances, available disk space inevitably fills up with logs, database files, caches, and other persistent data. If unchecked, your application risks crashing or becoming unavailable when disk capacity is saturated.

Having visibility into disk utilization trends is crucial for planning storage growth. In this comprehensive guide, you will learn proactive monitoring and capacity planning practices for expanding storage on AWS EC2 server instances.

Proactive Steps to Avoid Disk Space Issues

Monitor Disk Utilization

Keep a close watch on disk utilization percentage over time:

$ df -h
Filesystem     Used Available Capacity Mounted on
/dev/xvda1     18G       12G   60%     /

Sudden spikes in usage may indicate a process writing excess data. Consistent upward growth suggests your workload requires more capacity.

Set up CloudWatch alarms to notify when usage crosses thresholds, like 80% full. Monitor write activity with CloudWatch disk metrics as well.

Predict Future Growth

Analyze storage need patterns and extrapolate into the future:

• Database growth rate over the past 3-6 months
• Incoming data volumes from sources like mobile apps
• Log file generation estimates

Project when available capacity will be fully consumed and schedule expansions ahead of time.

Compare Vertical vs Horizontal Scaling

Vertical Scaling: Increase disk size of existing instance

• Simple, minimal configuration changes
• Downtime required during resize process
• Cost increases from higher provisioned storage

Horizontal Scaling: Distribute load across new smaller instances

• Achieves redundancy, high availability
• Complex architecture and data migration
• Incurs cost of additional instances

Combine both strategies to balance tradeoffs.

When to Increase Disk Space

Database or Log Volume Increase

applications generate higher than expected logs and database records over time:

MySQL DB Size Over 6 Months  

June   10GB
July   15GB  (+50% MoM)
Aug    20GB  (+33% MoM) 
Sept   30GB  (+50% MoM)
Oct    45GB  (+50% MoM)
Nov    67GB  (+49% MoM)

With 50% monthly database growth, storage needs to expand ahead of demand.

New Feature Launch

Releasing a major feature like video uploads or high-res images will instantly spike storage requirements. Size and replicate volumes ahead of launch to handle influx of data.

Application Problems

Misbehaving code that overwrites files or gets into write/delete loops can prematurely fill up disks within hours. Monitor usage actively to catch such scenarios before they crash systems.

EBS Volume Types Compared

General Purpose SSD (gp2)

• Balance of price and performance
• 3 IOPS per GiB of volume size
• Up to 16000 IOPS per volume

Provisioned IOPS SSD (io1)

• Most consistent high performance
• Expensive, best for mission critical apps
• Can provision up to 64000 IOPS per volume

Step 1: Create EBS Snapshot Backup

Before resizing, protect your data by creating EBS snapshots. There are two methods to take volume backups:

Stop Application, Freeze Filesystem

1. Halt app processes writing to disk
2. Unmount filesystems with umount
3. Take crash-consistent snapshot

Ensures no file consistency issues, but incurs app downtime.

Live Volume Snapshot

1. Initiate snapshot process
2. Briefly freeze filesystem and flush writes
3. Capture point-in-time snapshot

Avoids downtime but may have minor file inconsistencies. Test restore snapshots regularly.

Step 2: Detach vs. In-use Volume Resize

You can expand both detached and attached volumes online. Detaching incurs application outage which may be unacceptable for mission critical systems.

However, resizing an in-use volume has a higher risk of filesystem errors compared to offline detach and resize methods.

The choice depends on your application’s uptime needs and tolerance to minor filesystem corruption incidents.

Step 3: Extend the Partition

After increasing volume size, extend the partition layer before the filesystem:

$ lsblk
NAME   MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
xvda    202:0    0   64G  0 disk
└─xvda1 202:1    0   20G  0 part /

$ sudo growpart /dev/xvda 1 

$ lsblk
NAME    MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
xvda     202:0    0   64G  0 disk
└─xvda1 202:1    0   64G  0 part /

This ensures maximum capacity availability for the filesystem expansion in the next step.

Performance Impacts of Inadequate Disk Space

Applications performance degrades as available storage capacity dwindles:

Disk Usage %	Avg Read IOPS	Avg Write IOPS	Avg Latency
60%	3500	2250	20 ms
70%	3250	2000	25 ms
80%	3000	1750	30 ms
90%	2750	1500	45 ms

Disk contention escalates as the filesystem struggles to find free blocks for writes. Performance metrics clearly show the urgent need to add capacity before disks completely fill up.

Automating EBS Volume Resizing

Using Terraform

With Terraform AWS provider, infra can be described as code:

resource "aws_ebs_volume" "example" {
  availability_zone = "us-west-2a"
  size              = 100

  tags = {
    Name = "DB-Storage-v1"
  }
}

Change the size parameter and reapply to automatically resize volumes.

Using CloudWatch Alarms

Specify an alarm to trigger on high disk utilization to automatically resize volumes:

TRIGGER: 
  DiskUsage > 90% for 1 hour  

ACTION:  
  Resize Disk to next size tier: 
    100 GiB > 150 GiB > 250 GiB etc.

This enables your infrastructure to automatically scale itself independently.

Conclusion

Monitoring disk utilization, forecasting growth requirements, and resizing EBS volumes proactively are essential practices for optimizing costs and ensuring availability as application data volumes change.

Automating volume resizing improves efficiency and reduces risk from delays in manual processes. With the techniques covered in this guide, you can build EC2 server fleets that provide the right storage capacity at the right time for your dynamic workloads.