Capacity and Usage with df, du, and quotas

1. “Disk full” can mean several different exhausted resources

A write may fail because data blocks are exhausted, inodes are exhausted, a user or project quota is reached, reserved space protects the filesystem, a thin pool is full, metadata space is exhausted, or the backing storage has entered an error state. The first task is classification—not deletion.

flowchart LR
  A["Write fails or \nalert fires"] --> B{"Which limit?"}
  B --> C["Filesystem blocks: df -h"]
  B --> D["Inodes: df -i"]
  B --> E["Tree usage: du"]
  B --> F["Open deleted files: lsof +L1"]
  B --> G["Quota or thin-pool policy"]
  C --> H["Find responsible \nfilesystem"]
  D --> H
  E --> H
  F --> H
  G --> H

2. df reports the filesystem allocator's view

# Human-readable filesystem capacity and type
df -hT

# POSIX-sized output for scripts, restricted to real local filesystems
df -P -x tmpfs -x devtmpfs

# Inode capacity
df -i

# Ask about the filesystem containing a specific path
df -hT /var/lib/my-application 2>/dev/null || df -hT /var

df reports statistics from the mounted filesystem. Its used-space calculation includes allocated blocks not necessarily reachable through ordinary pathname traversal. Filesystem metadata, reserved blocks, snapshots, copy-on-write references, and deleted-but-open files can all affect the result.

Always anchor the investigation to the failing path. Systems commonly contain several mounts, and cleaning the wrong filesystem has no effect.

3. du walks directory entries and sums reachable allocation

# Stay on one filesystem and summarize first-level consumers
sudo du -x -h --max-depth=1 /var 2>/dev/null | sort -h

# Apparent bytes versus allocated bytes for a file or tree
du -h sample-file
du -h --apparent-size sample-file

# Largest regular files on the same filesystem
sudo find /var -xdev -type f -printf '%s\t%p\n' 2>/dev/null \
  | sort -n | tail -n 30 | numfmt --field=1 --to=iec

du normally counts allocated blocks reachable from the selected tree. Hard-linked files are counted according to traversal and implementation rules, sparse files may have large apparent size but small allocation, and mounted subtrees can be excluded with -x. Run broad scans with operational awareness: traversing a huge production tree can generate substantial metadata I/O.

4. Explain df/du gaps before taking action

# Deleted files still open by a process
sudo lsof +L1

# Show processes using one filesystem or mount point
sudo fuser -vm /var

# Verify the mount boundary used by the path
findmnt --target /var
findmnt --submounts /var

5. Inodes and sparse files create non-obvious failures

A filesystem can have free bytes but no free inodes when it contains enormous numbers of small files. Conversely, a sparse file can advertise a large logical size while consuming few blocks until holes are written.

# Compare byte and inode pressure
df -h /var
df -i /var

# Create a safe sparse file in your lab directory
mkdir -p "$HOME/devops-academy/linux/chapter10/lesson03"
cd "$HOME/devops-academy/linux/chapter10/lesson03"
truncate -s 2G sparse.img
ls -lh sparse.img
du -h sparse.img
du -h --apparent-size sparse.img
stat sparse.img

Backup, copy, archive, and quota tools may treat sparse holes differently. When capacity planning, distinguish logical data size from physical allocation and from potential future allocation.

6. Quotas enforce policy below application conventions

Linux filesystems can enforce limits by user, group, and—in supported filesystems—project or directory tree. A soft limit may be exceeded temporarily during a grace period. A hard limit is an immediate ceiling. Quota accounting and enforcement must be enabled for the filesystem and managed with filesystem-appropriate tooling.

# Inspect active mount options for quota-related flags
findmnt -o TARGET,FSTYPE,OPTIONS | grep -E 'quota|usrquota|grpquota|prjquota' || true

# Common reporting commands when quota tooling is installed
quota -s 2>/dev/null || true
sudo repquota -a 2>/dev/null || true

# XFS project-quota reporting on a configured system
sudo xfs_quota -x -c 'report -h' /mount/point 2>/dev/null || true

Quotas are not a substitute for alerting or retention design. They are a containment boundary. Automation must detect quota errors separately from global filesystem exhaustion because remediation ownership differs.

7. Hands-on lab: explain allocation and build a pressure report

lab="$HOME/devops-academy/linux/chapter10/lesson03"
mkdir -p "$lab/data/a" "$lab/data/b"
cd "$lab"

# Create ordinary, sparse, and many-small-file examples.
dd if=/dev/zero of=data/a/allocated.bin bs=1M count=16 status=none
truncate -s 512M data/b/sparse.bin
for n in $(seq 1 200); do printf 'record %s\n' "$n" > "data/b/item-$n.txt"; done

{
  echo '=== filesystem ==='
  df -hT "$lab"
  df -i "$lab"
  echo
  echo '=== allocated tree usage ==='
  du -x -h --max-depth=2 "$lab/data" | sort -h
  echo
  echo '=== apparent tree usage ==='
  du -x -h --apparent-size --max-depth=2 "$lab/data" | sort -h
  echo
  echo '=== file block metadata ==='
  stat data/a/allocated.bin data/b/sparse.bin
} > capacity-report.txt

cat capacity-report.txt

8. Common capacity-analysis mistakes

9. Knowledge check

10. Summary

Capacity diagnosis separates filesystem-wide allocation from reachable tree usage, bytes from inodes, apparent size from allocated blocks, active files from deleted-open descriptors, and global exhaustion from quota policy. Evidence first prevents destructive cleanup and directs remediation to the correct owner and layer.

11. Further reading

• df(1), du(1), stat(1), lsof(8), and fuser(1).
• Filesystem-specific quota documentation, including XFS project quotas where used.
• Application log-rotation, retention, and supported cleanup procedures.