Priorities, nice, cgroups, and Resource Limits

1. Resource controls shape competition and failure

Linux runs many workloads on finite CPUs, memory, storage, and kernel objects. Resource policy should prevent one process from degrading the whole host, while preserving enough capacity for critical services and recovery tools. Different controls operate at different layers: scheduler priority influences CPU competition, resource limits constrain individual process behavior, and control groups govern collections of processes.

2. Niceness is a hint within scheduler policy

For normal fair-scheduled tasks, niceness commonly ranges from -20, higher priority, to 19, lower priority. Ordinary users can usually increase the nice value of their own processes, making them less favored, but decreasing it normally requires privilege or a capability. Niceness matters when runnable tasks compete for CPU; it does not create a hard maximum.

# Launch a low-priority workload.
nice -n 15 bash -c 'while :; do :; done' &
pid=$!

# Inspect policy, niceness, priority, CPU placement, and use.
ps -o pid,ppid,cls,pri,ni,psr,stat,pcpu,comm,args -p "$pid"

# Make the process even less favored.
renice -n 19 -p "$pid"
ps -o pid,cls,pri,ni,stat,pcpu,comm -p "$pid"

kill -TERM "$pid"
wait "$pid" 2>/dev/null || true

3. Resource limits constrain each process lineage

Shells expose inherited resource limits through ulimit; programs can inspect and adjust permitted limits through system calls. A soft limit is the current enforced value. A hard limit is the ceiling an unprivileged process can set for its soft limit. Children inherit limits across process creation and program replacement.

# Show current soft limits in a subshell.
(
  ulimit -Sa
  printf 'open_files_soft=%s\n' "$(ulimit -Sn)"
  printf 'open_files_hard=%s\n' "$(ulimit -Hn)"
)

# Demonstrate a narrow open-file limit without changing the parent shell.
(
  ulimit -Sn 64
  printf 'child_open_file_limit=%s\n' "$(ulimit -Sn)"
  grep -E 'Max open files|Max processes|Max cpu time' "/proc/$$/limits"
)

Do not lower limits in your main interactive shell unless you understand the inheritance effect. Use a subshell, service unit, container, or dedicated lab identity.

4. Cgroups govern a workload as a unit

Control groups organize processes into a hierarchy and apply controllers to the group. Modern systems commonly use cgroup v2 with a unified hierarchy. Service managers and container runtimes create cgroups automatically, which lets operators manage an entire service or container rather than chasing individual worker PIDs.

flowchart LR
  R["Host cgroup root"] --> S["system.slice"]
  R --> U["user.slice"]
  S --> W["web.service"]
  S --> C["ci-runner.service"]
  U --> L["user-1000.slice"]
  W --> WP["web workers and children"]
  C --> CP["build and test processes"]
  L --> LP["interactive sessions"]
  W -. policy .-> P1["CPU weight, memory max, tasks max"]
  C -. policy .-> P2["CPU quota, I/O weight, memory high"]

# Determine cgroup version and current membership.
stat -fc '%T' /sys/fs/cgroup
cat /proc/self/cgroup

# Inspect the current systemd unit and cgroup path when systemd is present.
if command -v systemctl >/dev/null; then
  systemctl status "$(systemctl --user show -p Id --value 2>/dev/null || true)" 2>/dev/null || true
fi

# Useful service-level properties on systemd systems.
systemctl show --property=ControlGroup,CPUWeight,CPUQuotaPerSecUSec,MemoryCurrent,MemoryMax,TasksCurrent,TasksMax 2>/dev/null | head

Key cgroup v2 controls include CPU weight and maximum, memory high and maximum, task limits, I/O weights and maximums, and pressure/accounting files. Availability depends on enabled controllers, delegation, service-manager policy, and privileges.

5. Match the control to the failure mode

Resource controls can create new failure modes. A tight memory maximum may trigger OOM termination; a low descriptor limit may cause connection failures; an aggressive CPU quota may increase latency. Test under realistic load and ensure monitoring explains which limit was reached.

6. Transient scopes make controlled experiments safer

On a systemd host, systemd-run --scope can place a command in a transient scope with resource properties. User-manager support and delegation differ by distribution and session configuration.

lab="$HOME/devops-academy/linux/chapter08/lesson05"
mkdir -p "$lab"

if command -v systemd-run >/dev/null && systemctl --user show-environment >/dev/null 2>&1; then
  unit="da-ch8-l5-$RANDOM.scope"
  systemd-run --user --scope --unit="$unit" \
    -p CPUWeight=10 \
    -p MemoryMax=128M \
    -p TasksMax=32 \
    bash -c '
      printf "pid=%s\n" "$$"
      printf "cgroup:\n"
      cat /proc/self/cgroup
      printf "limits:\n"
      grep -E "Max open files|Max processes" /proc/self/limits
      sleep 5
    ' | tee "$lab/transient-scope.txt"
else
  printf 'User systemd scope unavailable; inspect limits in a subshell instead.\n'
  (
    ulimit -Sn 128
    grep -E 'Max open files|Max processes' "/proc/$$/limits"
    cat /proc/self/cgroup
  ) | tee "$lab/subshell-limits.txt"
fi

7. Hands-on lab: compare priority and limits

Run two controlled CPU loops with different niceness values, record repeated snapshots, inspect inherited limits, then clean up both processes.

lab="$HOME/devops-academy/linux/chapter08/lesson05"
mkdir -p "$lab"

nice -n 0 bash -c 'while :; do :; done' &
normal=$!
nice -n 19 bash -c 'while :; do :; done' &
low=$!

cleanup() {
  kill -TERM "$normal" "$low" 2>/dev/null || true
  wait "$normal" 2>/dev/null || true
  wait "$low" 2>/dev/null || true
}
trap cleanup EXIT

{
  printf 'normal_pid=%s low_priority_pid=%s\n' "$normal" "$low"
  for sample in 1 2 3 4 5; do
    printf '\n=== sample %s ===\n' "$sample"
    ps -o pid,ppid,cls,pri,ni,psr,stat,pcpu,time,comm \
      -p "$normal" -p "$low"
    sleep 1
  done
  printf '\n=== inherited limits ===\n'
  grep -E 'Max cpu time|Max processes|Max open files|Max locked memory' "/proc/$normal/limits"
  printf '\n=== cgroup membership ===\n'
  cat "/proc/$normal/cgroup"
} > "$lab/priority-comparison.txt"

less "$lab/priority-comparison.txt"
cleanup
trap - EXIT

8. Common mistakes

9. Knowledge check

10. Summary

Linux resource governance is layered. Niceness shapes relative CPU scheduling, rlimits constrain process resources, and cgroups govern complete workloads with hierarchical accounting and policy. Select controls based on the failure mode, test under realistic demand, monitor limit events, and preserve enough capacity for observability and recovery.

11. Further reading

• nice(1), renice(1), sched(7), and getrlimit(2).
• systemd.resource-control(5), systemd-run(1), and systemd.exec(5).
• Linux kernel cgroup v2 and pressure-stall-information documentation.