Temporary Files, Hidden Files, and Directory Layout

1. Place data according to ownership and lifecycle

A maintainable system separates what is shipped from what changes at runtime. Mixing application binaries, configuration, logs, caches, secrets, and temporary data in one directory complicates upgrades, backup policy, permissions, incident response, and cleanup.

flowchart TD
  A["Application or service"] --> C["Configuration"]
  A --> S["Persistent state"]
  A --> H["Cache"]
  A --> L["Logs"]
  A --> R["Runtime sockets and PID data"]
  A --> T["Temporary working data"]
  C --> ETC["/etc or user config"]
  S --> VAR["/var/lib or user data"]
  H --> CACHE["/var/cache or user cache"]
  L --> LOG["/var/log or journal"]
  R --> RUN["/run"]
  T --> TMP["/tmp or private temp dir"]

2. Temporary resources require unique, private names

Constructing a predictable pathname such as /tmp/my-script.tmp creates race and collision risks. Another user or process might create it first, replace it with a link, or read data written with permissive defaults. mktemp creates a unique file or directory securely according to the requested template and current permissions.

temporary_file=$(mktemp)
temporary_dir=$(mktemp -d)

printf 'temporary file: %s\n' "$temporary_file"
printf 'temporary dir:  %s\n' "$temporary_dir"

stat -c 'mode=%A owner=%U:%G type=%F path=%n' -- \
  "$temporary_file" "$temporary_dir"

rm -f -- "$temporary_file"
rm -rf -- "$temporary_dir"

3. /tmp, /var/tmp, /run, and TMPDIR serve different lifecycles

choose_tmp_base() {
  local candidate=${TMPDIR:-/tmp}
  if [[ -d $candidate && -w $candidate ]]; then
    printf '%s\n' "$candidate"
  else
    printf '%s\n' /tmp
  fi
}

base=$(choose_tmp_base)
work=$(mktemp -d --tmpdir="$base" devops-academy.XXXXXX)
printf 'private work directory: %s\n' "$work"
rm -rf -- "$work"

4. Clean up with a trap while preserving the real exit status

A script can exit through success, an error, or a signal. A cleanup function registered on EXIT provides one central removal path. It should quote variables, tolerate partially created resources, and avoid masking the script’s original status.

#!/usr/bin/env bash
set -u

work=''
cleanup() {
  status=$?
  trap - EXIT INT TERM
  if [[ -n $work && -d $work ]]; then
    rm -rf -- "$work"
  fi
  exit "$status"
}
trap cleanup EXIT INT TERM

work=$(mktemp -d "${TMPDIR:-/tmp}/academy-report.XXXXXX")
printf 'alpha\n' > "$work/input.txt"
tr '[:lower:]' '[:upper:]' < "$work/input.txt" > "$work/output.txt"
cat -- "$work/output.txt"
# EXIT trap removes the private directory.

For service-grade automation, cleanup policy may retain failed evidence instead of deleting it. That decision should be explicit: clean secrets and disposable data, but preserve diagnostic artifacts when required by the runbook.

5. Dotfiles are hidden by convention, not protected

A name whose first character is . is omitted by normal ls output and by standard globs such as *. This is a user-interface convention. Permissions and ownership—not the dot—control access.

lab="$HOME/devops-academy/linux/chapter04/lesson05"
rm -rf -- "$lab"
mkdir -p -- "$lab"
printf 'visible\n' > "$lab/readme.txt"
printf 'setting=true\n' > "$lab/.app.conf"
mkdir -p -- "$lab/.cache"

printf 'Normal listing:\n'
ls -- "$lab"

printf '\nInclude hidden entries except . and ..:\n'
ls -A -- "$lab"

printf '\nLong listing with hidden entries:\n'
ls -Al -- "$lab"

printf '\nNormal glob target set:\n'
cd -- "$lab"
printf '  %s\n' *

6. Conventional Linux directory roles

Real distributions and applications vary. Consult package documentation and service unit configuration rather than inventing arbitrary paths. The key design is separation: code, configuration, durable state, cache, logs, and runtime files have different backup and cleanup rules.

7. User-scoped configuration and XDG directories

Desktop and command-line applications increasingly follow XDG base-directory conventions. They separate user configuration, persistent application data, cache, and runtime state instead of placing everything directly in $HOME.

config_home=${XDG_CONFIG_HOME:-$HOME/.config}
data_home=${XDG_DATA_HOME:-$HOME/.local/share}
cache_home=${XDG_CACHE_HOME:-$HOME/.cache}
state_home=${XDG_STATE_HOME:-$HOME/.local/state}

printf 'config=%s\n' "$config_home"
printf 'data=%s\n'   "$data_home"
printf 'cache=%s\n'  "$cache_home"
printf 'state=%s\n'  "$state_home"

mkdir -p -- \
  "$config_home/devops-academy" \
  "$data_home/devops-academy" \
  "$cache_home/devops-academy" \
  "$state_home/devops-academy"

8. Virtual filesystems are interfaces, not ordinary storage

Paths such as /proc and /sys expose kernel and device information through filesystem-like interfaces. Many entries are generated dynamically and do not represent persistent disk files. /dev contains device nodes and related runtime-managed entries.

printf 'Process identity through /proc:\n'
readlink -- /proc/self/exe
cat -- /proc/self/status | head

printf '\nSelected virtual filesystem mounts:\n'
findmnt -n -o TARGET,FSTYPE /proc /sys /dev /run 2>/dev/null || true

9. Hands-on lab: build a lifecycle-separated user application tree

set -u
app=academy-demo
root="$HOME/devops-academy/linux/chapter04/lesson05-layout"
rm -rf -- "$root"
mkdir -p -- "$root"

config_root=${XDG_CONFIG_HOME:-$HOME/.config}
data_root=${XDG_DATA_HOME:-$HOME/.local/share}
cache_root=${XDG_CACHE_HOME:-$HOME/.cache}
state_root=${XDG_STATE_HOME:-$HOME/.local/state}

config_dir="$config_root/$app"
data_dir="$data_root/$app"
cache_dir="$cache_root/$app"
state_dir="$state_root/$app"
mkdir -p -- "$config_dir" "$data_dir" "$cache_dir" "$state_dir"

printf 'endpoint=https://example.invalid\n' > "$config_dir/app.conf"
printf 'record-001\n' > "$data_dir/records.txt"
printf 'compiled-cache\n' > "$cache_dir/index.cache"
printf 'last_run=never\n' > "$state_dir/status"

work=$(mktemp -d "${TMPDIR:-/tmp}/${app}.XXXXXX")
trap 'rm -rf -- "$work"' EXIT
printf 'transient-build\n' > "$work/build.tmp"

{
  printf 'CONFIG %s\n' "$config_dir/app.conf"
  printf 'DATA   %s\n' "$data_dir/records.txt"
  printf 'CACHE  %s\n' "$cache_dir/index.cache"
  printf 'STATE  %s\n' "$state_dir/status"
  printf 'TEMP   %s\n' "$work/build.tmp"
} > "$root/layout-report.txt"

cat -- "$root/layout-report.txt"
printf '\nTemporary object mode:\n'
stat -c '%A %U:%G %n' -- "$work" "$work/build.tmp"

10. Common temporary and layout mistakes

11. Knowledge check

12. Chapter summary

Chapter 4 established safe filesystem manipulation and interpretation. You can create, copy, move, and remove bounded targets; inspect object types and metadata; reason about inodes and links; control shell-generated target sets; and separate temporary, configuration, state, cache, log, runtime, and application data according to lifecycle.

13. Further reading

• Filesystem Hierarchy Standard — directory purposes and placement conventions.
• GNU Coreutils manuals for mktemp, ls, cp, stat, and install.
• GNU Bash Reference Manual — traps, parameter expansion, and shell startup files.
• XDG Base Directory Specification — user configuration, data, cache, state, and runtime directories.
• Linux manual pages for tmpfile, mkstemp, proc, sysfs, tmpfs, and file-hierarchy.