Test Fixtures, Golden Files, and Failure Injection

1. Reliable shell tests reproduce unpleasant production states

Useful test cases include unusual filenames, malformed configuration, stale checkpoints, command failures, concurrency conflicts, and partial output—not only the clean happy path.

flowchart LR
  F["fixture"] --> S["system under test"]
  I["injected failure"] --> S
  S --> O["actual result"]
  G["golden expected result"] --> C["compare"]
  O --> C

2. Fixtures are controlled inputs

fixture_dir="$TEST_TMPDIR/fixture"
mkdir -p -- "$fixture_dir"

printf 'hello\n' > "$fixture_dir/normal.txt"
printf 'world\n' > "$fixture_dir/file with spaces.txt"
touch -- "$fixture_dir/--leading-dash"

Include the edge cases your production script promises to support.

3. Golden files protect stable output contracts

./render-report.sh input.json > actual.txt

diff -u \
  test/golden/report.txt \
  actual.txt

Golden tests are useful for reports, generated configuration, or CLI output whose exact representation matters.

4. Golden updates must be reviewed

Regenerate expected files through an explicit developer command. Never let the test silently overwrite the expectation and call that a pass.

5. Inject fake commands through PATH

fakebin="$TEST_TMPDIR/bin"
mkdir -p -- "$fakebin"

cat > "$fakebin/curl" <<'EOF'
#!/usr/bin/env bash
printf '{"status":"ready"}\n'
EOF
chmod u+x "$fakebin/curl"

PATH="$fakebin:$PATH" ./api-check.sh

This lets tests control dependency behavior without making a real network request.

6. Make fakes return specific failures

cat > "$fakebin/curl" <<'EOF'
#!/usr/bin/env bash
printf 'simulated timeout\n' >&2
exit 28
EOF
chmod u+x "$fakebin/curl"

Now retry, classification, logging, and final status can be tested deterministically.

7. Test partial and stale state

mkdir -p "$state_dir"
printf 'artifact_id=missing-object\n' > "$state_dir/upload.done"

run_recovery

# Assert stale state was detected and reconciled.

8. Reproduce duplicate-run conflicts

./single-run.sh &
first=$!

sleep 0.2
./single-run.sh
second_status=$?

wait "$first"

printf 'second_status=%d\n' "$second_status"

Locking behavior should be tested with actual overlap, not only code inspection.

9. Test under realistic credentials

Tests running as root may bypass permission failures that ordinary users would experience. Build integration environments that reflect production privilege boundaries.

10. Make randomized failures reproducible

seed=${TEST_SEED:-12345}
printf 'test_seed=%s\n' "$seed" >&2

If fuzz-like inputs reveal a bug, logging or fixing the seed makes the CI failure reproducible locally.

11. Build a compact adversarial matrix

Cover pathnames, malformed config, transient/permanent command failures, stale checkpoints, duplicate execution, and partial state. These dimensions map directly to common shell reliability failures.

12. Hands-on lab: failure-injected dependency

mkdir -p "$HOME/devops-academy/bash/chapter17/lesson05/fakebin"
cd "$HOME/devops-academy/bash/chapter17/lesson05"

cat > check-api.sh <<'EOF'
#!/usr/bin/env bash
set -u

if curl --silent https://example.invalid/status >/dev/null; then
  printf 'api=ok\n'
else
  status=$?
  printf 'api=failed status=%d\n' "$status" >&2
  exit "$status"
fi
EOF
chmod u+x check-api.sh

cat > fakebin/curl <<'EOF'
#!/usr/bin/env bash
case ${FAKE_CURL_MODE:-success} in
  success) printf '{"status":"ok"}\n'; exit 0 ;;
  timeout) printf 'simulated timeout\n' >&2; exit 28 ;;
  *) exit 2 ;;
esac
EOF
chmod u+x fakebin/curl

PATH="$PWD/fakebin:$PATH" \
FAKE_CURL_MODE=success \
bash ./check-api.sh

status=0
PATH="$PWD/fakebin:$PATH" \
FAKE_CURL_MODE=timeout \
bash ./check-api.sh || status=$?

printf 'captured_timeout_status=%s\n' "$status"

13. Knowledge check

14. Summary

High-value shell tests reproduce edge cases and failures deliberately. Use adversarial fixtures, reviewable golden files, fake dependencies, stale-state scenarios, concurrency tests, and reproducible randomness.

15. Further reading

• Bats-core documentation.
• GNU Coreutils diff documentation.
• Testing literature on fixtures, golden tests, and fault injection.
• SRE guidance on failure testing and recovery.