Fail-Fast versus Best-Effort Pipeline Steps

1. A pipeline needs an explicit failure policy

Not every CI task should stop the pipeline on the first non-zero status. Compilation and mandatory tests are often fail-fast gates; optional reports or independent compatibility checks may be best-effort.

flowchart TD
  S["step result"] --> C{"critical?"}
  C -->|"yes + fail"| F["stop / fail pipeline"]
  C -->|"no + fail"| R["record warning / continue"]
  C -->|"success"| N["next step"]

2. Critical gates should fail clearly

if run_unit_tests; then
  printf 'unit_tests=pass\n'
else
  status=$?
  printf 'unit_tests=fail status=%d\n' "$status" >&2
  exit "$status"
fi

A mandatory gate should not be wrapped in || true merely to keep logs green.

3. Best-effort work still needs status capture

if generate_optional_report; then
  printf 'report=generated\n' >&2
else
  status=$?
  printf 'report=failed status=%d continuing=true\n' \
    "$status" >&2
fi

Continuing is a policy decision; the failure should remain visible.

4. Blanket || true destroys failure information

# Weak:
# important_command || true

# Better:
if important_command; then
  :
else
  status=$?
  printf 'important_command failed=%d\n' "$status" >&2
  decide_policy "$status"
fi

5. Cleanup should run without replacing the primary failure

if build_release; then
  status=0
else
  status=$?
fi

upload_diagnostics || true
cleanup_workspace || true

exit "$status"

Diagnostic upload and cleanup may be best-effort even when the primary build is a hard failure.

6. Independent matrix jobs may tolerate partial failure

A compatibility matrix can have policies such as “all required,” “experimental targets allowed to fail,” or “fail after N critical targets.” Keep those policies in pipeline configuration or a clear aggregation step rather than implicit shell tricks.

7. Collect multiple statuses before deciding

failed=0

for check in lint unit security; do
  if run_check "$check"; then
    printf '%s=pass\n' "$check"
  else
    status=$?
    printf '%s=fail status=%d\n' "$check" "$status" >&2
    ((failed += 1))
  fi
done

(( failed == 0 )) || exit 1

This wait-for-all style is useful when one run should reveal every independent failure.

8. Stop scheduling after the first critical failure

for target in "${targets[@]}"; do
  if deploy "$target"; then
    continue
  else
    status=$?
    printf 'deploy_failed target=%s status=%d\n' \
      "$target" "$status" >&2
    exit "$status"
  fi
done

Fail-fast minimizes further blast radius when later actions depend on earlier success.

9. Warnings must not masquerade as required gates

10. errexit does not define business policy

set -e can stop on unhandled failures, but it cannot decide whether a particular report is optional, whether a test is experimental, or whether rollback should run. Write those policies explicitly.

11. Aggregate exit codes should stay simple

if (( critical_failures > 0 )); then
  exit 1
fi

exit 0

Detailed per-check information belongs in logs, reports, or structured artifacts. The top-level process status can remain a stable success/failure contract.

12. Hands-on lab: mixed hard and soft gates

mkdir -p "$HOME/devops-academy/bash/chapter16/lesson04"
cd "$HOME/devops-academy/bash/chapter16/lesson04"

cat > gates.sh <<'EOF'
#!/usr/bin/env bash
set -u

hard_gate() {
  printf 'hard gate: unit tests\n'
  return 0
}

soft_gate() {
  printf 'soft gate: optional report\n'
  return 7
}

if hard_gate; then
  printf 'hard_gate=pass\n'
else
  status=$?
  printf 'hard_gate=fail status=%d\n' "$status" >&2
  exit "$status"
fi

if soft_gate; then
  printf 'soft_gate=pass\n'
else
  status=$?
  printf 'soft_gate=warn status=%d continuing=true\n' \
    "$status" >&2
fi

printf 'pipeline_decision=pass\n'
EOF

chmod u+x gates.sh
bash ./gates.sh

13. Knowledge check

14. Summary

Pipeline error handling is policy, not syntax. Mark hard gates, soft gates, and post-failure tasks explicitly; preserve the primary failure; continue only when the workflow intentionally permits it; and keep final pipeline status aligned with the real acceptance criteria.

15. Further reading

• GNU Bash Reference Manual — exit status and conditional commands.
• GNU Bash Reference Manual — errexit.
• CI platform documentation for conditional execution and allowed-failure semantics.
• Site Reliability Engineering guidance on failure containment and diagnostics.