Chapter 14Lesson 05~110 minutes

Fleet Automation without Turning Bash into a Config Manager

Bash can coordinate a small fleet effectively, but it should remain an orchestrator rather than evolve into a homegrown configuration-management system. Safe fleet work limits blast radius and makes each host outcome observable.

IntermediateNetworking & remote automationHands-on lab

Learning objectives

By the end of this lesson

  • Represent host inventory as data.
  • Preflight before remote mutation.
  • Roll out in controlled batches.
  • Track per-host status and bounded concurrency.
  • Know when to move to configuration-management tooling.

1. Bash can orchestrate a fleet, but should not become a configuration-management system

Shell is effective for small, explicit batches of remote operations. Once you need persistent desired state, inventory modeling, dependency graphs, drift remediation, rich rollback, and thousands of nodes, purpose-built tools are usually safer.

Know the boundary

Use Bash as an orchestrator around stable remote commands, not as an improvised replacement for Ansible, Salt, Puppet, Chef, or an infrastructure orchestrator.

2. Keep inventory as data

hosts=(
  "ops@web-01"
  "ops@web-02"
  "ops@worker-01"
)

for host in "${hosts[@]}"; do
  printf 'host=%s\n' "$host"
done

For larger inventories, use a file or structured source instead of hard-coding hostnames inside control flow.

3. Preflight before mutation

for host in "${hosts[@]}"; do
  if ssh -o BatchMode=yes -o ConnectTimeout=5 \
    "$host" 'command -v systemctl >/dev/null'; then
    printf 'PREFLIGHT OK host=%s\n' "$host"
  else
    printf 'PREFLIGHT FAIL host=%s\n' "$host" >&2
    exit 1
  fi
done

Discover missing credentials, unreachable nodes, or missing dependencies before changing any host.

4. Roll out in batches

Progressive fleet rollout
flowchart LR
  V["validate"] --> B1["batch 1"]
  B1 --> H["health check"]
  H -->|"pass"| B2["batch 2"]
  H -->|"fail"| S["stop / rollback"]

Batching reduces blast radius and gives health checks a chance to stop a bad deployment before it reaches every host.

5. Record per-host status

declare -A result

for host in "${hosts[@]}"; do
  if ssh -o BatchMode=yes "$host" \
    'sudo -n systemctl restart myapp'; then
    result[$host]=ok
  else
    result[$host]=failed
  fi
done

for host in "${hosts[@]}"; do
  printf '%s\t%s\n' "$host" "${result[$host]}"
done

One aggregate exit status is not enough for fleet diagnostics.

6. Bound remote concurrency

printf '%s\0' "${hosts[@]}" |
xargs -0 -n 1 -P 4 bash -c '
  host=$1
  ssh -o BatchMode=yes -o ConnectTimeout=5 \
    "$host" "hostname"
' _

Remote concurrency should respect bastion capacity, API limits, host load, and failure blast radius.

7. Define fail-fast versus best-effort behavior

PolicyBehaviorUse
Fail fastStop new batches after first critical failureDeployments with shared risk
Best effortAttempt every independent hostDiagnostics and inventory collection
ThresholdStop after error percentage/countLarge homogeneous fleets

8. Verify health after remote mutation

if ssh "$host" 'sudo -n systemctl restart myapp' &&
   curl --fail --silent --show-error \
     --max-time 5 "https://$host/health" >/dev/null; then
  printf 'DEPLOY OK host=%s\n' "$host"
else
  printf 'DEPLOY FAIL host=%s\n' "$host" >&2
fi

Command success is not the same as service readiness. Check the externally meaningful result.

9. Rollback must be designed before rollout

If a deployment can fail after changing several hosts, know how to identify changed hosts, what version to restore, and whether rollback is safe. A rollback command invented during an incident is not a recovery strategy.

10. Remote operations should be rerunnable

Use convergent remote commands so retrying after an SSH disconnect does not duplicate state. Ambiguous remote outcomes are common when the network fails after the remote command may already have executed.

11. Correlate one fleet run

run_id="deploy-$(date +%s)-$$"

printf 'run_id=%s host=%s event=start\n' \
  "$run_id" "$host" >&2

A common run ID ties together per-host logs, retries, health checks, and rollback actions.

12. Hands-on lab: safe fleet skeleton

mkdir -p "$HOME/devops-academy/bash/chapter14/lesson05"
cd "$HOME/devops-academy/bash/chapter14/lesson05"

cat > fleet-run.sh <<'EOF'
#!/usr/bin/env bash
set -u
set -o pipefail

hosts=("$@")
(( ${#hosts[@]} > 0 )) || {
  printf 'usage: %s HOST...\n' "$0" >&2
  exit 64
}

run_id="fleet-$(date +%s)-$$"
failed=0

for host in "${hosts[@]}"; do
  printf 'run_id=%s host=%s phase=preflight\n' \
    "$run_id" "$host" >&2

  if ! ssh \
    -o BatchMode=yes \
    -o ConnectTimeout=5 \
    -T \
    "$host" \
    'command -v uname >/dev/null'; then
    printf 'run_id=%s host=%s result=preflight_failed\n' \
      "$run_id" "$host" >&2
    ((failed += 1))
    continue
  fi

  if ssh -o BatchMode=yes -T "$host" 'uname -s'; then
    printf 'run_id=%s host=%s result=ok\n' \
      "$run_id" "$host" >&2
  else
    printf 'run_id=%s host=%s result=failed\n' \
      "$run_id" "$host" >&2
    ((failed += 1))
  fi
done

printf 'run_id=%s failed=%d total=%d\n' \
  "$run_id" "$failed" "${#hosts[@]}" >&2

(( failed == 0 )) || exit 1
EOF

chmod u+x fleet-run.sh
printf 'Run with one or more reachable SSH hosts.\n'

Verification checklist

13. Knowledge check

Question 1. When does Bash stop being a good fleet-management abstraction?

Question 2. Why use batches?

Question 3. Why record per-host status?

Question 4. Why must remote actions be idempotent?

14. Summary

Bash can safely orchestrate small or bounded fleets when inventory is data, preflight happens before mutation, concurrency is limited, per-host status is recorded, health is verified, and rollback is predefined. For persistent configuration management and large-scale drift control, use a purpose-built system.

15. Further reading

  • OpenSSH documentation.
  • GNU findutils/xargs documentation — bounded parallel execution.
  • Ansible, Salt, Puppet, and Chef documentation — configuration-management capabilities.
  • Site Reliability Engineering guidance — progressive rollout and blast-radius control.
Next lesson

Automating Git Safely from Scripts

Chapter 15 will apply these orchestration patterns to Git, Docker, Kubernetes, systemd, and cloud/infrastructure CLIs.

Keep the academy open

Support free, practical DevOps education.

Every lesson is designed to remain readable in a browser, downloadable from GitHub, and usable without a paid learning platform. Contributions help expand and maintain the curriculum.

Ethereum / ERC-20
0x716c4Ab160C4B66F31a28AE2448BfF68fc3a2ef0 Send only Ethereum/ERC-20 compatible assets to this address.