Chapter 15Lesson 04~100 minutes

systemctl, journalctl, and Service Workflows

On systemd hosts, service automation should ask the service manager for state instead of guessing from process tables. Configuration validation, privilege boundaries, and bounded logs are part of a safe service workflow.

IntermediateDevOps CLI integrationHands-on lab

Learning objectives

By the end of this lesson

  • Distinguish active, enabled, and failed state.
  • Validate config before restart.
  • Use non-interactive privilege escalation.
  • Query bounded journal diagnostics.
  • Verify service and application health after change.

1. systemd exposes service lifecycle and logs through stable commands

On systemd-based Linux hosts, Bash often wraps systemctl for service state and journalctl for diagnostics. Scripts should ask systemd for state instead of inferring it from process names.

systemd service workflow
flowchart LR
  B["Bash"] --> S["systemctl"]
  S --> D["systemd manager"]
  D --> U["unit"]
  D --> J["journal"]
  J --> Q["journalctl"]

2. Query state before acting

if systemctl is-active --quiet myapp.service; then
  printf 'service=active\n'
else
  printf 'service=not-active\n'
fi

is-active is a state query; it is better than searching ps output for a process name.

3. Active and enabled answer different questions

StateQuestionMeaning
activeIs the unit currently running/active?Runtime state
enabledIs it configured to start through install targets?Boot/start policy
failedIs the unit currently in failed state?Failure diagnosis
systemctl is-enabled myapp.service
systemctl is-failed myapp.service

4. Restart only after configuration validation

if myapp --check-config /etc/myapp/config; then
  sudo -n systemctl restart myapp.service
else
  printf 'configuration validation failed\n' >&2
  exit 65
fi

Validation before restart reduces the chance of turning a configuration mistake into an outage.

5. Reload and restart have different operational costs

if systemctl reload myapp.service; then
  printf 'reload=ok\n'
else
  printf 'reload failed; do not silently assume restart is safe\n' >&2
fi

A reload asks the service to apply configuration without full restart when supported. Scripts should not substitute one for the other without an explicit policy.

6. Privilege escalation must be non-interactive

sudo -n systemctl restart myapp.service

In unattended automation, a sudo password prompt can hang the job. Configure narrowly scoped privilege where appropriate or run under a service account with the required rights.

7. journalctl provides structured time and unit scoping

journalctl \
  --unit myapp.service \
  --since '-10 minutes' \
  --no-pager \
  --lines 100

Scope logs by unit and time window so failure diagnostics remain relevant and bounded.

8. Use machine-readable journal output when parsing

journalctl \
  --unit myapp.service \
  --since '-5 minutes' \
  --output json \
  --no-pager |
jq -r '.__REALTIME_TIMESTAMP, .MESSAGE'

Do not build parsers around colored or formatted human journal output.

9. Verify post-restart state

sudo -n systemctl restart myapp.service

for _ in 1 2 3 4 5; do
  if systemctl is-active --quiet myapp.service; then
    printf 'service healthy enough for active-state gate\n'
    break
  fi
  sleep 1
done

Active state is only one signal. Application-level health checks may still be required before declaring rollout success.

10. Collect service diagnostics without hiding the original failure

if sudo -n systemctl restart myapp.service; then
  :
else
  status=$?
  systemctl status myapp.service --no-pager >&2 || true
  journalctl -u myapp.service -n 100 --no-pager >&2 || true
  exit "$status"
fi

11. Hands-on lab: read-only service inspector

mkdir -p "$HOME/devops-academy/bash/chapter15/lesson04"
cd "$HOME/devops-academy/bash/chapter15/lesson04"

cat > service-check.sh <<'EOF'
#!/usr/bin/env bash
set -u

unit=${1:-}

[[ -n $unit ]] || {
  printf 'usage: %s UNIT\n' "$0" >&2
  exit 64
}

command -v systemctl >/dev/null 2>&1 || {
  printf 'systemctl unavailable\n' >&2
  exit 69
}

if systemctl is-active --quiet "$unit"; then
  active=active
else
  active=inactive
fi

if systemctl is-enabled --quiet "$unit" 2>/dev/null; then
  enabled=enabled
else
  enabled=not-enabled
fi

printf 'unit=%s active=%s enabled=%s\n' \
  "$unit" "$active" "$enabled"

journalctl \
  --unit "$unit" \
  --no-pager \
  --lines 5 2>/dev/null || true
EOF

chmod u+x service-check.sh
printf 'Example: ./service-check.sh ssh.service\n'

Verification checklist

12. Knowledge check

Question 1. What is the difference between active and enabled?

Question 2. Why validate config before restart?

Question 3. Why use bounded journal queries?

Question 4. Does systemd active state necessarily prove the application is healthy?

13. Summary

systemd automation should query unit state directly, distinguish runtime from enablement, validate configuration before mutation, use non-interactive privilege, bound journal diagnostics, and combine service state with application-level health where needed.

14. Further reading

  • systemd systemctl(1) documentation.
  • systemd journalctl(1) documentation.
  • systemd unit-state documentation.
  • sudo documentation for non-interactive execution.
Next lesson

Wrapping Cloud and Infrastructure CLIs

Continue Chapter 15 by applying safe Bash orchestration patterns to another core DevOps tool.

Keep the academy open

Support free, practical DevOps education.

Every lesson is designed to remain readable in a browser, downloadable from GitHub, and usable without a paid learning platform. Contributions help expand and maintain the curriculum.

Ethereum / ERC-20
0x716c4Ab160C4B66F31a28AE2448BfF68fc3a2ef0 Send only Ethereum/ERC-20 compatible assets to this address.