Observability, Metrics, Logs, and Machine-Readable Output

1. Observability explains one run

A production script should make it easy to answer what started, what changed, what failed, how long it took, and which resources were affected.

flowchart LR
  R["run"] --> L["logs"]
  R --> M["metrics"]
  R --> O["machine output"]
  R --> D["diagnostics"]
  L --> P["operator"]
  M --> P
  O --> P

2. Keep stdout for results and stderr for diagnostics

printf '%s\n' "$release_id"
printf 'level=INFO event=release_created id=%q\n' "$release_id" >&2

This lets downstream automation capture a clean result without swallowing useful logs.

3. Use stable structured fields

printf 'ts=%s level=INFO event=deploy service=%q env=%q status=ok\n'               "$(date -u '+%Y-%m-%dT%H:%M:%SZ')"               "$service" "$environment" >&2

4. Generate machine-readable JSON with a JSON-aware tool

jq -n               --arg release_id "$release_id"               --arg environment "$environment"               '{release_id:$release_id, environment:$environment, status:"success"}'

5. Give every invocation a run ID

RUN_ID=${RUN_ID:-"release-$(date +%s)-$$"}

log_info() {
  printf 'run_id=%q level=INFO msg=%q\n'                 "$RUN_ID" "$*" >&2
}

A correlation ID ties together retries, remote commands, deployment events, and rollback diagnostics.

6. Record useful counts, durations, and outcomes

start=$SECONDS
deploy_all
status=$?
elapsed=$((SECONDS - start))

printf 'metric=release_duration_seconds value=%d\n' "$elapsed" >&2
printf 'metric=release_status value=%d\n' "$status" >&2

7. Observability must remain secret-safe

8. Hands-on lab: structured release result

mkdir -p "$HOME/devops-academy/bash/chapter20/lesson02"
cd "$HOME/devops-academy/bash/chapter20/lesson02"

cat > observable.sh <<'EOF'
#!/usr/bin/env bash
set -u

RUN_ID="demo-$$"
start=$SECONDS

log() {
  local level=$1
  shift
  printf 'ts=%s run_id=%s level=%s msg=%q\n'                 "$(date -u '+%Y-%m-%dT%H:%M:%SZ')"                 "$RUN_ID" "$level" "$*" >&2
}

log INFO "release started"
sleep 1
release_id=rel-123
elapsed=$((SECONDS - start))
log INFO "release finished duration_s=$elapsed"

printf '{"release_id":"%s","status":"success"}\n' "$release_id"
EOF

chmod u+x observable.sh
./observable.sh > result.json
cat result.json

9. Knowledge check

10. Summary

Production observability combines clean stdout contracts, structured stderr logs, run IDs, durations, outcomes, and bounded diagnostic evidence.

11. Further reading

• POSIX standard streams.
• OpenTelemetry logging and metrics concepts.
• OWASP Logging Cheat Sheet.
• jq documentation.