When Bash Data Structures Are the Wrong Abstraction

1. Know what Bash data structures are good at

These structures align well with Bash's role as glue around command-line programs.

2. Complexity signals that Bash is stretching

• You need nested objects or arrays of objects.
• You are inventing delimiters to serialize maps.
• You need schema validation.
• You repeatedly flatten and reconstruct structured API data.
• You need sorting, grouping, joins, or aggregation over many records.
• You need persistent typed state.
• You need nontrivial concurrency or error objects.

3. JSON plus jq is often a better boundary

cat > services.json <<'EOF'
{
  "services": [
    {"name": "api", "replicas": 3, "enabled": true},
    {"name": "worker", "replicas": 2, "enabled": true}
  ]
}
EOF

jq -r '.services[] | select(.enabled) | [.name, .replicas] | @tsv' \
  services.json

Bash can orchestrate the command while jq owns parsing and structured transformation.

4. Keep API responses structured as long as possible

response=$(curl --fail --silent --show-error "$url") || exit 1

service=$(jq -r '.service' <<<"$response") || exit 1
replicas=$(jq -r '.replicas' <<<"$response") || exit 1

Avoid extracting data with grep and cut when the source format is JSON. Format-aware tools preserve meaning and escaping.

5. CSV is not just comma-separated text

Real CSV supports quoted delimiters, quoted newlines, and escaped quote characters. A Bash IFS=, read ... loop is only valid for a restricted format that explicitly forbids those features.

6. YAML parsing is even less suitable for ad-hoc shell logic

YAML supports nested structures, scalar styles, anchors, tags, and many syntax details. Use a trusted YAML-aware tool or application library. Do not parse general YAML with regular expressions.

7. Move algorithms and object manipulation into Python when appropriate

python3 - <<'PY'
services = [
    {"name": "api", "replicas": 3},
    {"name": "worker", "replicas": 2},
]

for service in services:
    print(f"{service['name']}\t{service['replicas']}")
PY

Bash can remain the outer orchestration layer while Python handles data modeling, validation, calculations, or algorithms.

8. Use stable serialization across process boundaries

flowchart LR
  B["Bash orchestrator"] -->|"JSON"| J["jq / Python"]
  J -->|"TSV or JSON"| B
  B -->|"argv"| C["CLI tools"]

Inside one Bash process, arrays are convenient. Across processes, use argv for simple arguments or a structured serialization such as JSON when the data itself is structured.

9. Do not use source as a general data parser

# Dangerous as a generic data format:
# source untrusted-config.sh

source executes shell code in the current process. It is appropriate only for trusted shell libraries or deliberately executable configuration under a strict trust model.

10. Refactor before data conventions become implicit

If many functions depend on undocumented array keys, flattened associative-array names, or global maps, pause and define a real data contract. A small refactor now is cheaper than debugging invisible coupling later.

11. A practical decision rule

12. Hands-on lab: keep JSON structured

mkdir -p "$HOME/devops-academy/bash/chapter07/lesson05"
cd "$HOME/devops-academy/bash/chapter07/lesson05"

cat > inventory.json <<'EOF'
{
  "services": [
    {"name":"api","environment":"prod","replicas":3},
    {"name":"worker","environment":"prod","replicas":2},
    {"name":"cache","environment":"staging","replicas":1}
  ]
}
EOF

if command -v jq >/dev/null 2>&1; then
  mapfile -t prod_services < <(
    jq -r '.services[] | select(.environment == "prod") | .name' \
      inventory.json
  )

  printf 'prod service count=%d\n' "${#prod_services[@]}"
  printf 'service=%s\n' "${prod_services[@]}"
else
  printf 'jq is required for this lab\n' >&2
fi

13. Knowledge check

14. Summary

Bash arrays and maps are powerful because they solve shell-sized problems. Keep them for arguments, flat lookups, counters, and small collections. Move structured parsing, nested data, large transformations, and application-level modeling to tools built for those jobs.

15. Further reading

• GNU Bash Reference Manual — Arrays.
• jq Manual — JSON processing.
• Python standard-library documentation — json and csv modules.
• ShellCheck documentation — data parsing and quoting pitfalls.