awk for Field-Oriented Processing

1. awk processes records and fields

flowchart LR
  R["record"] --> F["fields"]
  F --> P{"pattern?"}
  P -->|"yes"| A["action"]
  P -->|"no"| N["next record"]
  A --> N

By default, one input line is a record, $0 is the whole record, and $1, $2, ... are fields.

2. Print selected fields

printf '%s\n' 'api prod 3' 'worker staging 2' |
awk '{ print $1, $3 }'

3. Control FS and OFS explicitly

printf '%s\n' 'api:prod:3' 'worker:staging:2' |
awk -F: 'BEGIN { OFS="," } { print $1, $2, $3 }'

4. Patterns combine filtering and processing

awk -F: '$2 == "prod" { print $1 }' services.txt

A single awk program can often replace several tiny extraction/filter stages while keeping the record model visible.

5. Numeric aggregation is natural in awk

awk -F: '
  { total += $3 }
  END { print "replicas=" total }
' services.txt

6. Associative arrays support grouping

awk -F: '
  { count[$2]++ }
  END {
    for (env in count)
      print env, count[env]
  }
' services.txt

7. printf controls report formatting

awk -F: '
  { printf "%-12s env=%-8s replicas=%d\n", $1, $2, $3 }
' services.txt

8. Pass shell values with -v

environment='prod'

awk -F: -v env="$environment" '
  $2 == env { print $1 }
' services.txt

9. awk can produce a validation status

awk -F: '
  $3 !~ /^[0-9]+$/ { bad=1 }
  END { exit bad ? 1 : 0 }
' services.txt

printf 'status=%d\n' "$?"

10. Hands-on lab: aggregate service inventory

mkdir -p "$HOME/devops-academy/bash/chapter08/lesson04"
cd "$HOME/devops-academy/bash/chapter08/lesson04"

cat > services.txt <<'EOF'
api:prod:3
worker:prod:2
cache:staging:1
metrics:prod:1
scheduler:staging:2
EOF

awk -F: '
BEGIN { OFS="\t" }
{
  services[$2]++
  replicas[$2] += $3
}
END {
  for (env in services)
    print env, services[env], replicas[env]
}
' services.txt |
LC_ALL=C sort

11. Knowledge check

12. Summary

awk is a small record-processing language. Use it when text manipulation becomes field logic: selection, arithmetic, grouping, counting, and formatted reporting.

13. Further reading

• GNU Awk User’s Guide.
• POSIX awk utility specification.
• GNU Awk documentation on variable assignment.
• ShellCheck documentation on quoting around awk.