Working with YAML using yq

1. YAML is a structured language, not whitespace with colons

YAML can represent mappings, sequences, scalars, nested data, quoted values, block strings, anchors, tags, and multiple documents. General YAML should be handled with a parser.

2. Be explicit about which yq you mean

Several unrelated tools are named yq. This lesson uses the commonly deployed Go-based yq command with jq-like expressions. Production scripts should check the installed implementation and supported version before depending on specific syntax.

3. Read a scalar field

environment=$(
  yq -r '.environment' config.yaml
)
printf 'environment=%s\n' "$environment"

The exact raw-output flag and behavior should be verified against the yq implementation in your support matrix.

4. Navigate nested mappings and sequences

yq '.services[0].name' config.yaml

yq '.services[] | .name' config.yaml

5. Filter YAML collections with expressions

yq '
  .services[]
  | select(.enabled == true)
  | .name
' config.yaml

6. yq can update values structurally

yq '
  .environment = "staging"
  | .services[0].replicas = 3
' config.yaml

Structural updates preserve YAML meaning better than regex substitution, though formatting and comments may be affected depending on the tool and operation.

7. In-place editing should still be staged when failure matters

tmp=$(mktemp) || exit 1

if yq '
  .environment = "prod"
' config.yaml > "$tmp"; then
  mv -- "$tmp" config.yaml
else
  status=$?
  rm -f -- "$tmp"
  exit "$status"
fi

A temporary-file update gives you a place to validate output before replacing the live config.

8. Pass shell values through supported variable mechanisms

export DEPLOY_ENV=staging

yq '
  .environment = strenv(DEPLOY_ENV)
' config.yaml

9. JSON can be a useful interchange boundary

yq -o=json '.' config.yaml |
jq '.services | length'

Converting YAML to JSON can be useful when downstream tooling already uses jq, as long as the YAML parser performs the conversion rather than Bash text processing.

10. Multiple YAML documents require explicit handling

A YAML stream may contain multiple documents separated by ---. Ensure your yq expression and downstream consumer expect one document or intentionally iterate over several.

11. Kubernetes manifests are YAML-shaped APIs

yq '
  select(.kind == "Deployment")
  | .metadata.name
' manifest.yaml

For live Kubernetes objects, prefer kubectl structured output or server-side API operations when possible. File-level yq processing is best for manifest preparation and inspection, not a substitute for cluster semantics.

12. Hands-on lab: prepare a deployment manifest

mkdir -p "$HOME/devops-academy/bash/chapter13/lesson03"
cd "$HOME/devops-academy/bash/chapter13/lesson03"

cat > deployment.yaml <<'EOF'
apiVersion: apps/v1
kind: Deployment
metadata:
  name: api
spec:
  replicas: 2
  template:
    spec:
      containers:
        - name: api
          image: example/api:1.0
EOF

if command -v yq >/dev/null 2>&1; then
  tmp=$(mktemp) || exit 1

  if yq '
    .spec.replicas = 3
    | .spec.template.spec.containers[0].image = "example/api:2.0"
  ' deployment.yaml > "$tmp"; then
    mv -- "$tmp" deployment.updated.yaml
    yq '.metadata.name, .spec.replicas' deployment.updated.yaml
  else
    status=$?
    rm -f -- "$tmp"
    exit "$status"
  fi
else
  printf 'yq is required for this lab\n' >&2
fi

13. Knowledge check

14. Summary

Use a real YAML parser, pin or validate your yq implementation, perform structural selection and updates, and stage important changes before promotion. Converting YAML to JSON can create a clean boundary for jq-based downstream processing.

15. Further reading

• yq documentation for the implementation used in your environment.
• YAML specification.
• jq Manual — JSON transformations.
• Kubernetes documentation — manifest structure and API resources.