Searching Text with grep and Regular Expressions

1. grep selects records that match a pattern

grep reads lines and prints those that satisfy a pattern. The pattern is not a shell wildcard. The shell first processes quoting, variables, and filename expansion; then grep interprets the resulting pattern using its selected matching syntax.

flowchart TD
  C["Quoted command text"] --> S["Shell parsing and expansion"]
  S --> G["grep receives pattern and inputs"]
  G --> M["Matcher evaluates each record"]
  M --> O["Matching output and exit status"]

2. Start with fixed-string matching

When the target is a literal token such as an error code, URL, or configuration key, grep -F treats regular-expression metacharacters as ordinary characters. This reduces accidental complexity and is often faster.

log="$HOME/devops-academy/linux/chapter05/lesson02/orders.log"

# Literal period characters and brackets need no escaping in fixed mode.
grep -F 'version=1.2.3' -- "$log"
grep -F '[health]' -- "$log"

# Search for any fixed pattern listed in a file.
printf '%s\n' 'ERROR' 'CRITICAL' 'panic' > patterns.txt
grep -F -f patterns.txt -- "$log"

Use -i for case-insensitive matching only when case is semantically irrelevant. In identifiers, paths, and structured values, collapsing case may merge distinct data.

3. Build regular expressions from small parts

Regular expressions describe text structure. GNU grep supports basic regular expressions by default, extended regular expressions with -E, and Perl-compatible expressions with -P when the build supports them. Portable operational work usually favors fixed strings, BRE, or ERE.

# Lines that begin with an ISO-like date and contain ERROR or WARN.
grep -E '^[0-9]{4}-[0-9]{2}-[0-9]{2}.*level=(ERROR|WARN)' -- "$log"

# Match a whole word rather than a substring such as NOTERROR.
grep -w 'ERROR' -- "$log"

# Match only the selected portion, useful for extraction.
grep -Eo 'request_id=[[:alnum:]-]+' -- "$log"

Locale can affect character classes and case folding. POSIX classes such as [[:digit:]], [[:space:]], and [[:alnum:]] communicate intent more clearly than ad hoc ranges, though exact behavior remains locale-aware.

4. Preserve location and context

A matching line without its source may be insufficient evidence. Use filename and line-number options when searching multiple files, and add bounded context only when it helps explain the event.

logs="$HOME/devops-academy/linux/chapter05/lesson02/logs"

# Always show filename and line number.
grep -Hn -F 'request_id=req-1042' -- "$logs"/*.log

# Show two lines before and three after each match.
grep -Hn -B 2 -A 3 -F 'level=ERROR' -- "$logs"/*.log

# Recursive search constrained to log files and excluding archives.
grep -RIn \
  --include='*.log' \
  --exclude='*.gz' \
  -E 'level=(ERROR|CRITICAL)' \
  -- "$logs"

# Stop after the first match per input file.
grep -m 1 -Hn -F 'startup complete' -- "$logs"/*.log

5. Output and exit status are separate signals

grep conventionally returns status 0 when at least one selected line exists, 1 when no line matches, and 2 for an error. “No match” is often a normal result, not a command failure.

if grep -q -F 'deployment complete' -- "$log"; then
  printf '%s\n' 'Deployment completion record found.'
else
  status=$?
  if (( status == 1 )); then
    printf '%s\n' 'Completion record not found yet.'
  else
    printf 'grep failed with status %d\n' "$status" >&2
    exit "$status"
  fi
fi

-q suppresses normal output and is useful when only the condition matters. Be cautious in pipelines: an early successful exit can close the pipe before the producer finishes, and shell options such as pipefail may expose the producer’s resulting failure.

6. Hands-on lab: investigate a request across services

lab="$HOME/devops-academy/linux/chapter05/lesson02"
logs="$lab/logs"
mkdir -p -- "$logs"

cat > "$logs/gateway.log" <<'EOF'
2026-08-04T10:00:01Z level=INFO request_id=req-1042 route=/orders status=202
2026-08-04T10:00:02Z level=WARN request_id=req-1042 message="upstream slow"
2026-08-04T10:00:04Z level=ERROR request_id=req-1042 message="upstream timeout"
EOF
cat > "$logs/orders.log" <<'EOF'
2026-08-04T10:00:01Z level=INFO request_id=req-1042 action=create
2026-08-04T10:00:03Z level=INFO request_id=req-1042 dependency=payments
2026-08-04T10:00:04Z level=ERROR request_id=req-1042 code=PAYMENT_TIMEOUT
EOF
cat > "$logs/payments.log" <<'EOF'
2026-08-04T10:00:03Z level=INFO request_id=req-1042 provider=demo-pay
2026-08-04T10:00:04Z level=CRITICAL request_id=req-1042 message="provider unavailable"
EOF

printf '%s\n' '=== request timeline ==='
grep -Hn -F 'request_id=req-1042' -- "$logs"/*.log | sort -t: -k3,3

printf '%s\n' '=== severe events ==='
grep -HnE 'level=(ERROR|CRITICAL)' -- "$logs"/*.log

printf '%s\n' '=== extracted error codes ==='
grep -hEo 'code=[[:alnum:]_-]+' -- "$logs"/*.log | sort -u

7. Common mistakes and stronger habits

8. Knowledge check

9. Summary

Reliable search starts with a bounded input set and the simplest matching mode. Fixed strings reduce ambiguity; regular expressions express structure; context and source locations preserve evidence; and exit status makes searches usable in automation.

10. Further reading

• GNU Grep manual — invocation, matching modes, regular expressions, context, and exit status.
• GNU Bash Reference Manual — quoting, filename expansion, pipelines, and pipefail.
• POSIX regular-expression and utility specifications.
• Linux manual pages for grep, regex, and pcre2pattern where available.
• GNU Coreutils manual — sort and text-processing composition.