Chapter 08Lesson 02~80 minutes

cut, paste, tr, sort, uniq, and wc

Classic Unix text tools are effective because each solves one narrow transformation. The engineering task is to compose them only when the record and field boundaries remain explicit.

BeginnerText processingHands-on lab

Learning objectives

By the end of this lesson

Extract simple fields with cut.
Combine line-oriented data with paste.
Transform character streams with tr.
Sort and deduplicate records predictably.
Count explicit units with wc.

1. Six small tools, six narrow jobs

ToolPrimary purposeConstraint

cutSelect fields or character positionsBest for simple delimiter contracts

pasteMerge corresponding input linesUseful for aligned streams

trTranslate/delete/squeeze charactersCharacter-level transformation

sortOrder recordsLocale and keys matter

uniqCollapse adjacent duplicatesUsually paired with sort

wcCount lines, words, bytes, charactersChoose the metric explicitly

2. cut extracts fields from simple records

printf '%s\n' 'api:staging:3' 'worker:prod:5' |
cut -d: -f1,3

Not CSV

A delimiter-based cut is not a CSV parser. It is correct only when fields cannot contain the delimiter.

3. paste combines corresponding lines

printf '%s\n' api worker cache > names.txt
printf '%s\n' 8080 9090 6379 > ports.txt
paste -d: names.txt ports.txt

4. tr transforms character sets

printf 'prod-west-1\n' | tr '[:lower:]' '[:upper:]'
printf 'a   b     c\n' | tr -s ' '
printf 'abc123\n' | tr -d '[:digit:]'

5. sort needs the right comparison rule

printf '%s\n' 10 2 30 1 | sort -n

printf '%s\n' 'api 30' 'worker 10' 'cache 20' |
sort -k2,2n

For deterministic bytewise ordering, use a known locale such as LC_ALL=C when appropriate.

6. uniq compares adjacent records

printf '%s\n' api worker api cache api |
sort |
uniq -c

If duplicates are not adjacent, uniq does not see them as one group. sort -u is convenient when you need sorted unique output without counts.

7. wc counts explicit units

FormCountsNote

wc -lNewline countCommon line-count measure

wc -wWhitespace-separated wordsNot structured-field count

wc -cBytesUseful for binary/artifact size

wc -mCharactersLocale-aware where supported

bytes=$(wc -c < artifact.bin)
printf 'bytes=%s\n' "$bytes"

8. Compose only when the record contract stays clear

printf '%s\n' \
  'api:prod' \
  'worker:staging' \
  'api:prod' \
  'cache:prod' |
cut -d: -f1 |
LC_ALL=C sort |
uniq -c

9. Hands-on lab: summarize service data

mkdir -p "$HOME/devops-academy/bash/chapter08/lesson02"
cd "$HOME/devops-academy/bash/chapter08/lesson02"

cat > services.txt <<'EOF'
api:prod:3
worker:prod:2
cache:staging:1
api:prod:3
metrics:prod:1
EOF

printf '%s\n' '--- environments ---'
cut -d: -f2 services.txt | LC_ALL=C sort | uniq -c

printf '%s\n' '--- unique services ---'
cut -d: -f1 services.txt | LC_ALL=C sort -u

printf 'records=%s\n' "$(wc -l < services.txt)"

Verification checklist

Environment counts are grouped correctly.Unique service output is deduplicated.The record count uses redirected wc -l.The format is explicitly simple colon-delimited text.

10. Knowledge check

Question 1. Why is uniq often preceded by sort?

Question 2. What does sort -n change?

Question 3. What does tr operate on?

Question 4. Is cut -d, a full CSV parser?

11. Summary

The classic Unix text tools are powerful because each does one narrow job. Keep delimiter assumptions explicit, choose numeric and locale rules deliberately, and do not force simple character/line tools to parse structured formats.

12. Further reading

GNU Coreutils manual for cut, paste, tr, sort, uniq, and wc.
POSIX utility specifications.
GNU Coreutils documentation on locale-sensitive sorting.
ShellCheck pipeline guidance.

Next lesson

sed for Stream Editing

Continue Chapter 8 by expanding the text-processing toolkit and its reliability rules.

Keep the academy open

Support free, practical DevOps education.

Every lesson is designed to remain readable in a browser, downloadable from GitHub, and usable without a paid learning platform. Contributions help expand and maintain the curriculum.

Ethereum / ERC-20

0x716c4Ab160C4B66F31a28AE2448BfF68fc3a2ef0 Send only Ethereum/ERC-20 compatible assets to this address.