Chapter 16Lesson 05~165 minutes

Checkpoint Lab — Bisect, Blame, Pickaxe Search, Merge Bases, and Repository Forensics

Checkpoint a production-style repository forensic workflow by seeding a timeout regression, automating bisect, corroborating with pickaxe and ignore-revision blame, checking a support-line merge base, and writing an evidence report.

CheckpointIncident evidenceRegression isolationTriangulation

Learning objectives

  • Seed a reproducible regression and validate a deterministic oracle at both endpoints.
  • Run automated bisect and preserve the candidate classification log and first-bad OID.
  • Use -S, -G, and blame to answer complementary textual and line-origin questions.
  • Prove support-line divergence with merge-base and ancestry tests.
  • Write an incident-style report that explicitly separates what Git evidence proves from what it does not.

1. Checkpoint scenario — investigate a seeded production timeout regression

You will create twelve trunk commits. C7 introduces a deterministic timeout regression, C9 later reformats the same configuration line without changing the bad value, and a stable support branch diverges before the regression. Your job is to isolate the behavioral boundary, corroborate the value transition, explain why default blame points later, compare the support line through its merge base, and produce an incident-style evidence report.

2. Predictions before the investigation

  1. When bisect run starts with C12 bad and C1 good, should it rewrite any existing commits?
  2. Will default blame at C12 attribute the timeout line to C7 or the later C9 formatting commit?
  3. Will git log -S'900' select C7 if C9 preserves the literal 900 while changing whitespace?
  4. If release/stable branched at C5, can that branch reach the C7 regression by ancestry?

3. Build the disposable checkpoint history

Git Bash, Bash, or zsh

mkdir git-forensics-checkpoint
cd git-forensics-checkpoint
git init -b trunk repo
cd repo
git config user.name "Forensics Checkpoint"
git config user.email "forensics-checkpoint@example.invalid"

cat > service.conf <<'EOF'
timeout_ms=100
mode=safe
EOF

cat > oracle.sh <<'EOF'
#!/bin/sh
value=$(sed -n 's/^timeout_ms[[:space:]]*=[[:space:]]*//p' service.conf)
test -n "$value" || exit 125
test "$value" -le 300
EOF

cat > oracle.ps1 <<'EOF'
$line = Get-Content service.conf | Where-Object { $_ -match '^timeout_ms\s*=' } | Select-Object -First 1
if (-not $line) { exit 125 }
$value = [int](($line -split '=',2)[1].Trim())
if ($value -le 300) { exit 0 } else { exit 1 }
EOF

git add .
git commit -m "C1: known-good baseline and deterministic oracle"
C1=$(git rev-parse HEAD)

for n in 2 3 4 5 6; do
  printf "note_%s=pre-regression\n" "$n" >> service.conf
  git add service.conf
  git commit -m "C$n: pre-regression service change"
  eval "C$n=$(git rev-parse HEAD)"
done

# C7 regression: change timeout_ms=100 to timeout_ms=900.
git add service.conf
git commit -m "C7: introduce excessive timeout"
C7=$(git rev-parse HEAD)

printf "note_8=post-regression\n" >> service.conf
git add service.conf
git commit -m "C8: add post-regression note"
C8=$(git rev-parse HEAD)

# C9 formatting only: change "timeout_ms=900" to "timeout_ms = 900".
git add service.conf
git commit -m "C9: format timeout assignment"
C9=$(git rev-parse HEAD)

for n in 10 11 12; do
  printf "note_%s=post-regression\n" "$n" >> service.conf
  git add service.conf
  git commit -m "C$n: later service change"
  eval "C$n=$(git rev-parse HEAD)"
done

git branch release/stable "$C5"
git log --graph --decorate --oneline --all

PowerShell setup note

Use the same commit sequence, replacing the POSIX file append with Add-Content and variables such as $C7 = git rev-parse HEAD. The committed oracle.ps1 provides the PowerShell-compatible automated test path.

4. Record immutable investigation inputs outside the mutable checkout

git status --short --branch
git rev-parse --is-shallow-repository
git show-ref --heads
git rev-parse "$C1"
git rev-parse "$C12"

printf "incident=INC-204\n" > ../INC-204-evidence.txt
printf "known_good=%s\n" "$C1" >> ../INC-204-evidence.txt
printf "known_bad=%s\n" "$C12" >> ../INC-204-evidence.txt

The evidence file is outside the worktree because bisect will repeatedly replace checked-out repository files.

5. Verify the oracle at both endpoints

git switch --detach "$C1"
sh oracle.sh
GOOD_RC=$?

git switch --detach "$C12"
sh oracle.sh
BAD_RC=$?

printf "good_rc=%s\nbad_rc=%s\n" "$GOOD_RC" "$BAD_RC"
git switch trunk

Expected: good_rc=0 and bad_rc=1. Prediction: if this precondition is wrong, automated bisect must not begin.

6. Run automated bisect and preserve its log

git bisect start "$C12" "$C1"
git bisect run sh oracle.sh
CULPRIT=$(git rev-parse refs/bisect/bad)

git bisect log | tee -a ../INC-204-evidence.txt
printf "bisect_culprit=%s\n" "$CULPRIT" >> ../INC-204-evidence.txt
git show --no-patch --format='%H %P %s' "$CULPRIT" | tee -a ../INC-204-evidence.txt
git diff "$CULPRIT^" "$CULPRIT" -- service.conf | tee -a ../INC-204-evidence.txt
git bisect reset

Verify prediction 1: bisect moved checkout state and created temporary bisect refs, but existing commits were unchanged. Expected culprit: C7.

7. Corroborate the bad value transition with pickaxe

git log -S'900' --format='%H %s' -- service.conf | tee -a ../INC-204-evidence.txt
git show "$C7" -- service.conf

Verify prediction 3: because the literal 900 appears at C7 and remains present through C9's whitespace-only formatting, -S'900' should identify C7 as the count-changing introduction.

9. Default blame points to the later textual modifier

git blame -L 1,1 -- service.conf
git show "$C9" -- service.conf

Verify prediction 2: the current first line was mechanically changed at C9, so default blame may identify C9 even though automated bisect isolated behavioral failure at C7.

10. Ignore the known formatting revision and compare attribution

printf "%s\n" "$C9" > .git-blame-ignore-revs
git blame --ignore-revs-file .git-blame-ignore-revs -L 1,1 -- service.conf | tee -a ../INC-204-evidence.txt
git blame --ignore-rev "$C9" -L 1,1 -- service.conf

When reattribution succeeds, the timeout line points back to C7. This supports the distinction between last textual modifier and behavioral boundary identified by the oracle.

11. Add a support-line fix and inspect the relevant merge base

git switch release/stable
printf "support_patch=enabled\n" >> service.conf
git add service.conf
git commit -m "R1: stable support patch"
R1=$(git rev-parse HEAD)

git switch trunk
BASE=$(git merge-base trunk release/stable)
printf "merge_base=%s\n" "$BASE" | tee -a ../INC-204-evidence.txt
git show --no-patch --oneline "$BASE"

git merge-base --is-ancestor "$C7" release/stable
echo "C7 ancestor of stable? exit=$?"

git diff "$BASE" trunk -- service.conf
git diff "$BASE" release/stable -- service.conf

Verify prediction 4: the merge base is C5 and C7 is not an ancestor of the stable branch. The stable line may need an independent fix only if its own behavior reproduces the problem; ancestry alone says it did not inherit C7.

12. Complete an incident-style evidence report

INC-204 — Repository Forensics Summary

Repository state:
- Full/non-shallow history verified: ...
- Known good OID: ...
- Known bad OID: ...

Behavioral evidence:
- Oracle definition: timeout_ms <= 300 is good.
- Endpoint results: good=0, bad=1.
- git bisect run first bad: C7 OID ...

Text/history evidence:
- git log -S'900' identifies the commit where literal 900 count appears.
- git log -G... lists patches that modify timeout assignment lines.
- Default blame at HEAD identifies C9 because C9 reformatted the line.
- Blame ignoring approved formatting C9 reattributes the line toward C7.

Ancestry evidence:
- merge-base(trunk, release/stable) = C5.
- C7 is not an ancestor of release/stable.

What this proves:
- Under the deterministic oracle and searched ancestry, C7 is the first bad commit.
- Literal 900 was introduced at C7.
- C9 was a later textual formatting change.
- The stable branch diverged before C7.

What this does NOT prove:
- Who intended the regression.
- Who should be held responsible.
- Whether C7 is the only contributing operational cause outside Git.
- Whether external configuration/deployment systems changed runtime behavior.

13. Verification checklist

  • Repository is not shallow and all required refs/OIDs were captured.
  • Oracle returns 0 at C1 and non-zero bad status at C12.
  • Automated bisect reports C7 as first bad and bisect log is preserved.
  • git diff C7^ C7 shows the timeout value transition.
  • -S'900' independently selects the value-introduction commit.
  • -G shows timeout-line patches including later textual changes.
  • Default blame demonstrates why last modifier can be later than the regression.
  • Ignore-revision blame is documented as adjusted attribution, not deletion of C9.
  • Support-line merge base is C5 and C7 is not its ancestor.
  • Evidence report explicitly separates proof from inference/non-proof.

14. Cleanup

Copy the external incident report somewhere durable if you want to retain it.
git bisect reset 2>/dev/null || true
cd ../..
pwd
rm -rf git-forensics-checkpoint

PowerShell equivalent: git bisect reset 2>$null; Set-Location ../..; Remove-Item -Recurse -Force git-forensics-checkpoint.

15. Knowledge check

Question 1. Why is C7 stronger regression evidence than the default C9 blame result?

Question 2. Why does -S'900' not necessarily select C9 in this checkpoint?

Question 3. What does the C5 merge base prove?

Question 4. Does the evidence report prove human fault?

Question 5. What would invalidate the checkpoint before bisect even starts?

16. What Chapter 16 adds to a production Git operating model

You can now isolate deterministic regressions with an auditable oracle, distinguish line attribution from causation, choose the correct pickaxe semantics, reason from merge bases rather than dates, detect incomplete-history risks, and write incident evidence that clearly separates proof from inference.

17. Chapter checkpoint summary

Good repository forensics is triangulation: behavioral evidence from bisect, textual evidence from pickaxe, line-origin evidence from blame, topology evidence from merge bases, and independent operational evidence outside Git. No single command should be overinterpreted.

Next chapter

Hooks, Aliases, Templates, Attributes, and Local Automation

Chapter 17 moves from read-only forensic tooling to controlled local automation—where trust boundaries matter because Git can invoke executable hooks and configuration-driven behavior.

Authoritative references

 git-bisect
 git-blame
 git-log
 git-merge-base

Keep the academy open

Support free, practical DevOps education.

Every lesson is designed to remain readable in a browser, downloadable from GitHub, and usable without a paid learning platform. Contributions help expand and maintain the curriculum.

Ethereum / ERC-20
0x716c4Ab160C4B66F31a28AE2448BfF68fc3a2ef0 Send only Ethereum/ERC-20 compatible assets to this address.