Chapter 16Lesson 04~120 minutes

Bisect, Blame, Pickaxe Search, Merge Bases, and Repository Forensics: Diagnostics, Failure Modes, Security, and Performance

Diagnose misleading forensic conclusions caused by reversed or flaky test oracles, skip misuse, overinterpreted blame, -S/-G confusion, multiple merge bases, shallow history, and post-rewrite evidence loss.

DiagnosticsBad oracleCriss-crossIncomplete history

Learning objectives

  • Prove an inverted bisect oracle from endpoint exit codes and repair its classification contract.
  • Explain how skip clusters and flaky tests weaken first-bad conclusions.
  • Triangulate blame results with pickaxe and exact commit diffs.
  • Engineer a criss-cross history and expose multiple best merge bases with --all.
  • Detect shallow/incomplete history before making forensic claims.

1. Diagnostic sequence — preserve evidence before changing the theory

  1. Preserve evidence: exact OIDs, refs, history completeness, oracle code/output, bisect log, blame/pickaxe commands, and merge-base results.
  2. Inspect status, refs, history, and config.
  3. Identify the affected layer: behavioral oracle, line attribution, diff search, ancestry topology, or incomplete clone.
  4. Choose the least destructive correction: rerun/requery/deepen history rather than rewrite.
  5. Verify with an independent evidence source.

2. Intentionally broken example — the oracle's exit meaning is reversed

Suppose the regression is timeout_ms=900, but the “test” is:

grep -q '^timeout_ms=900$' service.conf
echo $?

On the bad revision, grep finds the line and exits 0. Bisect interprets 0 as good. On the good revision, grep does not find it and exits 1. Bisect interprets 1 as bad. Every classification is inverted.

Repair the oracle

if grep -q '^timeout_ms=900$' service.conf; then
  exit 1
else
  exit 0
fi

Validate the repaired oracle manually on both known endpoints before rerunning bisect. The original problem was not Git's search algorithm—it was the contract between the test and bisect.

3. Failure mode — a flaky oracle violates the good/bad model

If the same commit sometimes passes and sometimes fails because of network timing, random seeds, shared services, or clock-dependent state, binary search can converge on a misleading boundary. Re-run endpoints multiple times, isolate dependencies, fix the seed, and capture environment versions.

4. Failure mode — treating “uncertain” as skip

git bisect skip or exit 125 means the current revision cannot be classified by the oracle. It is not a neutral substitute for thinking. If skipped commits cluster around the transition, Git may only report that the first bad commit could be one of several revisions.

Repair by making old revisions testable where safe, changing the test to a compatible invariant, or documenting the unresolved candidate set instead of declaring one culprit.

5. Failure mode — blame is interpreted as original design responsibility

A line may have been moved, reformatted, mechanically rewritten, cherry-picked, or generated. Default blame answers which commit last modified the line under its analysis. It does not identify the architect, reviewer, requester, or original introducer of the idea.

git blame -L 20,30 -- app.py
git show --stat <blamed-oid>
git log -S'important_symbol' --oneline -- app.py
git log --follow --oneline -- app.py

Use the blamed commit as a starting point, then triangulate with pickaxe, exact diffs, surrounding commits, and external review/incident records.

6. Failure mode — -S and -G are used interchangeably

If a commit changes:

- timeout_ms=100
+ timeout_ms=900

-G'^timeout_ms=' selects the commit because matching patch lines changed. -S'timeout_ms' may not because the token count stays one. Conversely -S'900' is well-suited to the question “when did literal 900 appear?”

7. Engineer a criss-cross topology with two best merge bases

This disposable graph demonstrates why a single merge base is not universal.

git init -b trunk criss-cross-lab
cd criss-cross-lab
git config user.name "Merge Base Lab"
git config user.email "merge-base@example.invalid"

printf "base\n" > base.txt
git add base.txt
git commit -m "Base"
BASE=$(git rev-parse HEAD)

git switch -c left
printf "left\n" > left.txt
git add left.txt
git commit -m "Left"
L=$(git rev-parse HEAD)

git switch -c right "$BASE"
printf "right\n" > right.txt
git add right.txt
git commit -m "Right"
R=$(git rev-parse HEAD)

git switch left
git merge --no-ff "$R" -m "Left merges original right"
M1=$(git rev-parse HEAD)

git switch right
git merge --no-ff "$L" -m "Right merges original left"
M2=$(git rev-parse HEAD)

git merge-base --all left right

Both original side commits L and R are best common ancestors; neither is an ancestor of the other. Without --all, Git may output one unspecified best merge base. Diagnostic tooling that assumes there is always exactly one base is incomplete.

8. Failure mode — forensic conclusions from a shallow clone

git rev-parse --is-shallow-repository
git rev-list --count HEAD
git log --oneline --decorate --all --max-count=30

A shallow boundary can hide the known-good commit, older symbol transitions, blame origins, or a real merge base. Before declaring “no earlier commit exists” or “these branches have no common ancestor,” obtain the required history with deepen/unshallow/fetch under your repository policy.

9. Failure mode — investigating after history rewrite without preserving old IDs

Rebase or filtering can replace the commits an earlier incident referred to. If rewrite is unavoidable, preserve an evidence mapping/bundle/archived clone according to policy. Never rewrite first and ask “what happened?” later.

10. Security relevance — forensic output can expose sensitive historical data

Pickaxe and diffs can rediscover historical secrets, private URLs, account names, or removed configuration. Treat forensic logs as sensitive evidence. If a credential leak is discovered, revoke/rotate the credential first; locating or rewriting the commit is not credential remediation by itself.

11. Performance relevance — the test and search scope dominate cost

Bisect reduces candidate revisions roughly by repeated halving, but if each build/test takes 30 minutes, the oracle dominates elapsed time. Pickaxe and blame can scan substantial history, especially across many refs or move/copy detection. Narrow refs/path ranges after proving the scope is correct, and cache build dependencies outside the Git object model when safe.

12. Red-zone response — do not destroy evidence to “clean up” the incident

Do not expire reflogs, prune unreachable objects, force-update refs, rewrite history, delete branches/tags, or aggressively garbage-collect while evidence is still being established. Those operations can remove or rename the very objects the investigation needs.

13. Symptom → likely cause → verification

Symptom Likely cause Verification
Bisect names obviously unrelated commit Bad/flaky oracle or wrong endpoints Test both endpoints repeatedly; inspect bisect log
Several commits remain possible Skipped revisions around boundary Review skips; improve testability
Blame points at formatter Last textual modifier Ignore approved format rev + inspect pickaxe/diffs
-S misses a line replacement Occurrence count unchanged Use matching -G query
Merge-base differs across tools/scripts Multiple best bases or incomplete graph merge-base --all + verify clone completeness
No old evidence visible Shallow/incomplete/re-written history Inspect shallow state/refs and archival sources

14. Knowledge check

Question 1. A grep command exits 0 exactly when the bug is present. Why is that wrong for bisect run?

Question 2. When is exit 125 appropriate?

Question 3. Why can a criss-cross history have multiple merge bases?

Question 4. Why can a shallow clone invalidate a pickaxe or merge-base conclusion?

Question 5. What is the safest response when a forensic query looks wrong?

15. Criss-cross lab cleanup

Confirm you are outside the disposable repository before deleting it.
cd ..
rm -rf criss-cross-lab

PowerShell equivalent: Set-Location ..; Remove-Item -Recurse -Force criss-cross-lab.

16. Summary

Forensic tools can be perfectly consistent and still support the wrong conclusion when the oracle, graph, search semantics, or clone completeness is misunderstood. Diagnose the evidence contract first: test classification, line attribution, pickaxe mode, merge-base multiplicity, and history availability.

Next

Run a production-style regression checkpoint and write an evidence report

Lesson 5 seeds a regression, automates bisect, corroborates it with pickaxe and ignore-revision blame, checks a support-line merge base, and records exactly what each tool proves and does not prove.

Authoritative references

 git-bisect
 git-blame
 gitdiffcore
 git-merge-base

Keep the academy open

Support free, practical DevOps education.

Every lesson is designed to remain readable in a browser, downloadable from GitHub, and usable without a paid learning platform. Contributions help expand and maintain the curriculum.

Ethereum / ERC-20
0x716c4Ab160C4B66F31a28AE2448BfF68fc3a2ef0 Send only Ethereum/ERC-20 compatible assets to this address.