Bisect, Blame, Pickaxe Search, Merge Bases, and Repository Forensics: Diagnostics, Failure Modes, Security, and Performance
Diagnose misleading forensic conclusions caused by reversed or flaky test oracles, skip misuse, overinterpreted blame, -S/-G confusion, multiple merge bases, shallow history, and post-rewrite evidence loss.
Learning objectives
- Prove an inverted bisect oracle from endpoint exit codes and repair its classification contract.
- Explain how skip clusters and flaky tests weaken first-bad conclusions.
- Triangulate blame results with pickaxe and exact commit diffs.
- Engineer a criss-cross history and expose multiple best merge bases with --all.
- Detect shallow/incomplete history before making forensic claims.
1. Diagnostic sequence — preserve evidence before changing the theory
- Preserve evidence: exact OIDs, refs, history completeness, oracle code/output, bisect log, blame/pickaxe commands, and merge-base results.
- Inspect status, refs, history, and config.
- Identify the affected layer: behavioral oracle, line attribution, diff search, ancestry topology, or incomplete clone.
- Choose the least destructive correction: rerun/requery/deepen history rather than rewrite.
- Verify with an independent evidence source.
2. Intentionally broken example — the oracle's exit meaning is reversed
Suppose the regression is timeout_ms=900, but the
“test” is:
grep -q '^timeout_ms=900$' service.conf
echo $?
On the bad revision, grep finds the line and exits 0. Bisect interprets 0 as good. On the good revision, grep does not find it and exits 1. Bisect interprets 1 as bad. Every classification is inverted.
Repair the oracle
if grep -q '^timeout_ms=900$' service.conf; then
exit 1
else
exit 0
fi
Validate the repaired oracle manually on both known endpoints before rerunning bisect. The original problem was not Git's search algorithm—it was the contract between the test and bisect.
3. Failure mode — a flaky oracle violates the good/bad model
If the same commit sometimes passes and sometimes fails because of network timing, random seeds, shared services, or clock-dependent state, binary search can converge on a misleading boundary. Re-run endpoints multiple times, isolate dependencies, fix the seed, and capture environment versions.
4. Failure mode — treating “uncertain” as skip
git bisect skip or exit 125 means the current revision
cannot be classified by the oracle. It is not a neutral substitute
for thinking. If skipped commits cluster around the transition, Git
may only report that the first bad commit could be one of several
revisions.
Repair by making old revisions testable where safe, changing the test to a compatible invariant, or documenting the unresolved candidate set instead of declaring one culprit.
5. Failure mode — blame is interpreted as original design responsibility
A line may have been moved, reformatted, mechanically rewritten, cherry-picked, or generated. Default blame answers which commit last modified the line under its analysis. It does not identify the architect, reviewer, requester, or original introducer of the idea.
git blame -L 20,30 -- app.py
git show --stat <blamed-oid>
git log -S'important_symbol' --oneline -- app.py
git log --follow --oneline -- app.py
Use the blamed commit as a starting point, then triangulate with pickaxe, exact diffs, surrounding commits, and external review/incident records.
6. Failure mode — -S and -G are used
interchangeably
If a commit changes:
- timeout_ms=100
+ timeout_ms=900
-G'^timeout_ms=' selects the commit because matching
patch lines changed. -S'timeout_ms' may not because the
token count stays one. Conversely -S'900' is
well-suited to the question “when did literal 900 appear?”
7. Engineer a criss-cross topology with two best merge bases
This disposable graph demonstrates why a single merge base is not universal.
git init -b trunk criss-cross-lab
cd criss-cross-lab
git config user.name "Merge Base Lab"
git config user.email "merge-base@example.invalid"
printf "base\n" > base.txt
git add base.txt
git commit -m "Base"
BASE=$(git rev-parse HEAD)
git switch -c left
printf "left\n" > left.txt
git add left.txt
git commit -m "Left"
L=$(git rev-parse HEAD)
git switch -c right "$BASE"
printf "right\n" > right.txt
git add right.txt
git commit -m "Right"
R=$(git rev-parse HEAD)
git switch left
git merge --no-ff "$R" -m "Left merges original right"
M1=$(git rev-parse HEAD)
git switch right
git merge --no-ff "$L" -m "Right merges original left"
M2=$(git rev-parse HEAD)
git merge-base --all left right
Both original side commits L and R are
best common ancestors; neither is an ancestor of the other. Without
--all, Git may output one unspecified best merge base.
Diagnostic tooling that assumes there is always exactly one base is
incomplete.
8. Failure mode — forensic conclusions from a shallow clone
git rev-parse --is-shallow-repository
git rev-list --count HEAD
git log --oneline --decorate --all --max-count=30
A shallow boundary can hide the known-good commit, older symbol transitions, blame origins, or a real merge base. Before declaring “no earlier commit exists” or “these branches have no common ancestor,” obtain the required history with deepen/unshallow/fetch under your repository policy.
9. Failure mode — investigating after history rewrite without preserving old IDs
Rebase or filtering can replace the commits an earlier incident referred to. If rewrite is unavoidable, preserve an evidence mapping/bundle/archived clone according to policy. Never rewrite first and ask “what happened?” later.
10. Security relevance — forensic output can expose sensitive historical data
Pickaxe and diffs can rediscover historical secrets, private URLs, account names, or removed configuration. Treat forensic logs as sensitive evidence. If a credential leak is discovered, revoke/rotate the credential first; locating or rewriting the commit is not credential remediation by itself.
11. Performance relevance — the test and search scope dominate cost
Bisect reduces candidate revisions roughly by repeated halving, but if each build/test takes 30 minutes, the oracle dominates elapsed time. Pickaxe and blame can scan substantial history, especially across many refs or move/copy detection. Narrow refs/path ranges after proving the scope is correct, and cache build dependencies outside the Git object model when safe.
12. Red-zone response — do not destroy evidence to “clean up” the incident
13. Symptom → likely cause → verification
| Symptom | Likely cause | Verification |
|---|---|---|
| Bisect names obviously unrelated commit | Bad/flaky oracle or wrong endpoints | Test both endpoints repeatedly; inspect bisect log |
| Several commits remain possible | Skipped revisions around boundary | Review skips; improve testability |
| Blame points at formatter | Last textual modifier | Ignore approved format rev + inspect pickaxe/diffs |
-S misses a line replacement |
Occurrence count unchanged | Use matching -G query |
| Merge-base differs across tools/scripts | Multiple best bases or incomplete graph |
merge-base --all + verify clone completeness
|
| No old evidence visible | Shallow/incomplete/re-written history | Inspect shallow state/refs and archival sources |
14. Knowledge check
Question 1. A grep command exits 0 exactly when the bug is present. Why is that wrong for bisect run?
Question 2. When is exit 125 appropriate?
Question 3. Why can a criss-cross history have multiple merge bases?
Question 4. Why can a shallow clone invalidate a pickaxe or merge-base conclusion?
Question 5. What is the safest response when a forensic query looks wrong?
15. Criss-cross lab cleanup
cd ..
rm -rf criss-cross-lab
PowerShell equivalent:
Set-Location ..; Remove-Item -Recurse -Force
criss-cross-lab.
16. Summary
Forensic tools can be perfectly consistent and still support the wrong conclusion when the oracle, graph, search semantics, or clone completeness is misunderstood. Diagnose the evidence contract first: test classification, line attribution, pickaxe mode, merge-base multiplicity, and history availability.
Authoritative references
Keep the academy open
Support free, practical DevOps education.
Every lesson is designed to remain readable in a browser, downloadable from GitHub, and usable without a paid learning platform. Contributions help expand and maintain the curriculum.
0x716c4Ab160C4B66F31a28AE2448BfF68fc3a2ef0
Send only Ethereum/ERC-20 compatible assets to this
address.