Checkpoint Lab — Troubleshooting Imports, Scope, Timing, Encoding, and Flaky Automation

1. Checkpoint scenario

You inherit the incident-lab project from Lesson 2, but this time you are given only four failing tests and their raw result directories. Your task is not to make everything green as fast as possible. You must identify the first failing layer, preserve evidence, write a root-cause statement, apply the least invasive repair, rerun the smallest affected slice, and document why the repair is correct.

2. Evidence architecture

flowchart TD
F[Failing source and environment] --> R[Raw first-failure results]
R --> H[Layer hypothesis]
H --> P[Prediction before repair]
P --> C[Minimal source or configuration change]
C --> T[Smallest targeted rerun]
T --> V[Independent verification]
V --> S[Root-cause statement]
S --> K[Reusable incident checklist]

Raw first-failure outputs are immutable evidence for the exercise. Repaired runs go to new directories. Your written hypothesis must exist before the repair so a passing result cannot retroactively justify an unrelated change.

3. Preflight

1. Confirm you are inside the disposable incident-lab.
2. Record Python and Robot Framework versions.
3. Record the exact failing test path/name and command.
4. Confirm no real credentials, production URLs, or external systems are referenced.
5. Create a new results/checkpoint/ directory; do not overwrite earlier evidence.

python --version
python -m robot --version
python -m pip show robotframework

# Optional only if Pabot is installed for your environment:
pabot --version

The mandatory checkpoint does not require Pabot. If you add a parallel extension, pin/record Pabot 5.2.2 and give workers isolated mutable files/ports/data.

4. Required predictions before execution

Write at least two predictions for each incident. Use this format:

After each rerun, mark each prediction as confirmed, rejected, or inconclusive. If rejected, revise the hypothesis before making another change.

5. Incident 29-A — import graph

Evidence-first order

1. Read the raw console/import error.
2. Inspect only the importing file and referenced relative path.
3. Run a targeted dry-run into a new directory.
4. Do not change cwd, PYTHONPATH, or dependencies unless evidence points there.
5. Repair the filename/path and rerun the identical dry-run.

python -m robot --dryrun --test "Import incident" -d results/checkpoint/29-A-before tests/incident_import.robot

# after the one-line resource repair
python -m robot --dryrun --test "Import incident" -d results/checkpoint/29-A-after tests/incident_import.robot

Independent verification: the intended resource exists at the resolved project-relative location and the repaired source points to that exact file. Root cause must name the import graph, not “Robot sometimes loses files.”

6. Incident 29-B — variable provenance

1. List all definitions of ${MODE} in the relevant suite/resource/variable files and command.
2. Use dry-run only to exclude structural issues; remember it does not validate variables.
3. Run the single scope test and preserve the assertion/log showing the winning value.
4. Remove or rename the unintended higher-priority owner rather than introducing a global setter.
5. Run the same single test again.

python -m robot --test "Scope incident" -d results/checkpoint/29-B-before tests/incident_scope.robot

python -m robot --test "Scope incident" -d results/checkpoint/29-B-after tests/incident_scope.robot

Independent verification: run once with --variable MODE:cli to prove command-line priority deliberately, then remove that override and confirm the project default again. This is evidence of precedence, not a production recommendation to inject arbitrary values.

7. Incident 29-C — timing race

1. Confirm the child writer is local and writes only the guarded state/ready.txt.
2. Preserve the failure showing the marker was missing at assertion time.
3. Record writer delay and timeout budget.
4. Replace immediate/fixed-sleep synchronization with condition polling.
5. Retain a content assertion after readiness.

python -m robot --test "Timing incident" -d results/checkpoint/29-C-before tests/incident_timing.robot

python -m robot --test "Timing incident" -d results/checkpoint/29-C-after tests/incident_timing.robot

Stress verification: within the disposable lab, change the synthetic writer delay to a value still below the timeout and confirm the test remains correct without editing the wait. Then set it above the timeout and confirm the test fails with a bounded readiness error. Restore the original delay afterward.

8. Incident 29-D — encoding boundary

1. Preserve the UTF-8 fixture unchanged.
2. Identify the decoder in the custom library and its explicit codec.
3. Run only the encoding test; enable a debug file only if needed and only for synthetic content.
4. Change the boundary to explicit UTF-8; do not use ignore/replace.
5. Assert the full exact Unicode string.

python -m robot --test "Encoding incident" --debugfile debug.txt -d results/checkpoint/29-D-before tests/incident_encoding.robot

python -m robot --test "Encoding incident" -d results/checkpoint/29-D-after tests/incident_encoding.robot

Independent verification: inspect the file with a Python snippet that reports the raw bytes and UTF-8-decoded text without changing the file:

from pathlib import Path
p = Path("state/unicode.txt")
raw = p.read_bytes()
print(raw)
print(raw.decode("utf-8"))

9. Root-cause statement rubric

Each incident statement must contain:

• First incorrect layer, not only the final failed assertion.
• Triggering condition such as wrong filename, priority source, readiness ordering, or codec mismatch.
• Evidence naming the exact artifact/observation.
• Minimal repair and why it addresses ownership rather than the symptom.
• Proof from the smallest repaired rerun.

10. Produce the reusable Robot Framework incident checklist

1. Preserve: copy/link raw first-failure output, console, screenshots/process/library evidence before rerun.
2. Version: record Python, Robot, external libraries, Pabot, browser/runtime, container/runner where relevant.
3. Execution identity: exact command, selected tests/tags, cwd, ${EXECDIR}, environment mode, synthetic/authorized target.
4. Parse/import: use dry-run when structural; verify resource path relative to importer and Python-library search path separately.
5. Scope: list variable definitions and priorities; avoid hidden suite/global mutation.
6. Keyword/library: confirm keyword namespace/signature/version and library instance lifecycle.
7. External state: inspect the target condition/state before assuming timing.
8. Timing: wait for explicit conditions with bounded timeouts; reject giant sleeps.
9. Encoding: preserve bytes/text boundary and explicit codec; reject lossy ignore/replace by default.
10. Parallel/CI/container: record worker/job IDs, resource isolation, paths, permissions, DNS/ports, and capacity.
11. Repair: change the narrowest owning layer; do not delete evidence or reinstall blindly.
12. Verify: rerun the smallest slice, then expand only if the root cause requires interaction coverage.

11. Required evidence packet

Do not include real secrets, customer payloads, Authorization headers, private keys, or personal data. Production implementations need redaction and access-controlled retention; this checkpoint intentionally uses synthetic content.

12. Final verification and cleanup

• Each of the four incidents has one supported root cause, not merely a passing rerun.
• At least two pre-repair predictions per incident are marked confirmed/rejected/inconclusive.
• No repair added fixed sleeps, blanket retries, broad catches, global variables, arbitrary search paths, silent decoding, disabled security controls, or production targets.
• Failing and repaired artifacts are stored separately.
• All local child processes have exited and state/ready.txt can be safely removed.
• The UTF-8 source fixture is either retained as the known test fixture or removed only as part of the lab directory cleanup.

Knowledge check

Summary and bridge to Chapter 30

Chapter 29 adds a repeatable incident operating model: preserve first-failure evidence, identify the first incorrect layer, predict before changing, repair the narrowest owner, and prove the result with the smallest controlled rerun. In Chapter 30, these practices become governance controls for an end-to-end Robot Framework automation platform: reproducible environments, explicit ownership, secret-safe evidence, parallel isolation, CI artifact policy, static/style gates, performance budgets, and incident-ready diagnostics.

Further reading

• Robot Framework User Guide — current syntax, imports, variable priority/scope, dry-run, log levels, debug file, search paths, output artifacts, and execution semantics.
• BuiltIn library, OperatingSystem library, and Process library — assertions, variable inspection, filesystem checks, and local process control used by the labs.
• Robot Framework releases and Robot Framework on PyPI — verify the stable/pre-release boundary before reproducing incident behavior.
• Pabot documentation and Pabot releases — parallel worker behavior and evidence when diagnosing contention or collisions.
• Python codecs documentation — text/bytes conversion and explicit encoding behavior at custom-library boundaries.

Troubleshooting commands are intentionally conservative and local. Re-check primary documentation when Robot Framework, Python, external libraries, Pabot, containers, or CI runners change.