Troubleshooting Imports, Scope, Timing, Encoding, and Flaky Automation: Diagnostics, Failure Modes, and Production Practices

1. Diagnostic sequence: preserve first, then move outward

flowchart TD
A[Preserve first-failure artifacts] --> B[Record Python, Robot, library, Pabot versions]
B --> C[Confirm command, selection, environment, data]
C --> D[Validate parse and import graph]
D --> E[Inspect variable scope and keyword resolution]
E --> F[Inspect library and external state]
F --> G[Inspect timing, parallel, CI, container state]
G --> H[Apply least destructive correction]
H --> I[Rerun smallest controlled slice]
I --> J[Expand only after root cause is supported]

This ordering makes deterministic, cheap checks precede expensive environment changes. It also prevents a passing retry from erasing the question “why did the first attempt fail?” The sequence is a default, not a rigid law: if a CI worker clearly ran out of disk, inspect that evidence immediately. But do not skip inward layers merely because “flaky usually means timing.”

2. Anti-pattern: adding sleeps to timing races

A fixed sleep changes elapsed time, not readiness semantics. It can make a race rarer while increasing every run's latency. Worse, when the system becomes slower, the same race returns.

# Symptom patch
Sleep    3 s
Element Should Be Visible    ${locator}

# Design intent
Wait Until Element Is Visible    ${locator}    timeout=10 s
Element Should Be Visible         ${locator}

The exact wait keyword depends on the external library. The production principle is library-neutral: observe the condition that proves readiness, bound the wait, and retain the assertion that proves the business outcome.

3. Anti-pattern: arbitrary PYTHONPATH accumulation

When an import fails, teams sometimes append the repository root, parent, grandparent, home directory, and tool directories until Python can find something with the requested name. This can make a different module shadow the intended one and creates a local/CI mismatch.

# Avoid folklore like:
PYTHONPATH=.:..:../..:/opt/random-libs:$PYTHONPATH

# Prefer an explicit project command when a search path is truly required:
python -m robot --pythonpath libraries -d results tests

For Robot resource files, first verify the path relative to the importing file. For Python libraries, verify interpreter ownership, installed package/module, and intentional search path. The repair should reduce ambiguity, not increase it.

4. Anti-pattern: globalizing a missing variable

A missing or unexpected variable often reflects a provenance or lifecycle defect. Set Global Variable can hide it by changing scope for everything that executes later. This creates order dependence and makes parallel execution harder.

# Risky incident patch
Set Global Variable    ${SESSION_ID}    ${value}

# Prefer explicit data flow when ownership is local
${session_id}=    Create Disposable Session
Use Session    ${session_id}

If configuration is genuinely global, inject it explicitly at execution start and document its priority. If mutable runtime state belongs to one test, keep it local/test-scoped or in a test-scoped library instance.

5. Anti-pattern: decoding with ignore or replace to “stop errors”

# Dangerous unless lossy decoding is explicitly required:
text = raw.decode("utf-8", errors="ignore")

# Evidence-preserving failure:
text = raw.decode("utf-8")

Strict decoding makes the mismatch visible. If bytes are corrupt or use another codec, determine the source contract. Silent loss can turn “customer name differs” into a false pass because the distinguishing character was discarded.

6. Anti-pattern: blanket catch/retry

Broad TRY/EXCEPT, Run Keyword And Ignore Error, repeated Wait Until Keyword Succeeds, or test-level reruns can each be legitimate in narrow designs. They become dangerous when applied to unknown failures because they collapse different semantics into “try again.”

7. Anti-pattern: deleting or overwriting first-failure outputs

Robot result artifacts are evidence. If a cleanup job removes output.xml, log, screenshots, process output, or worker artifacts before diagnosis, a later passing rerun cannot reconstruct the initial state. Use distinct output directories or timestamped incident IDs.

python -m robot -d results/incident-001-fail tests/incidents.robot
# after repair:
python -m robot -d results/incident-001-repaired --test "Import incident" tests/incidents.robot

Generated log.html and report.html are presentations of result data. Fixing text in a generated report does not repair the source suite, library, or external state. Always repair the owning source/configuration layer and regenerate results.

8. Anti-pattern: reinstalling everything before recording versions

“Delete the venv and reinstall latest” destroys a critical comparison point. Before dependency changes, record Python, Robot Framework, external library, Pabot, browser/runtime, container image, and lockfile state. If the reinstall fixes the incident, you need a diff to know which dependency changed.

python --version
python -m robot --version
python -m pip freeze > results/incident-001-freeze.txt

Do not publish a freeze file that contains private package indexes or credentials. Course labs use public packages only.

9. “Flaky means external” is a false assumption

Intermittence can originate inside the automation estate: unordered data, mutable globals, stale library instances, nondeterministic test selection, file collisions, random ports, locale/encoding defaults, filesystem case sensitivity, or teardown races. External systems are only one category.

10. Intentionally broken example: parallel-only file collision

Consider two Pabot workers executing tests that both write the same filename:

*** Settings ***
Library    OperatingSystem

*** Test Cases ***
Worker A
    Create File    ${EXECDIR}/state/result.txt    A
    ${text}=    Get File    ${EXECDIR}/state/result.txt
    Should Be Equal    ${text}    A

Worker B
    Create File    ${EXECDIR}/state/result.txt    B
    ${text}=    Get File    ${EXECDIR}/state/result.txt
    Should Be Equal    ${text}    B

Serial execution may pass. Parallel execution can fail depending on interleaving. Increasing sleeps only changes which worker wins. The root cause is shared mutable artifact identity.

Repair by giving each worker/test a unique disposable path. Pabot exposes worker-related variables/options in its current documentation; alternatively derive uniqueness from test name plus a generated run ID in a safe local helper. The key is to eliminate shared ownership rather than serialize all workers with a lock unless sharing is genuinely required.

11. Write a root-cause statement that can be reviewed

A useful statement names four things:

1. Layer: where the first incorrect state occurred.
2. Trigger: the condition that exposed it.
3. Evidence: the artifact or observation supporting the claim.
4. Repair/proof: the minimal change and targeted rerun demonstrating correction.

Example: “Under two Pabot workers, both tests wrote state/result.txt; worker artifacts showed cross-overwritten content; unique per-test paths removed the collision and the same parallel selection passed repeatedly with unchanged assertions.”

12. Production troubleshooting practices

• Preserve failing and repaired artifacts separately.
• Record exact commands and version/environment context before upgrades.
• Use allowlisted environment diagnostics rather than dumping all secrets.
• Keep deep debug/TRACE instrumentation time-bounded and access-controlled.
• Never disable TLS/SSH verification, authorization, MFA, or anti-abuse controls to “rule them out.”
• Never reproduce incidents against uncontrolled production targets. Use authorized staging or disposable local fixtures.
• Expand from one failing test to the full suite only after the root-cause hypothesis requires broader interaction.

Knowledge check

Summary and bridge

Production troubleshooting rejects symptom patches in favor of layer ownership, preserved evidence, and minimal verified repairs. Lesson 5 combines the chapter into a checkpoint: four incidents, predictions, root-cause statements, smallest-slice reruns, an evidence packet, and a reusable incident checklist ready for the governed platform capstone.

Further reading

• Robot Framework User Guide — current syntax, imports, variable priority/scope, dry-run, log levels, debug file, search paths, output artifacts, and execution semantics.
• BuiltIn library, OperatingSystem library, and Process library — assertions, variable inspection, filesystem checks, and local process control used by the labs.
• Robot Framework releases and Robot Framework on PyPI — verify the stable/pre-release boundary before reproducing incident behavior.
• Pabot documentation and Pabot releases — parallel worker behavior and evidence when diagnosing contention or collisions.
• Python codecs documentation — text/bytes conversion and explicit encoding behavior at custom-library boundaries.

Troubleshooting commands are intentionally conservative and local. Re-check primary documentation when Robot Framework, Python, external libraries, Pabot, containers, or CI runners change.