Troubleshooting Imports, Scope, Timing, Encoding, and Flaky Automation: Core Concepts and Mental Model

1. The practical problem: “flaky” is a symptom, not a root cause

By Chapter 29, a Robot Framework project can fail before a test starts, while resolving a variable, while converting arguments, inside a Python or external library, while waiting for a browser/API/process, during teardown, while Pabot workers share state, or only in a container/CI environment. These failures can look similar in a red pipeline, yet they demand different evidence and different repairs.

An evidence-first troubleshooter asks which layer first became inconsistent with the intended model? A missing resource file is not a timing problem. A command-line variable overriding a suite variable is not an import problem. A UTF-8 byte sequence decoded as ASCII is not “random data.” A Pabot file collision is not fixed by increasing a timeout. Treating all of these as generic flakiness creates folklore: more sleeps, broader retries, larger PYTHONPATH, global variables, and deleted outputs.

2. The troubleshooting ladder

flowchart TD
A[Source parse] --> B[Suite and import graph]
B --> C[Variable resolution and scope]
C --> D[Keyword resolution and argument conversion]
D --> E[Library or external action]
E --> F[Wait and timing]
F --> G[Assertion and status]
G --> H[Teardown]
H --> I[output.xml, log, debug]
I --> J[CI, container, or Pabot layer]
J -. environment evidence .-> A

Source parse asks whether Robot data is structurally valid. The suite/import graph asks whether resources, variable files, and libraries can be found and loaded. Variable resolution asks which definition won and in what scope. Keyword resolution and argument conversion asks which implementation Robot selected and whether inputs match its signature/types. Only after these layers are credible should you blame the external system or timing.

The outer layers matter too. A teardown can obscure the initial failure if it fails noisily. A CI job can execute from another directory, inject higher-priority variables, or use another Python interpreter. Pabot can expose collisions that serial execution never creates. The ladder is not a claim that every incident uses every layer; it is a disciplined order for eliminating cheaper, more deterministic causes first.

3. Map each layer to its evidence and corrective action

4. Dry-run is a structural diagnostic, not a universal validator

Robot Framework 7.4.2 supports --dryrun. In dry-run mode, keywords originating from test libraries are not executed. This makes it valuable for validating test data, unresolved imports, missing keywords, wrong argument counts, and invalid user-keyword syntax without touching the external target.

python -m robot --dryrun -d results/dry tests
python -m robot --dryrun --test "Import incident" -d results/import tests/incidents.robot

A crucial limitation is easy to miss: current Robot Framework documentation states that dry-run does not validate variables. A passing dry-run therefore does not prove that ${MODE} has the intended value, that a runtime-created variable exists on the executed branch, or that command-line variable precedence matches your expectation. Record this limitation in incident notes instead of turning “dry-run passed” into a false all-clear.

5. Imports: prefer deterministic project-relative resolution

For resource files, a relative path is first resolved relative to the directory of the importing file. If not found there, Robot can search Python's module search path. Variable-file path imports follow a similar rule. Python test libraries are resolved through Python import mechanics and Robot's configured module search path.

*** Settings ***
Resource     ../resources/common.resource
Variables    ../config/runtime_vars.py
Library      IncidentLab

This ordering is why “run it from a different current directory” is often a poor explanation for a correctly written Resource ../resources/common.resource import: the resource path is anchored to the importing file, not blindly to the shell cwd. By contrast, ad-hoc Python-library imports can be affected by interpreter/module-search configuration. If a custom library legitimately lives outside installed packages, use an explicit, documented project path such as --pythonpath libraries in the reproducible run command rather than appending random directories until the error disappears.

6. Variable provenance: name equality does not imply source equality

Robot variables have both priority and scope. A command-line variable can override a same-named variable declared in a suite file. Variables from imported resource/variable files are lower priority than variables declared in the importing test file. Runtime VAR or BuiltIn setters can create local, test, suite, or global values depending on the mechanism used.

*** Settings ***
Resource    ../resources/common.resource

*** Variables ***
${MODE}    suite-file

*** Test Cases ***
Provenance example
    Log    MODE before local override = ${MODE}
    VAR    ${MODE}    local-test
    Log    MODE after local override = ${MODE}

If the same suite is run with --variable MODE:cli, that pre-execution command-line variable has higher priority than the Variable-section value. Troubleshooting therefore requires a provenance statement such as “${MODE} came from CLI injection in this run,” not merely a screenshot showing its final text.

Globalizing a missing value is a dangerous shortcut because it changes ownership and lifetime. Prefer arguments, return values, and narrow local/test scope unless the data is genuinely suite/global configuration.

7. Timing failures: observe a condition, not elapsed hope

A timing race exists when correctness depends on an external or concurrent condition becoming true before an assertion executes. Fixed sleeps delay every run yet still fail when the system is slower than the guessed duration. The diagnostic evidence should include a start timestamp, the condition being awaited, when it actually became true, and the timeout budget.

# Fragile: time passes, but readiness is not observed.
Sleep    500 ms
File Should Exist    ${READY_FILE}

# Better design concept: poll the readiness condition until timeout.
Wait For Ready File    ${READY_FILE}    timeout=2.0    interval=0.05

The second keyword can be a small custom-library helper that uses a monotonic clock and checks a local disposable file. In browser/API libraries, use their explicit wait/polling primitives instead. Do not convert every failure into a retry: a missing import or wrong variable will never become correct merely because time passes.

8. Encoding: keep Unicode text and bytes as different states

Robot Framework test data is Unicode text. Files containing non-ASCII test data must be UTF-8. External protocols and filesystem APIs may expose bytes, so a custom Python library must state where encoding/decoding occurs. A byte sequence is not “weird text”; it is bytes until decoded with the correct codec.

from pathlib import Path

def read_utf8(path: str) -> str:
    return Path(path).read_text(encoding="utf-8")

def raw_bytes(path: str) -> bytes:
    return Path(path).read_bytes()

Never “fix” a Unicode error with errors="ignore" unless data loss is an explicit product requirement. That setting can silently remove characters and make later assertions pass against corrupted content. Preserve the original bytes or source file, record the expected codec, and make the conversion explicit.

9. Read-only incident preflight

python --version
python -m robot --version
python -m robot --help
python -m pip show robotframework

# Structural check with no test-library keyword execution:
python -m robot --dryrun -d results/dry tests

# Only when deeper execution tracing is justified:
python -m robot --loglevel TRACE --debugfile debug.txt -d results/trace tests/incidents.robot

Capture the command rather than reconstructing it from memory. TRACE and debug files can contain arguments, paths, messages, or data that are inappropriate for broad CI retention, so use them narrowly, with synthetic fixtures in this course, and delete or restrict them according to your evidence policy after the incident is resolved.

10. Why this matters in DevOps

Stable automation is infrastructure. Every folklore workaround adds hidden state: a new path, a longer timeout, a retry layer, a global variable, or a log suppression. Over time, those changes lengthen mean time to repair because no one knows which layer the workaround was intended to compensate for. A fixed diagnostic ladder creates comparable incident records, reviewable repairs, and a smaller operational surface.

Knowledge check

Summary and bridge

You now have a diagnostic ladder that starts with deterministic model/import/scope questions and moves outward to external timing, encoding, parallelism, containers, and CI. Lesson 2 turns that model into a four-incident local lab and records before/after evidence for each repair.

Further reading

• Robot Framework User Guide — current syntax, imports, variable priority/scope, dry-run, log levels, debug file, search paths, output artifacts, and execution semantics.
• BuiltIn library, OperatingSystem library, and Process library — assertions, variable inspection, filesystem checks, and local process control used by the labs.
• Robot Framework releases and Robot Framework on PyPI — verify the stable/pre-release boundary before reproducing incident behavior.
• Pabot documentation and Pabot releases — parallel worker behavior and evidence when diagnosing contention or collisions.
• Python codecs documentation — text/bytes conversion and explicit encoding behavior at custom-library boundaries.

Troubleshooting commands are intentionally conservative and local. Re-check primary documentation when Robot Framework, Python, external libraries, Pabot, containers, or CI runners change.