Troubleshooting Imports, Scope, Timing, Encoding, and Flaky Automation: Configuration, Design Patterns, and Trade-Offs

1. Decision model: diagnostic power must match the suspected layer

More logging, more retries, and a larger rerun are not automatically better diagnostics. Each tool has a cost and blind spot. Dry-run is safe and fast for model/import/keyword/signature problems but intentionally skips test-library keyword execution and does not validate variables. TRACE can expose detailed messages but may increase noise and sensitive-data risk. A CI rerun reproduces the runner environment but can destroy the exact first-failure state if artifacts are overwritten.

The design goal is minimum sufficient instrumentation: collect enough evidence to distinguish competing root causes, then stop increasing diagnostic surface.

2. Troubleshooting trade-off matrix

3. Dry-run versus full execution

Use dry-run first when the console says “resource not found,” “keyword not found,” “expected N arguments,” or the suite appears malformed. Its safety advantage is important: library keywords are not executed. This makes it appropriate before touching a target system when you only need model validation.

python -m robot --dryrun --test "Import incident" -d results/dry tests/incidents.robot

Switch to a real targeted run when your hypothesis involves variable provenance, library instance state, returned values, external timing, process/network state, encoding performed by a Python library, or teardown. Document why the transition from dry-run to execution is necessary.

4. TRACE and debug files versus privacy/noise

Robot's default log threshold is INFO. DEBUG and TRACE can reveal more detail, and --debugfile writes execution events/messages to a plain-text file. These are powerful but should be scoped to the smallest reproducible slice because arguments and library messages may contain paths, payloads, IDs, or secrets in real projects.

python -m robot --test "Encoding incident" --loglevel TRACE:INFO --debugfile debug.txt -d results/encoding-trace tests/incident_encoding.robot

TRACE:INFO means the execution retains TRACE-level messages while the HTML log initially displays INFO and above. In real environments, that richer retained data can itself be sensitive. Synthetic course values make the mandatory lab safe, but production teams should define who can enable deep traces, where they are stored, and how long they are retained.

5. Explicit condition wait versus retry

Three mechanisms are often conflated:

If a file is created asynchronously, poll file existence. If an API returns a job state, poll that state with a bounded budget. If a browser element becomes interactable, use the browser library's explicit wait. Retrying an entire test repeats setup, mutations, and external side effects and can create a false pass if the first failure is discarded.

6. Project-root path discipline versus cwd hacks

Robot exposes several different path concepts: ${CURDIR} is the directory containing the current data file; ${EXECDIR} is the directory where execution started; the OS cwd belongs to the process; --pythonpath/PYTHONPATH affect Python's module search path. These mechanisms are not interchangeable.

# Resource anchored to the importing file:
Resource    ../resources/common.resource

# Explicit custom-library search path in a project command:
python -m robot --pythonpath libraries -d results tests

# Avoid relying on "cd some/magic/folder" as the only reason imports work.

A maintainable project chooses a canonical execution command, package/layout boundary, and explicit resource paths. If CI requires a different cwd, the test project should still resolve its own resources deterministically or the command should set the intended project root explicitly.

7. UTF-8 normalization versus platform defaults

Text files in the Robot project that contain non-ASCII characters must be UTF-8. Custom Python code should also specify encodings for files instead of inheriting locale defaults. This matters on Windows, Linux containers, and CI runners where default encodings can differ.

# Deterministic text boundary
text = Path(path).read_text(encoding="utf-8")
Path(out).write_text(text, encoding="utf-8")

# Protocol exception example: only use another codec when the protocol/file format says so.
legacy = Path(path).read_text(encoding="iso-8859-1")

If the external system provides bytes, record the contract: HTTP charset, file specification, process encoding, database driver behavior, or binary format. Do not infer a codec from what “looks readable” on one workstation.

8. Smallest failing subset versus whole-suite rerun

Targeted execution improves signal-to-noise and reduces accidental state mutation. Start with one test or suite when the failure is independent:

python -m robot --test "Scope incident" -d results/repro tests/incident_scope.robot

Expand the selection only if evidence supports an interaction hypothesis: suite setup order, shared library scope, shared test data, Pabot collision, or CI sharding. A test that fails only after another test is valuable evidence about hidden coupling; do not “fix” it by permanently changing execution order.

9. Local reproduction versus CI-only instrumentation

Local reproduction is preferable when you can match Python/Robot/library versions, variables, fixture data, working directory, and execution command. But some failures genuinely depend on CI/container state: filesystem permissions, service DNS, environment-variable injection, ephemeral ports, locale, CPU pressure, or parallel workers.

10. Keep configuration ownership explicit

11. Worked decision: CI-only “resource not found”

Suppose local execution succeeds but CI says a resource file cannot be found. Do not start with a retry or install everything again. First preserve the CI error and compare repository path/case, checked-out files, exact importing file, and command. On case-sensitive Linux, Common.resource and common.resource differ even if a Windows workstation previously hid the mistake. If the resource path is correctly project-relative, changing PYTHONPATH is not the appropriate repair.

If the missing item is instead a Python test library, compare the actual Python interpreter, installed package, and explicit search path. Same red symptom category—different owner and evidence.

Knowledge check

Summary and bridge

Troubleshooting quality comes from matching diagnostic depth to the suspected layer. Lesson 4 now applies the same model to production anti-patterns: sleeps, arbitrary search paths, globalized variables, silent decoding, broad catches/retries, evidence deletion, environment reinstallations, and parallel-state collisions.

Further reading

• Robot Framework User Guide — current syntax, imports, variable priority/scope, dry-run, log levels, debug file, search paths, output artifacts, and execution semantics.
• BuiltIn library, OperatingSystem library, and Process library — assertions, variable inspection, filesystem checks, and local process control used by the labs.
• Robot Framework releases and Robot Framework on PyPI — verify the stable/pre-release boundary before reproducing incident behavior.
• Pabot documentation and Pabot releases — parallel worker behavior and evidence when diagnosing contention or collisions.
• Python codecs documentation — text/bytes conversion and explicit encoding behavior at custom-library boundaries.

Troubleshooting commands are intentionally conservative and local. Re-check primary documentation when Robot Framework, Python, external libraries, Pabot, containers, or CI runners change.