Writing Python Test Libraries and the Library API: Diagnostics, Failure Modes, and Production Practices

1. Production diagnostic sequence: preserve evidence, then narrow the layer

1. Preserve first-failure artifacts: Robot output/log/report, Python traceback, unit-test output, Libdoc output, exact command, environment/version information.
2. Confirm versions/interpreter: python --version, python -m robot --version, library/package location.
3. Confirm executed path/selection/data: which suite/test called which keyword with which safe synthetic inputs.
4. Validate parse/import graph: module path, package __init__.py, explicit --pythonpath or installed version.
5. Inspect keyword discovery/signature: Libdoc list/show; decorator boundary; argument types/defaults.
6. Inspect Python object scope/state: TEST/SUITE/GLOBAL and any external clients/caches.
7. Inspect external state, timing, parallel/CI/container layers only when relevant.
8. Apply the least destructive correction and rerun the smallest unit/Robot slice into a new evidence directory.

2. Failure: helper methods become keywords accidentally

# BROKEN: no @library boundary, so both public methods can become keywords.
class BillingLibrary:
    def create_invoice(self, customer):
        return self.parse_internal_record(customer)

    def parse_internal_record(self, raw):
        ...

Symptoms: Libdoc contains implementation-only keywords; autocomplete suggests them; tests start depending on internals. Repair with @library plus explicit @keyword, or with ROBOT_AUTO_KEYWORDS=False. Renaming helpers with an underscore can help but is less explicit as a public API policy.

3. Failure: mutable GLOBAL scope makes test order part of the contract

@library(scope="GLOBAL")
class BrokenInventoryLibrary:
    def __init__(self):
        self.items = []

    @keyword
    def remember(self, item: str):
        self.items.append(item)

    @keyword
    def count(self) -> int:
        return len(self.items)

If test B expects zero but runs after test A appended an item, B fails only in the full suite. Rerunning B alone may pass. That is classic shared-state evidence. Repair to TEST scope unless the shared object represents a deliberate resource. If broader scope is genuinely needed, add explicit reset and prove it in lifecycle setup/teardown.

4. Failure: import-time side effects mutate the machine before a keyword runs

# BROKEN: runs during Python import / Libdoc inspection.
from pathlib import Path

CACHE = Path.home() / ".my-real-cache"
CACHE.mkdir(exist_ok=True)
REMOTE_CLIENT = connect_to_service()

Libdoc, editor discovery, dry-run, and test import can now create directories or network connections. Import must be as safe and deterministic as possible. Move resource acquisition to the library constructor or an explicit keyword with a documented owner, and use disposable paths/authorized targets.

5. Failure: Secret wrapper is unwrapped and logged

from robot.api import logger
from robot.api.types import Secret

# BROKEN: explicit disclosure.
def authenticate(token: Secret):
    logger.info(f"Using token {token.value}")

The Robot Secret wrapper did its job until the library accessed the plaintext and logged it. Repair by logging only safe metadata such as a synthetic credential ID, never .value. Search output.xml, log.html, stdout/stderr, debug files, screenshots, and subprocess arguments after security-sensitive changes.

6. Failure: generic exceptions provide no useful automation context

# BROKEN
raise Exception("failed")

# Better adapter boundary
try:
    record = domain_lookup(record_id)
except ValueError as exc:
    raise Failure(f"Cannot load synthetic record {record_id}: {exc}") from exc

Preserve identifiers that help reproduce the failure but exclude secrets/PII. Do not catch every exception and replace it with a generic PASS/FAIL string; unexpected exceptions should remain unexpected evidence.

7. Failure: opaque/custom return objects leak implementation details

Robot can carry arbitrary Python objects in scalar variables, but that does not make every object a good public return type. Returning an internal HTTP client, database cursor, ORM session, or mutable cache lets test data couple directly to implementation APIs and lifetimes.

Prefer domain values such as a mapping, immutable identifier, status object with documented properties, or a higher-level follow-up keyword. If a custom object is intentionally public, document lifetime/thread-safety/serialization expectations and test them.

8. Failure: library depends on private Robot internals

# FRAGILE: internal package names are not the supported integration contract.
from robot.running.context import EXECUTION_CONTEXTS
from robot.output import librarylogger

Robot's internal modules can change without the stability promises of robot.api. Use robot.api.logger, robot.api.deco, robot.api.exceptions, robot.api.types, and other documented public APIs when available. Chapter 21 will introduce dynamic/hybrid/Remote extension architecture; that is still not a license to reach into private execution state.

9. Failure: library import works only from one current working directory

# Works by accident from project root:
python -m robot --pythonpath src tests/robot/inventory.robot

# Fails if a CI job changes cwd and "src" no longer points to project source.
cd tests
python -m robot --pythonpath src robot/inventory.robot

The fix is a documented command contract anchored to the project root, an absolute/resolved search path, or an installed packaged dependency. Do not add arbitrary parent directories to PYTHONPATH until the import succeeds; that can shadow unrelated modules and create a different failure later.

10. Failure: module shadowing changes which library imports

A local file named robot.py, json.py, requests.py, or the same name as the custom package can shadow the intended module. Inspect module.__file__ with the same Python interpreter used by Robot. Rename the collision and recreate evidence rather than manipulating sys.path in library source.

python -c "import rf20_domain, robot; print(rf20_domain.__file__); print(robot.__file__)"

11. Failure: logging ERROR is mistaken for a keyword assertion

logger.error() reports a test execution error; it is not the same contract as raising Failure. If the business condition must fail the current keyword/test, raise an appropriate exception. Reserve ERROR logs for execution-level problems that truly belong in Robot's execution-errors channel.

Similarly, do not raise FatalError merely to avoid writing proper cleanup or error handling. Fatal stopping is for conditions that make the remaining execution meaningless.

12. Intentionally broken scope example: interpret the evidence before fixing

# Change exactly one line in the Chapter 20 lab:
@library(scope="GLOBAL", version="1.0.0")
class InventoryLibrary:
    ...

First Test Mutates State
    ${count}=    Quoted Count
    Should Be Equal As Integers    ${count}    0
    Quote Line    SKU-A    1.0    1

Second Test Requires Isolation
    ${count}=    Quoted Count
    Should Be Equal As Integers    ${count}    0

Predict: the first test passes and leaves one SKU in the shared instance; the second test sees count 1 and fails. Preserve that output. Then restore scope="TEST" and rerun into a new directory. The repaired run demonstrates architecture—not retry magic—changed the state lifetime.

13. Performance: initialization cost is only one layer

Measure import time, constructor/resource initialization, Python keyword time, external-system latency, Robot logging/output serialization, Pabot scheduling, CI/container startup, and artifact upload separately. A GLOBAL singleton may reduce constructor cost but increase correctness risk; do not trade determinism for unmeasured speed.

Unit-test pure Python hot paths independently. Robot is an orchestration/integration framework, not a microbenchmark harness.

14. Troubleshooting shortcuts to reject

• Do not swallow all Python exceptions and return False so Robot stays green.
• Do not convert every library to GLOBAL scope to “fix” missing state.
• Do not append random parent paths to PYTHONPATH until imports work.
• Do not log plaintext Secret values to diagnose authentication.
• Do not use private Robot internals when a public API exists.
• Do not delete first-failure output before comparing the repair.
• Do not use production APIs/files/databases as a library debugging fixture.

Knowledge check

15. Summary and bridge

Library failures become manageable when import/discovery, Python scope, conversion, domain logic, status exceptions, logging, secrets, external state, and Robot evidence are treated as separate layers. Lesson 5 combines these ideas in a checkpoint lab that intentionally introduces both a scope leak and a conversion failure, then repairs the public API contract with independent evidence.

References and version anchors

• Robot Framework 7.4.2 — Creating test libraries — discovery, status, logging, conversion
• robot.api package — stable public extension APIs
• robot.api.exceptions — Failure/Error/Skip/Fatal semantics
• robot.api.logger — public logging behavior
• Robot Framework Libdoc — discovery/spec diagnostics