Setups, Teardowns, Tags, Timeouts, and Suite Lifecycle: Diagnostics, Failure Modes, and Production Practices

1. Diagnostic sequence

1. Preserve first-failure artifacts: copy/retain output.xml, log.html, report, marker files, and owned external evidence.
2. Confirm versions and exact command: Robot/Python, executed path, include/exclude/skip filters, variables, output directory.
3. Classify lifecycle phase: suite setup, test setup, body, test teardown, suite teardown, or timeout.
4. Inspect variable and keyword resolution: setup/teardown overrides, imported resource, active timeout, runtime tags.
5. Inspect owned external state: files/processes/records created by this test only.
6. Apply the smallest correction and rerun the smallest controlled slice into a new evidence directory.

2. Failure mode: expensive suite setup leaks mutable state

Suppose suite setup creates one shared file and test A edits it. Test B assumes the original content. B now depends on execution order. Re-running B alone may pass if suite setup recreates the original file, while the full suite fails.

*** Settings ***
Suite Setup       Create Shared File
Suite Teardown    Remove Shared File

*** Test Cases ***
A Mutates Shared File
    Append To File    ${SHARED}    changed

B Assumes Clean File
    ${text}=    Get File    ${SHARED}
    Should Be Equal    ${text}    initial

Repair: move the mutable file into per-test setup, or make the suite fixture immutable and copy it per test. Do not randomize order or add retries to disguise the dependency.

3. Failure mode: teardown noise obscures the primary failure

A test fails because the system behavior is wrong. Teardown then executes a fragile “should exist” assertion on a file that was never created because setup partially failed. The final result now contains multiple failures. Robot Framework preserves failures, but humans may focus on the last cleanup message instead of the initiating defect.

*** Test Cases ***
Broken Example
    [Teardown]    Brittle Cleanup
    Fail    primary-business-failure

*** Keywords ***
Brittle Cleanup
    File Should Exist    ${OPTIONAL_FILE}
    Remove File          ${OPTIONAL_FILE}

Repair: make cleanup conditional on resource ownership/existence, record whether cleanup was needed, and keep a meaningful postcondition. Do not suppress all cleanup errors; distinguish “nothing to clean” from “owned resource could not be removed.”

4. Failure mode: giant timeouts become operational fog

A 30-minute test timeout does not explain whether the browser, API, process, or database operation is stuck. It delays feedback and makes incident timing ambiguous. First configure a specific timeout in the operation that can block. Keep a smaller Robot-level timeout only as a final guard.

5. Failure mode: tags become environment configuration

A suite tagged staging does not magically point clients at a staging endpoint. If keyword code branches on tags to derive credentials or URLs, changing selection metadata changes system behavior in a hidden way.

Repair: tags can express requirements such as requires-database; concrete endpoint/account values come from explicit configuration with safe provenance.

6. Failure mode: order-dependent tests

Robot Framework executes tests in source order by default and nested suites according to documented suite-order rules. Stable order is not permission to share dirty state. A production suite should remain correct if a single test is selected or if future parallel execution changes scheduling.

7. Failure mode: teardown kills or deletes by broad pattern

For filesystem state, normalize and guard the exact path. For processes, retain a handle/alias or PID owned by the test. For database/API records, retain generated IDs. For browser sessions, teardown the exact driver/session object owned by the test.

8. Failure mode: “teardown passed” is mistaken for external cleanliness

A cleanup keyword returning PASS only proves the keyword reported success. Production-grade cleanup may require an independent postcondition: directory absent, process stopped, row deleted, session closed, or synthetic account disabled. Verify the important state—without querying unrelated resources.

9. Controlled broken lifecycle case

*** Settings ***
Library          OperatingSystem
Test Setup       Create Per Test File
Test Teardown    Broken Cleanup

*** Variables ***
${ROOT}    ${TEMPDIR}/rf09-diagnostic

*** Test Cases ***
Primary Failure Must Stay Visible
    Fail    primary-failure-RF09

*** Keywords ***
Create Per Test File
    Create Directory    ${ROOT}
    ${path}=    Join Path    ${ROOT}    owned.txt
    Set Test Variable    ${OWNED}    ${path}
    Create File    ${OWNED}    evidence

Broken Cleanup
    Remove File    ${OWNED}
    File Should Exist    ${OWNED}

Expected result: the body fails, cleanup removes the file, then the bad teardown assertion also fails. Preserve this result. Repair the final assertion to File Should Not Exist, add a guarded directory cleanup, and rerun into a new output directory. The repaired run should still FAIL because the intentionally injected primary failure remains.

10. Evidence packet for diagnosis

11. Troubleshooting shortcuts to reject

• Blanket reruns until green.
• Increasing every timeout without measuring the blocked operation.
• Adding robot:skip-on-failure to flaky production tests.
• Ignoring teardown failures globally.
• Turning mutable per-test state into suite/global variables.
• Deleting the failing output directory before diagnosis.
• Disabling TLS/SSH verification or experimenting on production targets to “reproduce faster.”

12. Knowledge check

13. Summary and next step

Lifecycle diagnosis is phase-oriented and evidence-first. Preserve the original run, identify whether setup/body/teardown/timeout/selection failed, verify only owned external state, and repair the smallest boundary. Lesson 5 combines the chapter into a lifecycle matrix checkpoint with PASS, FAIL, TIMEOUT, teardown evidence, tag selection, and a shared-state refactor.

Further reading

• Robot Framework 7.4.2 User Guide — test setup and teardown — defaults, overrides, teardown guarantees, and task aliases.
• Robot Framework 7.4.2 User Guide — execution flow — suite/test/keyword setup and teardown ordering and failure semantics.
• Robot Framework 7.4.2 User Guide — timeouts — test versus user-keyword timeout behavior and safety cautions.
• Robot Framework 7.4.2 User Guide — tags — Test Tags, reserved tags, include/exclude/skip semantics, and deprecations.
• Robot Framework 7.4.2 BuiltIn — Set Tags, Remove Tags, Skip/Skip If, and status/control helpers.
• Robot Framework on PyPI — current stable/pre-release stream and Python requirement.