Assertions, Error Handling, Expected Failures, and Recovery Patterns: Core Concepts and Mental Model

1. The problem: green is meaningful only when failure semantics are explicit

Chapter 12 made imports deterministic. That makes the next question more important: when a keyword reports a problem, what status should the test have? A test that catches every error can be green while the system is broken. A negative test can be green for the wrong error. A “continue on failure” step can continue and still correctly leave the test red. A retry can eventually pass while hiding a recurring defect unless its attempts are preserved.

Robot Framework gives you several mechanisms because they express different intent. The production skill is not memorizing the keywords; it is choosing a status contract that tells CI and future readers the truth.

2. Read-only preflight before changing status behavior

python --version
python -m robot --version
python -m robot --dryrun --outputdir results/preflight tests

Record the interpreter/framework version, executed path, selected tests/tags, and any pre-existing output.xml/log.html/report.html. Dry-run can expose parse/import/keyword-resolution problems before runtime recovery logic is even relevant. Do not add an EXCEPT around a failure until you know which layer produced it.

3. Status propagation mental model

flowchart TD
A[Keyword call] --> B{Outcome}
B -->|PASS| P[Continue normally]
B -->|Hard FAIL| F[Stop normal body]
B -->|Continuable FAIL| C[Continue but record failure]
B -->|Expected / narrowly recovered| R[Recovery contract decides next step]
B -->|SKIP| S[Stop test body as skipped]
F --> T[Test teardown]
C --> T
R --> T
S --> T
P --> T
T --> X[Test status PASS / FAIL / SKIP]
X --> Y[Suite status]
Y --> Z[output.xml / log.html / report.html / process return code]

The first arrow is the keyword result. A normal failure stops the body; a continuable failure records failure but allows later work; an expected/recovered failure is only a PASS when the recovery contract positively proves the observed error was the one intended; a skip is neither PASS nor FAIL. Teardown is a cleanup/evidence phase, not a mechanism for erasing the original outcome. The final status feeds suite status and the runner return code used by CI.

4. Define the state owners before handling an error

5. Six contracts that must not be collapsed

6. Assertions are evidence-producing contracts, not generic booleans

Prefer the assertion that states what is wrong. A containment failure should say which collection/string missed which item; a numeric comparison should use a numeric contract; a pattern should use an explicit pattern assertion. Generic equality is valuable when exact equality is the real requirement, but it should not become the only assertion because it can force the reader to reverse-engineer intent.

*** Test Cases ***
Precise assertions
    Should Contain    release-ready    ready
    Should Be Equal As Integers    42    42
    Should Match    artifact-2026.08    artifact-2026.*
    Should Be Equal    expected    expected    msg=Release state must match the approved fixture

7. Expected error is a positive assertion about the error contract

*** Keywords ***
Validate Amount
    [Arguments]    ${amount}
    IF    $amount < 0
        Fail    amount must be non-negative
    END

*** Test Cases ***
Negative amount is rejected
    ${message}=    Run Keyword And Expect Error
    ...    EQUALS:amount must be non-negative
    ...    Validate Amount    ${-1}
    Should Be Equal    ${message}    amount must be non-negative

The test passes because it proves the input was rejected for the exact intended reason. A broad * would also match unrelated failures and is therefore weaker evidence. Native TRY/EXCEPT is preferred for richer recovery; Run Keyword And Expect Error remains concise when the whole contract is simply “this keyword must fail with this error.”

8. TRY/EXCEPT: exact by default, patterns only when intentional

*** Test Cases ***
Recover one known technical condition
    TRY
        Fail    cache-not-ready: node-a
    EXCEPT    cache-not-ready:    type=START    AS    ${error}
        Log    Known disposable condition: ${error}
    END
    Log    Recovery path completed

In 7.4.2, a message on EXCEPT is an exact match unless type= changes it. GLOB, REGEXP, START, and LITERAL are explicit choices. A message-less EXCEPT catches ordinary errors broadly and should be rare in production tests because it can convert unknown defects into success. Syntax errors and errors that stop the whole execution are not catchable by native TRY/EXCEPT.

9. Continue-on-failure means “collect more evidence,” not “ignore the failure”

*** Test Cases ***
Collect two observations but remain failed
    Run Keyword And Continue On Failure    Should Be Equal    alpha    beta
    Log    This observation is still collected
    Should Be Equal    ${1}    ${1}

The second and third lines run, but the test is still FAIL because the first assertion was continuable—not successful. This is appropriate for independent diagnostics that are safe after a validation failure. It is inappropriate before destructive actions that assume the failed precondition was satisfied.

10. SKIP and retry answer different questions

SKIP says the scenario was not valid to execute or cannot currently provide pass/fail evidence. Retry says repeated attempts are part of a bounded transient-condition contract. Neither should be used to make an unstable test “look better.” A skipped test is visible as SKIP; a retried test should retain attempt evidence in log.html.

11. DevOps connection: status is an API consumed by CI

Robot's process return code is derived from final execution status. That means error handling is part of the delivery interface. If a helper turns a failed assertion into an unused Boolean, the pipeline may receive zero and deploy. If a test uses continuable failures, the pipeline remains red while diagnostics are richer. If a capability is unavailable and the test is legitimately skipped, reports distinguish “not executed” from “verified.”

12. Knowledge check

13. Summary and next step

You now have the status model: failures, recovery, continuation, skip, and retry are different contracts. Lesson 2 turns that model into a disposable suite where each mechanism is exercised with concrete result evidence and a deliberately false-green case.

Further reading

• Robot Framework 7.4.2 User Guide — Failures — failure propagation and final error messages.
• Robot Framework 7.4.2 User Guide — TRY/EXCEPT syntax — exact/pattern matching, ELSE, FINALLY, and error capture.
• Robot Framework 7.4.2 User Guide — Continuing on failure — continuable failures and reserved tags.
• Robot Framework 7.4.2 User Guide — SKIP status — skip, exclude, dynamic skip, and skip-on-failure.
• Robot Framework 7.4.2 BuiltIn library — assertions, status helpers, expected errors, skip, fail, and bounded retry.
• RFCP syllabus — SKIP status — conceptual distinction between skip and exclude.
• Robot Framework 7.4.2 on PyPI — pinned stable package metadata and Python requirement.
• Robot Framework releases — re-check stable/pre-release status when updating the chapter.