Assertions, Error Handling, Expected Failures, and Recovery Patterns: Guided Hands-On Workflow

1. Disposable scenario: validate synthetic release records

Create a fresh directory named rf13-failure-lab. The suite operates only on literal strings, numbers, and Robot variables. There is no browser, network, database, SSH target, process manipulation, credential, or production state. Generated evidence goes under results/.

2. Preflight and evidence directory

python --version
python -m robot --version
mkdir -p results

PowerShell equivalent for the directory is New-Item -ItemType Directory -Force results. Confirm Robot Framework 7.4.2 before comparing the documented messages/matching behavior. Do not delete an existing results directory: use unique subdirectories so the first failure survives.

3. Start with a small assertion suite

*** Settings ***
Documentation    Synthetic failure-semantics laboratory.

*** Keywords ***
Validate Release State
    [Arguments]    ${state}
    Should Be Equal    ${state}    ready    msg=Release state must be exactly 'ready'

Reject Negative Amount
    [Arguments]    ${amount}
    IF    $amount < 0
        Fail    amount must be non-negative
    END

*** Test Cases ***
Precise assertion passes
    Validate Release State    ready

Precise assertion fails
    Validate Release State    pending

Run this once into results/assertions. The second test should FAIL with the custom contract message and actual/expected evidence from the assertion. Keep that output directory; it is your baseline failure artifact.

4. Preserve first-failure artifacts explicitly

python -m robot --outputdir results/assertions lab.robot
# Keep results/assertions/output.xml, log.html, report.html unchanged.

A repair should go to a new output directory. Overwriting the original result removes the evidence needed to prove whether the failure mode changed.

5. Add an exact expected-error negative case

*** Test Cases ***
Negative amount is rejected for the intended reason
    ${error}=    Run Keyword And Expect Error
    ...    EQUALS:amount must be non-negative
    ...    Reject Negative Amount    ${-5}
    Should Be Equal    ${error}    amount must be non-negative

This test is green only when the intended error occurs. Change the expected text by one character and it must fail. That controlled break proves the negative test is not merely suppressing an error.

6. Recover one known condition and expose unknown failures

*** Keywords ***
Read Synthetic Cache
    [Arguments]    ${mode}
    IF    $mode == 'cold'
        Fail    cache-not-ready: node-a
    ELSE IF    $mode == 'broken'
        Fail    checksum-corrupt: node-a
    END
    RETURN    cached-value

Read With Narrow Recovery
    [Arguments]    ${mode}
    TRY
        ${value}=    Read Synthetic Cache    ${mode}
    EXCEPT    cache-not-ready:    type=START    AS    ${error}
        Log    Expected transient condition: ${error}
        ${value}=    Set Variable    fallback-value
    END
    RETURN    ${value}

*** Test Cases ***
Known condition is recovered
    ${value}=    Read With Narrow Recovery    cold
    Should Be Equal    ${value}    fallback-value

Unknown condition still fails
    Read With Narrow Recovery    broken

The first case can PASS because recovery is explicitly part of the keyword contract. The second must FAIL because checksum-corrupt does not match the START pattern. If both pass, the handler is too broad.

7. Compare hard failure with continuable failure

*** Test Cases ***
Hard failure stops normal body
    Should Be Equal    alpha    beta
    Log    This line is not executed

Continuable failure collects later evidence
    Run Keyword And Continue On Failure    Should Be Equal    alpha    beta
    Log    This line executes
    Log    But final test status remains FAIL

Run them separately or inspect their keyword trees in log.html. The distinction is flow, not final truth.

8. Intentionally create and repair a false green

*** Test Cases ***
False green — intentionally broken design
    ${ok}=    Run Keyword And Return Status    Should Be Equal    expected    actual
    Log    Validation status was ${ok}
    # No assertion on ${ok}: the test itself PASSes. This is the defect.

Repaired status contract
    ${ok}=    Run Keyword And Return Status    Should Be Equal    expected    actual
    Should Be True    ${ok}    msg=Validation result must be true

The first test demonstrates why a status-returning helper is not an assertion. It converts a normal failure into a Boolean. If the Boolean is unused, CI can see PASS. The repaired test turns the Boolean back into a required contract and correctly FAILs. In many real cases the clearer repair is to remove the wrapper and let the original assertion fail directly.

9. Add a legitimate skip without calling it success

*** Test Cases ***
Capability intentionally unavailable
    Skip    Synthetic capability is not provisioned in this local profile.
    Fail    This is never executed.

The result is SKIP. A test teardown, if present, still runs. Use a concrete reason so the report explains why no verification occurred.

10. Preserve a direct first failure before the bounded retry experiment

*** Keywords ***
Reset Attempt Counter
    VAR    ${ATTEMPT}    ${0}    scope=TEST

Eventually Ready
    ${next}=    Evaluate    $ATTEMPT + 1
    VAR    ${ATTEMPT}    ${next}    scope=TEST
    Log    attempt=${ATTEMPT}
    IF    $ATTEMPT < 3
        Fail    synthetic resource not ready yet
    END
    RETURN    ready

*** Test Cases ***
Bounded transient retry
    Reset Attempt Counter
    ${value}=    Wait Until Keyword Succeeds    3x    0s    Eventually Ready
    Should Be Equal    ${value}    ready
    Should Be Equal As Integers    ${ATTEMPT}    3

Before enabling the wrapper, run Eventually Ready once in a separate test/output directory and preserve the failure. Then run the bounded retry case. The three attempts should be visible in the log. This is deterministic teaching data—not permission to wrap flaky browser/API checks in blanket retries.

11. Expected status matrix

12. Challenge: choose the correct layer

A deployment-readiness test receives maintenance-window, checksum-corrupt, or ready. Only maintenance-window is an approved reason to skip; checksum-corrupt must fail; ready must pass. Decide whether you need IF, TRY/EXCEPT, Skip, or an expected-error assertion—and justify the choice from final status semantics. Do not use a broad EXCEPT or retry.

13. Knowledge check

14. Summary and next step

You have now exercised every major status contract using synthetic data and concrete result evidence. Lesson 3 compares the design choices behind these mechanisms so teams can encode consistent policies instead of choosing wrappers ad hoc.

Further reading

• Robot Framework 7.4.2 User Guide — Failures — failure propagation and final error messages.
• Robot Framework 7.4.2 User Guide — TRY/EXCEPT syntax — exact/pattern matching, ELSE, FINALLY, and error capture.
• Robot Framework 7.4.2 User Guide — Continuing on failure — continuable failures and reserved tags.
• Robot Framework 7.4.2 User Guide — SKIP status — skip, exclude, dynamic skip, and skip-on-failure.
• Robot Framework 7.4.2 BuiltIn library — assertions, status helpers, expected errors, skip, fail, and bounded retry.
• RFCP syllabus — SKIP status — conceptual distinction between skip and exclude.
• Robot Framework 7.4.2 on PyPI — pinned stable package metadata and Python requirement.
• Robot Framework releases — re-check stable/pre-release status when updating the chapter.