Checkpoint Lab — Assertions, Error Handling, Expected Failures, and Recovery Patterns

1. Checkpoint scenario and safety boundary

Create rf13-checkpoint/ with one failure_matrix.robot file and a results/ directory. The entire lab uses Robot Framework core/BuiltIn plus synthetic literals and Robot variables. No filesystem deletion, child process, browser, API, database, SSH, Remote library, container, CI account, real secret, or production target is required.

2. Preflight

python --version
python -m robot --version
python -c "import sys; print(sys.executable)"

Record Robot Framework 7.4.2, Python version/interpreter, current directory, and the exact suite path. If your environment is different, record it before interpreting message/matching details.

3. Predict the status matrix before execution

Also predict two state changes: (1) the retry attempt counter should move from 0 to 3 within its test; (2) the false-green wrapper should turn the inner assertion failure into returned False without changing the test status unless you assert it.

4. Implement the complete matrix

*** Settings ***
Documentation    Chapter 13 failure-semantics checkpoint using synthetic data only.

*** Keywords ***
Reject Negative Amount
    [Arguments]    ${amount}
    IF    $amount < 0
        Fail    amount must be non-negative
    END

Read Synthetic Cache
    [Arguments]    ${mode}
    IF    $mode == 'cold'
        Fail    cache-not-ready: node-a
    ELSE IF    $mode == 'corrupt'
        Fail    checksum-corrupt: node-a
    END
    RETURN    cached-value

Read With Approved Recovery
    [Arguments]    ${mode}
    TRY
        ${value}=    Read Synthetic Cache    ${mode}
    EXCEPT    cache-not-ready:    type=START    AS    ${error}
        Log    approved recovery for ${error}
        ${value}=    Set Variable    fallback-value
    END
    RETURN    ${value}

Reset Attempt Counter
    VAR    ${ATTEMPT}    ${0}    scope=TEST

Eventually Ready
    ${next}=    Evaluate    $ATTEMPT + 1
    VAR    ${ATTEMPT}    ${next}    scope=TEST
    Log    retry-attempt=${ATTEMPT}
    IF    $ATTEMPT < 3
        Fail    synthetic resource not ready yet
    END
    RETURN    ready

*** Test Cases ***
01 Hard failure
    Should Be Equal    pending    ready    msg=Release state must be ready

02 Expected exact error
    ${error}=    Run Keyword And Expect Error
    ...    EQUALS:amount must be non-negative
    ...    Reject Negative Amount    ${-1}
    Should Be Equal    ${error}    amount must be non-negative

03 Narrow recovered condition
    ${value}=    Read With Approved Recovery    cold
    Should Be Equal    ${value}    fallback-value

04 Continuable failure remains failed
    Run Keyword And Continue On Failure    Should Be Equal    alpha    beta
    Log    evidence-after-continuable-failure

05 Explicit skip
    Skip    Synthetic optional capability is intentionally unavailable.
    Fail    This line must never run.

06 Bounded deterministic retry
    Reset Attempt Counter
    ${value}=    Wait Until Keyword Succeeds    3x    0s    Eventually Ready
    Should Be Equal    ${value}    ready
    Should Be Equal As Integers    ${ATTEMPT}    3

07 False green — intentionally broken
    ${ok}=    Run Keyword And Return Status    Should Be Equal    expected    actual
    Log    wrapped-validation=${ok}
    # No assertion on ${ok}. This test is intentionally wrong.

Do not “fix” the failing/skip cases before the baseline run. The point is to create a result matrix with different truthful statuses and one intentionally untruthful green.

5. Baseline run: preserve the matrix evidence

python -m robot --outputdir results/baseline failure_matrix.robot

The process should return non-zero because the suite contains failed tests. Open results/baseline/report.html for the high-level matrix and log.html for keyword flow. Keep output.xml unchanged; Chapter 14 will go deeper into result post-processing.

6. Verify each case independently

7. Prove the recovery is narrow

Add a temporary test and run it into results/unmatched-recovery:

*** Test Cases ***
Unknown recovery condition must fail
    Read With Approved Recovery    corrupt

Expected: FAIL with checksum-corrupt: node-a. If it passes, the recovery contract is broader than intended. Remove the temporary case after capturing evidence.

8. Preserve a first direct failure before accepting retry behavior

Create a temporary test that calls Reset Attempt Counter and Eventually Ready once without Wait Until Keyword Succeeds. Run it into results/retry-first-failure. It should FAIL on attempt 1. Keep that evidence, then compare with the baseline retry test that passes on attempt 3. The comparison proves what the retry actually changed.

9. Repair the intentionally false-green test

Replace only Test 07 with the following and run into results/false-green-repaired:

07 Status result must be asserted
    ${ok}=    Run Keyword And Return Status    Should Be Equal    expected    actual
    Should Be True    ${ok}    msg=Wrapped validation must pass

Expected: Test 07 is now FAIL. In production the simpler design would usually call Should Be Equal directly. The wrapper is retained here only to demonstrate that status-returning helpers create data that must be consumed deliberately.

10. Required evidence packet

• Python/Robot Framework version and interpreter path.
• The completed prediction matrix before execution.
• Exact baseline command and process return code.
• results/baseline/output.xml, log.html, and report.html.
• Per-test PASS/FAIL/SKIP statuses and key messages.
• Expected-error message evidence.
• Narrow recovery evidence plus the unmatched checksum-corrupt failure.
• Continuable failure evidence showing the later log plus final FAIL.
• Skip reason.
• Direct first retry failure and the three-attempt successful retry trace.
• False-green baseline and repaired FAIL evidence.
• A one-page team policy describing when to fail, recover, continue, skip, or retry.

11. Production failure policy distilled from the lab

12. Cleanup / rollback

No external resource was created. Keep the evidence directories if they are part of your learning/CI record. Otherwise remove only the disposable rf13-checkpoint directory after confirming its path. Do not recursively delete broad parent directories, source repositories, or unrelated results.

13. What Chapter 13 adds to a production Robot Framework operating model

Chapters 01–12 gave you reproducible environments, syntax, suites, keywords, variables, standard libraries, lifecycle, control flow, data-driven patterns, and deterministic imports. Chapter 13 adds the rule that makes all of those trustworthy in CI: every abnormal condition has an explicit status contract and preserved evidence. A recovered error is narrowly proven, a continuable failure stays red, a skipped scenario is not counted as success, a retry is bounded and observable, and wrappers cannot silently turn failed assertions green.

14. Knowledge check

15. Checkpoint complete

You have built and verified a complete Robot Framework failure-semantics matrix. You can now distinguish “handled” from “successful,” preserve first-failure evidence, prove negative tests fail for the intended reason, keep continuable failures red, use skip honestly, bound retries, and detect false-green status wrappers. Chapter 14 turns those statuses into durable, post-processable execution evidence.

Further reading

• Robot Framework 7.4.2 User Guide — Failures — failure propagation and final error messages.
• Robot Framework 7.4.2 User Guide — TRY/EXCEPT syntax — exact/pattern matching, ELSE, FINALLY, and error capture.
• Robot Framework 7.4.2 User Guide — Continuing on failure — continuable failures and reserved tags.
• Robot Framework 7.4.2 User Guide — SKIP status — skip, exclude, dynamic skip, and skip-on-failure.
• Robot Framework 7.4.2 BuiltIn library — assertions, status helpers, expected errors, skip, fail, and bounded retry.
• RFCP syllabus — SKIP status — conceptual distinction between skip and exclude.
• Robot Framework 7.4.2 on PyPI — pinned stable package metadata and Python requirement.
• Robot Framework releases — re-check stable/pre-release status when updating the chapter.