Assertions, Error Handling, Expected Failures, and Recovery Patterns: Configuration, Design Patterns, and Trade-Offs

1. Design from the result contract backward

Start with the statement the report must be able to make: “the value was correct,” “the invalid input was rejected for reason X,” “a known transient condition was recovered,” “additional diagnostics ran but the scenario failed,” or “the scenario could not be executed and was skipped.” Then select the mechanism that naturally yields that result. Starting from a convenient helper and deciding later what its Boolean/status means is how false greens are created.

2. Assertion-specific keyword versus generic equality

3. Exact versus pattern error matching

Native EXCEPT is exact by default. That is the safest default when the message is a stable public contract. Use type=START when a stable prefix is followed by volatile context; GLOB for a deliberately simple wildcard contract; REGEXP only when the structure truly needs it; and LITERAL when you want to make exactness explicit. Run Keyword And Expect Error differs by using glob matching by default, so use the EQUALS: prefix for exact negative tests.

4. TRY/EXCEPT versus status-returning BuiltIn helpers

Native TRY/EXCEPT keeps the decision close to the error message and makes unmatched failures propagate automatically. Run Keyword And Return Status is useful when a Boolean is truly the desired data value, but it removes the original failure from normal propagation. Run Keyword And Ignore Error similarly returns PASS/FAIL plus a value/message. Both require the caller to do something meaningful with the returned state.

5. Continue-on-failure versus fail-fast

Fail-fast is the default because later steps may be invalid or destructive after a failed prerequisite. Continue-on-failure is appropriate when later observations are independent and safe: for example, collect three read-only configuration assertions in one diagnostic scenario while still failing the scenario at the end.

6. Retry versus deterministic wait/recovery

A retry is not a diagnosis. Use it only when the system contract explicitly allows eventual readiness and the operation is idempotent/safe to repeat. Prefer a condition-specific library wait when one exists because it observes the real condition and can provide better diagnostics. Wait Until Keyword Succeeds is generic and can amplify output; it should have a small count/time and meaningful interval.

Do not retry syntax/import errors, deterministic validation failures, credential denial, or data corruption. Those conditions do not become correct by repetition.

7. Negative testing versus suppressing errors

*** Test Cases ***
Good negative test
    Run Keyword And Expect Error
    ...    EQUALS:amount must be non-negative
    ...    Reject Negative Amount    ${-1}

Bad suppression
    ${status}    ${message}=    Run Keyword And Ignore Error
    ...    Reject Negative Amount    ${-1}
    Log    ${status}: ${message}
    # No assertion: any ordinary failure is effectively accepted.

Negative testing asserts both that failure occurs and that it is the correct failure. Suppression merely prevents propagation.

8. Skip is a governance signal, not recovery

Skip a scenario when the precondition for meaningful verification is absent and that absence is an accepted execution state. Examples include an intentionally unprovisioned optional capability or a platform-specific test on the wrong platform. Do not skip because an assertion failed. robot:skip-on-failure/--skiponfailure exist for known not-ready cases, but they should be governed carefully because they transform failed tests into SKIP and can conceal a growing backlog if applied broadly.

9. Worked decision table

10. Production design checklist

• Every caught error has a documented reason it is safe to recover.
• Every pattern matcher is as narrow as practical and has a test proving an unrelated error is rejected.
• Every returned status is asserted or explicitly used in a decision.
• Continuable failures are followed only by independent, safe observations/cleanup.
• Retries are bounded, safe to repeat, and preserve attempt evidence.
• Skip reasons explain why verification was not performed.
• Real secrets/PII are not embedded in assertion messages, expected patterns, or logs.
• Output directories are retained long enough to diagnose the first failure.

11. Knowledge check

12. Summary and next step

Good failure handling is policy expressed in executable form. Lesson 4 applies that policy to production failure modes: catch-all recovery, weak expected patterns, blanket retry, weak messages, continuable destructive flows, teardown masking, and normalized flakiness.

Further reading

• Robot Framework 7.4.2 User Guide — Failures — failure propagation and final error messages.
• Robot Framework 7.4.2 User Guide — TRY/EXCEPT syntax — exact/pattern matching, ELSE, FINALLY, and error capture.
• Robot Framework 7.4.2 User Guide — Continuing on failure — continuable failures and reserved tags.
• Robot Framework 7.4.2 User Guide — SKIP status — skip, exclude, dynamic skip, and skip-on-failure.
• Robot Framework 7.4.2 BuiltIn library — assertions, status helpers, expected errors, skip, fail, and bounded retry.
• RFCP syllabus — SKIP status — conceptual distinction between skip and exclude.
• Robot Framework 7.4.2 on PyPI — pinned stable package metadata and Python requirement.
• Robot Framework releases — re-check stable/pre-release status when updating the chapter.