HTTP/API Automation, JSON Validation, and Service-Level Testing: Diagnostics, Failure Modes, and Production Practices

1. Diagnostic sequence: preserve evidence, then isolate ownership

1. Preserve first-failure artifacts: output.xml/log/report, exact command, safe request identity, response status/selected headers/body excerpt, fixture log.
2. Confirm versions: Python, Robot, RequestsLibrary, Requests.
3. Confirm execution identity: selected test, base URL, variables, test data, output directory.
4. Validate parse/import graph: resource path, RequestsLibrary import, keyword resolution.
5. Inspect variable scope: response/owned ID/correlation provenance.
6. Inspect HTTP client state: session alias, cookies, headers, timeout, TLS verification, retry adapters.
7. Inspect server state: route, record existence, fixture logs, expected mutation.
8. Inspect timing/parallel/CI/container only if relevant: port collisions, shared IDs, localhost namespace, proxies, CA bundles.
9. Apply the least destructive correction and rerun the smallest slice into a new result directory.

2. Failure: a public or production API slipped into the test

Symptoms include unexpected DNS names, real tenant IDs, rate-limit responses, or data that predates the lab. The repair is not “make the assertion more flexible.” Stop the run, preserve command/variables, restore a guarded local/private target, and audit whether any mutation occurred. Never use this chapter's reset/delete/failure-injection operations against production.

3. Failure: asserting only the status code

# BROKEN: transport-only evidence masquerading as business proof
${response}=    POST    ${API_BASE}/items    json=${payload}    expected_status=201
# Test ends here.

A 201 response does not prove the record has the correct fields or identity. Repair by parsing the body, asserting the returned ID and selected domain fields, then GET the record and confirm persisted state.

4. Failure: expected-error pattern passes for the wrong reason

# BROKEN: any 4xx-ish behavior could be mistaken for the intended missing-record contract
${response}=    GET    ${API_BASE}/items/not-an-id    expected_status=any
Should Be True    ${response.status_code} >= 400

# REPAIR: exact route/input and exact business error
${response}=    GET    ${API_BASE}/items/9999    expected_status=any    timeout=3
Status Should Be    404    ${response}
${error}=    Get From Dictionary    ${response.json()}    error
Dictionary Should Contain Item    ${error}    code    not-found

The repaired version proves the intended condition, not merely “some error happened.”

5. Intentionally broken example: malformed JSON

${response}=    GET    ${API_BASE}/malformed    timeout=3    expected_status=200
${body}=    Set Variable    ${response.json()}    # expected to fail JSON decoding

Preserve the 200 status, Content-Type, correlation header, safe body excerpt, and Robot exception. Do not catch every exception and return ${None}. The failure is useful: transport succeeded but representation parsing failed. Repair the server/contract; do not weaken JSON validation.

6. Failure: shared mutable server data and order dependence

Two tests both create record ID 1 after assuming an empty global store, or one test deletes data another test expects. This may pass serially and fail under reordering/parallelism. The repair is unique synthetic identities plus ownership-aware cleanup. A suite-level reset can establish a disposable baseline, but each test must still own what it creates.

7. Failure: unbounded or blanket retries

RequestsLibrary sessions can configure retry adapters, and Robot has its own retry helpers. Neither should be the first response to a failing assertion. First determine whether the operation is idempotent, whether the server actually processed the request, and which layer is transient. Preserve each attempt if a retry is justified. A bounded retry policy is infrastructure design, not a way to turn flaky behavior green.

8. Failure: TLS or authentication error is swallowed

Do not set verify=False, suppress certificate warnings, or broaden an expected status to any and then ignore the result. TLS/authentication failures are trust-boundary evidence. Confirm hostname, CA bundle, proxy, client certificate, token provenance, and time validity. Use fake credentials only in this course lab.

9. Failure: JSON comparison depends on irrelevant ordering

JSON objects are semantic mappings. Comparing serialized strings can fail because of whitespace/key order. Arrays may or may not be order-sensitive depending on the contract. Parse first, then assert the fields and collection semantics that matter. Sort only when order is explicitly irrelevant and document why.

10. Failure: request/response evidence leaks secrets or PII

Headers and JSON bodies can carry credentials and personal data. Do not log Authorization headers, cookies, private tokens, or complete production bodies at DEBUG/TRACE. Redact at the domain-resource/evidence boundary and use synthetic fixtures. Retention policy is part of test design, not an afterthought.

11. Performance only where causal

Do not “optimize” by dropping assertions, sharing dirty server state, disabling logs needed for diagnosis, or increasing concurrency blindly.

Knowledge check

12. Summary and bridge

Reliable API diagnosis preserves the first response, identifies the owning layer, and fixes the narrowest contract. Lesson 5 turns that workflow into a scored checkpoint with CRUD state, malformed JSON, safe evidence, and cleanup proof.

References and version anchors

• Robot Framework 7.4.2 User Guide — execution, variables, lifecycle, status, result, and library boundaries.
• RequestsLibrary keyword documentation, RequestsLibrary 0.9.7 on PyPI, and maintainer repository — HTTP keyword signatures, session behavior, status handling, and release status.
• Python Requests documentation and Requests 2.34.2 on PyPI — underlying HTTP client semantics and current transport-library release anchor.