Test Data, Parameterization, Fixtures, and Environment Configuration: Diagnostics, Failure Modes, and Production Practices

1. Diagnostic sequence: preserve before changing

1. Preserve first-failure case ID, traceback, target name/URL, sanitized configuration, Selenium/browser versions, session ID, and relevant AUT state.
2. Confirm the exact parameter row/seed and whether any prior test mutated shared state.
3. Confirm fixture scope and whether teardown executed.
4. Inspect session/context/locator/synchronization state only after the data/environment preconditions are proven.
5. Inspect AUT/network evidence and Grid/CI capacity if remote.
6. Apply the least destructive correction and rerun the smallest identical case.

2. Failure: shared mutable test account

The following example makes the Failure: shared mutable test account behavior concrete. Read it with the stated assumptions, then compare its observable output or state changes with the explanation that follows.

# Intentionally broken design
SHARED_EMAIL = "shared@example.test"

def test_disable_account():
    api_set_enabled(SHARED_EMAIL, False)
    assert ui_login(SHARED_EMAIL) == "disabled"

def test_change_role():
    api_set_role(SHARED_EMAIL, "admin")
    assert ui_role(SHARED_EMAIL) == "admin"

Run concurrently and these scenarios race on one AUT record. The browser sessions may be perfectly isolated. Repair by assigning each case a unique deterministic synthetic identity and deleting/resetting only that identity.

3. Failure: order-dependent data

The following example makes the Failure: order-dependent data behavior concrete. Read it with the stated assumptions, then compare its observable output or state changes with the explanation that follows.

# Intentionally broken: test_02 depends on test_01 side effects.
def test_01_create_user():
    api_create({"email": "ordered@example.test"})

def test_02_show_user():
    assert RegistrationPage(driver, base_url).find("ordered@example.test")

If test_02 runs first or alone, it fails. That is not a locator defect. Each test must create or fixture its own prerequisite inside its lifecycle.

4. Failure: unseeded random generator

The following example makes the Failure: unseeded random generator behavior concrete. Read it with the stated assumptions, then compare its observable output or state changes with the explanation that follows.

# Broken evidence: the failing input cannot be reconstructed later.
import random
quantity = random.randint(1, 1000)
run_case(quantity)

The following example makes the Failure: unseeded random generator behavior concrete. Read it with the stated assumptions, then compare its observable output or state changes with the explanation that follows.

# Repair: either use explicit boundary cases or record a deterministic seed.
import random
seed = 20260828
rng = random.Random(seed)
quantity = rng.randint(1, 1000)
print({"seed": seed, "quantity": quantity})
run_case(quantity)

5. Failure: teardown skipped after assertion

The following example makes the Failure: teardown skipped after assertion behavior concrete. Read it with the stated assumptions, then compare its observable output or state changes with the explanation that follows.

# Broken: reset is never reached when the assertion fails.
driver = webdriver.Chrome()
api_reset(base_url)
page = RegistrationPage(driver, base_url)
page.open()
assert page.submit(row) == "wrong expectation"
driver.quit()
api_reset(base_url)

The following example makes the Failure: teardown skipped after assertion behavior concrete. Read it with the stated assumptions, then compare its observable output or state changes with the explanation that follows.

# Repair: lifecycle ownership is expressed structurally.
with isolated_case(base_url, browser) as driver:
    page = RegistrationPage(driver, base_url)
    page.open()
    assert page.submit(row) == f"created:{row['email']}"

The original failure remains visible; the fixture only guarantees teardown. It must not swallow or replace the assertion exception.

6. Failure: secrets committed or printed

The following example makes the Failure: secrets committed or printed behavior concrete. Read it with the stated assumptions, then compare its observable output or state changes with the explanation that follows.

# Never do this.
TEST_USERS = [{"email": "real-user@example.com", "password": "P@ssw0rd!"}]
print(os.environ)

Use fake synthetic identities and inject credentials only when a disposable authenticated environment truly requires them. Evidence may record token_present=true or a non-sensitive credential alias, never the secret value.

7. Failure: hard-coded environment URL

The following example makes the Failure: hard-coded environment URL behavior concrete. Read it with the stated assumptions, then compare its observable output or state changes with the explanation that follows.

# Brittle and potentially unsafe.
BASE_URL = "https://prod.example.invalid/"
driver.get(BASE_URL)

Hard-coding makes source code choose an environment. The mandatory academy labs use loopback-only targets and reject any override that is not localhost or a loopback IP.

8. Production-target guard

The following example makes the Production-target guard behavior concrete. Read it with the stated assumptions, then compare its observable output or state changes with the explanation that follows.

from urllib.parse import urlparse
import ipaddress


def require_loopback(url: str) -> str:
    parsed = urlparse(url)
    if parsed.scheme not in {"http", "https"} or not parsed.hostname:
        raise RuntimeError("base URL must be absolute http(s)")
    host = parsed.hostname
    if host == "localhost":
        return url
    try:
        address = ipaddress.ip_address(host)
    except ValueError as exc:
        raise RuntimeError("academy lab refuses non-loopback host") from exc
    if not address.is_loopback:
        raise RuntimeError("academy lab refuses non-loopback host")
    return url

This guard is intentionally stricter than a production suite would be. It makes the learning lab incapable of “falling through” to staging or production. Real organizations can use allowlisted environment IDs and deployment metadata, but the default must remain deny-by-default for destructive tests.

9. Failure: opaque parameter IDs

The following table organizes the key choices and evidence for Failure: opaque parameter IDs. Use it together with the surrounding prose so the rows serve as a comparison aid rather than standalone rules.

Case IDs should not contain secrets or personal data. They describe the behavioral dimension, not the literal user identity.

10. Intentionally broken local example and repair

Use the Lesson 2 fixture but deliberately give two rows the same email. The first row succeeds; the second returns error:duplicate email from the AUT while the test expects created:.... Preserve the browser result and API state.

Observed browser state: error:duplicate email
AUT state: {"users":[{"email":"duplicate@example.test", ...}]}
Interpretation: deterministic data collision, not Selenium timing.
Repair: make case identity unique or reset the AUT per case; do not retry the click.

11. Performance only where causally relevant

Data setup can dominate a suite, but diagnose the layer: browser/session startup, fixture server startup, API seeding, AUT/network latency, evidence I/O, Grid queue, or retry cost. Do not broaden fixture scope until measurement shows startup is the bottleneck and the proposed sharing does not create mutable-state coupling.

12. Production practice

Use environment allowlists, synthetic account namespaces, idempotent cleanup, per-test workspaces, redacted config evidence, and explicit matrix metadata. Make teardown observable: a cleanup failure is a real test-infrastructure failure and should be reported separately from the scenario’s first failure.

13. Summary and next step

State/configuration incidents become actionable when the first evidence includes case identity, environment, seed/data, fixture scope, session provenance, and post-failure AUT state. Fix ownership—not symptoms.

Knowledge check

Official references and version notes

• Selenium 4.47 release notes — stable binding/Grid baseline pinned for this chapter.
• Selenium downloads — current stable Selenium client and Server/Grid versions.
• Overview of Test Automation — keep browser setup/actions/evaluation small and use lower layers where they provide the right signal.
• Avoid sharing state — isolate test data and prefer a new WebDriver instance per test.
• Fresh browser per test — start from a clean known browser state.
• Test independency — do not make one scenario depend on another scenario's state.
• Generating application state — repetitive application setup is usually more stable through a lower-layer API than through browser UI.