Checkpoint Lab — Test Data, Parameterization, Fixtures, and Environment Configuration

1. Checkpoint contract and predictions

Before running anything, write two predictions:

1. Each parameter row will create one new WebDriver session and exactly one synthetic AUT user, then cleanup will return the AUT to users=[].
2. Run 1 and Run 2 will have the same case IDs, deterministic emails, roles, and business outcomes, but different WebDriver session IDs and potentially different ephemeral loopback ports.

2. Create the disposable project

The following example makes the Create the disposable project behavior concrete. Read it with the stated assumptions, then compare its observable output or state changes with the explanation that follows.

mkdir selenium-ch14-checkpoint
cd selenium-ch14-checkpoint
python -m venv .venv
# Windows PowerShell: .\.venv\Scripts\Activate.ps1
# POSIX shell:       source .venv/bin/activate
python -m pip install "selenium==4.47.0"
mkdir evidence

3. app_fixture.py — in-memory loopback AUT

The following example makes the app_fixture.py — in-memory loopback AUT behavior concrete. Read it with the stated assumptions, then compare its observable output or state changes with the explanation that follows.

from contextlib import contextmanager
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
from threading import Thread
import json

HTML = """<!doctype html>
<html lang="en"><head><meta charset="utf-8"><title>Synthetic Registration Lab</title>
<style>body{font-family:system-ui,sans-serif;max-width:760px;margin:2rem auto;padding:0 1rem}label{display:block;margin:.75rem 0}input,select,button{padding:.45rem .65rem}#result{margin-top:1rem;font-weight:700}</style></head>
<body><h1>Registration Lab</h1><form id="signup">
<label>Name <input data-testid="name" required></label>
<label>Email <input data-testid="email" type="email" required></label>
<label>Role <select data-testid="role"><option>viewer</option><option>editor</option><option>admin</option></select></label>
<button data-testid="submit" type="submit">Create synthetic user</button></form><p id="result" data-testid="result">ready</p>
<script>
const form=document.querySelector('#signup');
form.addEventListener('submit',async e=>{e.preventDefault();const payload={name:document.querySelector('[data-testid=name]').value,email:document.querySelector('[data-testid=email]').value,role:document.querySelector('[data-testid=role]').value};const r=await fetch('/api/users',{method:'POST',headers:{'Content-Type':'application/json'},body:JSON.stringify(payload)});const body=await r.json();document.querySelector('#result').textContent=r.ok?`created:${body.email}`:`error:${body.error}`;});
</script></body></html>"""

class Handler(BaseHTTPRequestHandler):
    state = {"users": []}

    def log_message(self, fmt, *args):
        return

    def _json(self, status, payload):
        raw = json.dumps(payload).encode()
        self.send_response(status)
        self.send_header("Content-Type", "application/json")
        self.send_header("Content-Length", str(len(raw)))
        self.end_headers()
        self.wfile.write(raw)

    def do_GET(self):
        if self.path == "/":
            raw = HTML.encode()
            self.send_response(200)
            self.send_header("Content-Type", "text/html; charset=utf-8")
            self.send_header("Content-Length", str(len(raw)))
            self.end_headers()
            self.wfile.write(raw)
        elif self.path == "/health":
            self._json(200, {"ok": True})
        elif self.path == "/api/state":
            self._json(200, self.state)
        else:
            self._json(404, {"error": "not found"})

    def do_POST(self):
        length = int(self.headers.get("Content-Length", "0"))
        body = self.rfile.read(length) if length else b"{}"
        if self.path == "/api/reset":
            self.state = {"users": []}
            type(self).state = self.state
            self._json(200, {"reset": True})
            return
        if self.path == "/api/users":
            payload = json.loads(body or b"{}")
            email = payload.get("email", "")
            if any(x["email"] == email for x in self.state["users"]):
                self._json(409, {"error": "duplicate email"})
                return
            record = {"name": payload.get("name", ""), "email": email, "role": payload.get("role", "viewer")}
            self.state["users"].append(record)
            self._json(201, record)
            return
        self._json(404, {"error": "not found"})

@contextmanager
def running_app():
    Handler.state = {"users": []}
    server = ThreadingHTTPServer(("127.0.0.1", 0), Handler)
    thread = Thread(target=server.serve_forever, daemon=True)
    thread.start()
    host, port = server.server_address
    try:
        yield f"http://{host}:{port}/"
    finally:
        server.shutdown()
        server.server_close()
        thread.join(timeout=2)

4. pages.py — keep UI mechanics in the page boundary

The following example makes the pages.py — keep UI mechanics in the page boundary behavior concrete. Read it with the stated assumptions, then compare its observable output or state changes with the explanation that follows.

from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import Select, WebDriverWait

class RegistrationPage:
    NAME = (By.CSS_SELECTOR, "[data-testid='name']")
    EMAIL = (By.CSS_SELECTOR, "[data-testid='email']")
    ROLE = (By.CSS_SELECTOR, "[data-testid='role']")
    SUBMIT = (By.CSS_SELECTOR, "[data-testid='submit']")
    RESULT = (By.CSS_SELECTOR, "[data-testid='result']")

    def __init__(self, driver, base_url):
        self.driver = driver
        self.base_url = base_url

    def open(self):
        self.driver.get(self.base_url)
        WebDriverWait(self.driver, 3).until(lambda d: d.title == "Synthetic Registration Lab")

    def submit(self, row):
        name = self.driver.find_element(*self.NAME)
        email = self.driver.find_element(*self.EMAIL)
        name.clear(); name.send_keys(row["name"])
        email.clear(); email.send_keys(row["email"])
        Select(self.driver.find_element(*self.ROLE)).select_by_visible_text(row["role"])
        self.driver.find_element(*self.SUBMIT).click()
        WebDriverWait(self.driver, 3).until(
            lambda d: d.find_element(*self.RESULT).text != "ready"
        )
        return self.driver.find_element(*self.RESULT).text

5. config.py — explicit environment and a hard safety guard

The following example makes the config.py — explicit environment and a hard safety guard behavior concrete. Read it with the stated assumptions, then compare its observable output or state changes with the explanation that follows.

from dataclasses import dataclass
from urllib.parse import urlparse
import ipaddress, os


def require_loopback(url: str) -> str:
    parsed = urlparse(url)
    if parsed.scheme not in {"http", "https"} or not parsed.hostname:
        raise RuntimeError("SELENIUM_BASE_URL must be absolute http(s)")
    host = parsed.hostname
    if host == "localhost":
        return url
    try:
        address = ipaddress.ip_address(host)
    except ValueError as exc:
        raise RuntimeError("checkpoint refuses non-loopback host") from exc
    if not address.is_loopback:
        raise RuntimeError("checkpoint refuses non-loopback host")
    return url

@dataclass(frozen=True)
class Settings:
    browser: str
    environment: str
    explicit_base_url: str | None
    seed: str

    @classmethod
    def from_env(cls):
        raw_url = os.getenv("SELENIUM_BASE_URL")
        return cls(
            browser=os.getenv("SELENIUM_BROWSER", "chrome").lower(),
            environment=os.getenv("TEST_ENV", "local-checkpoint"),
            explicit_base_url=require_loopback(raw_url) if raw_url else None,
            seed=os.getenv("TEST_DATA_SEED", "chapter14-v1"),
        )

The environment name is metadata; it does not grant permission to target arbitrary hosts. The optional URL is independently validated. No “missing staging → production” fallback exists.

6. data_cases.py — deterministic synthetic identities

The following example makes the data_cases.py — deterministic synthetic identities behavior concrete. Read it with the stated assumptions, then compare its observable output or state changes with the explanation that follows.

from hashlib import sha256


def synthetic_email(seed: str, case_id: str) -> str:
    token = sha256(f"{seed}:{case_id}".encode()).hexdigest()[:12]
    return f"user-{token}@example.test"


def build_cases(seed: str):
    definitions = [
        ("viewer-valid", "Ava Example", "viewer"),
        ("editor-valid", "Omar Example", "editor"),
        ("admin-valid", "Lin Example", "admin"),
    ]
    return [
        {"id": case_id, "name": name, "role": role,
         "email": synthetic_email(seed, case_id)}
        for case_id, name, role in definitions
    ]

The email is deterministic for the same seed/case ID and remains synthetic. No random generator, timestamp, personal data, or shared account is required.

7. test_registration.py — isolated parameterized suite

The following example makes the test_registration.py — isolated parameterized suite behavior concrete. Read it with the stated assumptions, then compare its observable output or state changes with the explanation that follows.

from contextlib import contextmanager
from pathlib import Path
from urllib.request import Request, urlopen
import json, os, unittest, selenium
from selenium import webdriver

from app_fixture import running_app
from config import Settings, require_loopback
from data_cases import build_cases
from pages import RegistrationPage


def api_post(base_url, path):
    req = Request(base_url.rstrip("/") + path, data=b"{}",
                  headers={"Content-Type": "application/json"}, method="POST")
    with urlopen(req, timeout=2) as response:
        return json.loads(response.read())


def api_state(base_url):
    with urlopen(base_url.rstrip("/") + "/api/state", timeout=2) as response:
        return json.loads(response.read())


def new_driver(browser):
    if browser == "chrome": return webdriver.Chrome()
    if browser == "firefox": return webdriver.Firefox()
    raise ValueError(f"unsupported local teaching browser: {browser}")

@contextmanager
def case_fixture(base_url, browser):
    api_post(base_url, "/api/reset")
    driver = new_driver(browser)
    try:
        yield driver
    finally:
        try:
            driver.quit()
        finally:
            api_post(base_url, "/api/reset")

class RegistrationCheckpoint(unittest.TestCase):
    def test_parameterized_registration(self):
        settings = Settings.from_env()
        run_id = os.getenv("RUN_ID", "manual")
        with running_app() as fixture_url:
            # An explicit override is allowed only when it is still loopback.
            base_url = require_loopback(settings.explicit_base_url or fixture_url)
            for row in build_cases(settings.seed):
                with self.subTest(case=row["id"]):
                    with case_fixture(base_url, settings.browser) as driver:
                        page = RegistrationPage(driver, base_url)
                        page.open()
                        observed = page.submit(row)
                        self.assertEqual(observed, f"created:{row['email']}")
                        self.assertEqual(api_state(base_url)["users"], [{
                            "name": row["name"], "email": row["email"], "role": row["role"]
                        }])
                        out = Path("evidence") / run_id
                        out.mkdir(parents=True, exist_ok=True)
                        record = {
                            "case": row["id"], "email": row["email"], "role": row["role"],
                            "seed": settings.seed, "environment": settings.environment,
                            "selenium": selenium.__version__, "session_id": driver.session_id,
                            "browser": driver.capabilities.get("browserName"),
                            "browser_version": driver.capabilities.get("browserVersion"),
                            "url": driver.current_url, "result": observed,
                        }
                        (out / f"{row['id']}.json").write_text(json.dumps(record, indent=2), encoding="utf-8")
                        driver.save_screenshot(out / f"{row['id']}.png")
                    self.assertEqual(api_state(base_url)["users"], [])

if __name__ == "__main__":
    unittest.main(verbosity=2)

8. Run the same suite twice

The following example makes the Run the same suite twice behavior concrete. Read it with the stated assumptions, then compare its observable output or state changes with the explanation that follows.

# PowerShell
$env:RUN_ID="run1"; python -m unittest -v test_registration.py
$env:RUN_ID="run2"; python -m unittest -v test_registration.py

# POSIX equivalent:
# RUN_ID=run1 python -m unittest -v test_registration.py
# RUN_ID=run2 python -m unittest -v test_registration.py

Both runs should report three passing subtests. Session IDs and ephemeral fixture ports may differ; that difference is expected and proves fresh runtime state rather than nondeterminism.

9. compare_runs.py — compare deterministic business evidence

The following example makes the compare_runs.py — compare deterministic business evidence behavior concrete. Read it with the stated assumptions, then compare its observable output or state changes with the explanation that follows.

from pathlib import Path
import json

fields = ["case", "email", "role", "seed", "environment", "selenium", "browser", "result"]
for p1 in sorted(Path("evidence/run1").glob("*.json")):
    p2 = Path("evidence/run2") / p1.name
    a, b = json.loads(p1.read_text()), json.loads(p2.read_text())
    left = {k: a[k] for k in fields}
    right = {k: b[k] for k in fields}
    assert left == right, (p1.name, left, right)
    assert a["session_id"] != b["session_id"], "expected fresh WebDriver sessions"
    print(p1.stem, "repeatable", left["result"])

The following example makes the compare_runs.py — compare deterministic business evidence behavior concrete. Read it with the stated assumptions, then compare its observable output or state changes with the explanation that follows.

python compare_runs.py

Do not compare ephemeral port numbers or session IDs for equality. Repeatability means controlled inputs produce the same intended business outcomes while disposable runtime identities are fresh.

10. Prove the production-like target guard

The following example makes the Prove the production-like target guard behavior concrete. Read it with the stated assumptions, then compare its observable output or state changes with the explanation that follows.

# This must fail before any WebDriver session is created.
# PowerShell:
$env:SELENIUM_BASE_URL="https://prod.example.invalid/"
python -c "from config import Settings; print(Settings.from_env())"
Remove-Item Env:SELENIUM_BASE_URL

# POSIX:
# SELENIUM_BASE_URL=https://prod.example.invalid/ python -c 'from config import Settings; print(Settings.from_env())'

Expected outcome: RuntimeError: checkpoint refuses non-loopback host. The reserved .invalid name is used only as a rejected string; the guard prevents browser/network access.

11. Controlled failure and cleanup proof

Temporarily change one expected created: string to an incorrect value. Preserve the first traceback. After the failure, restore the expectation and rerun the single suite. The per-case finally path must still quit the browser and reset the in-memory AUT, so the corrected rerun begins clean without manual deletion.

12. Evidence packet

• Run 1 and Run 2 unittest output.
• Three sanitized JSON records per run: case ID, deterministic synthetic email/role/seed, environment name, Selenium/browser versions, session ID, URL, and result.
• One screenshot per case containing only the synthetic fixture.
• compare_runs.py output proving deterministic business fields and fresh session IDs.
• The rejected non-loopback guard traceback.
• One controlled assertion failure traceback plus the successful rerun.

No credentials, personal profiles, production data, tokens, or full environment dumps belong in this evidence packet.

13. Verification checklist

• Exactly three named cases are generated from the same recorded seed.
• Each case creates a fresh WebDriver session and exactly one AUT user.
• Cleanup returns /api/state to users=[] after every case, including a controlled failure.
• The application server binds only to loopback and shuts down at fixture exit.
• A non-loopback URL override is refused before browser creation.
• Run 1 and Run 2 have matching deterministic business evidence.
• Run 1 and Run 2 have different WebDriver session IDs.
• No fixed sleeps, blanket retries, JavaScript interaction bypasses, TLS disabling, or production experiments are used.

14. Cleanup and rollback

Ensure all browsers are closed, then delete selenium-ch14-checkpoint/ after preserving only the sanitized evidence required for review. The in-memory server and data vanish when the process exits; no external account, browser profile, database, Grid, or repository was mutated.

15. What Chapter 14 adds to the operating model

The automation platform now treats data and environment as controlled inputs: named parameters, deterministic synthetic identities, explicit target/browser configuration, per-case browser/AUT ownership, idempotent cleanup, production-target refusal, and sanitized provenance evidence. That architecture enables safe parallelization and meaningful reruns.

Chapter 15 builds directly on this foundation by defining assertion failure semantics, bounded retry policy, and systematic flaky-test control. Retries are only interpretable when the case, environment, and fixture state are already deterministic.

16. Summary

The checkpoint proved repeatability as an engineering property: same controlled inputs and business outcomes across two runs, fresh disposable runtime state each time, and deterministic cleanup after success or failure. That is the prerequisite for treating a CI failure as evidence rather than noise.

Knowledge check

Official references and version notes

• Selenium 4.47 release notes — stable binding/Grid baseline pinned for this chapter.
• Selenium downloads — current stable Selenium client and Server/Grid versions.
• Overview of Test Automation — keep browser setup/actions/evaluation small and use lower layers where they provide the right signal.
• Avoid sharing state — isolate test data and prefer a new WebDriver instance per test.
• Fresh browser per test — start from a clean known browser state.
• Test independency — do not make one scenario depend on another scenario's state.
• Generating application state — repetitive application setup is usually more stable through a lower-layer API than through browser UI.