Checkpoint Lab — Test Architecture, Governance, Coding Standards, and Suite Evolution

1. Checkpoint goal

You will produce a governance package for a four-test synthetic portfolio. The package is a local set of reviewable files—not a new framework. It captures architecture boundaries, coding rules, ownership, flake/runtime budgets, update/deprecation policy, portfolio metrics, a risk-prioritized refactoring plan, and a runbook.

2. Preflight and assumptions

• Python 3.10+ is sufficient for the mandatory package generator.
• The current course baseline is Selenium 4.47.0; no live browser is required for mandatory package generation.
• If you add a live smoke rehearsal, use a supported local browser, Selenium Manager default resolution, a disposable AUT, and record returned capabilities.
• No production URLs/accounts, paid browser cloud, public Grid, real secrets, or destructive repository changes are allowed.

3. Predict before generating

Write two predictions before running the generator. The supplied checkpoint expects you to predict that retiring an obsolete journey reduces runtime without reducing supported product coverage, and that explicit ownership plus quarantine expiry makes instability accountable. You may add a third prediction about browser-matrix or update cadence.

4. Generate the governance package

The following example makes the Generate the governance package behavior concrete. Read it with the stated assumptions, then compare its observable output or state changes with the explanation that follows.

# file: build_governance_package.py
from pathlib import Path
import json

root = Path("governance-package")
root.mkdir(exist_ok=True)

inventory = [
    {"id": "checkout-happy", "owner": "commerce", "risk": 5, "runtime_s": 18, "flake_rate": 0.03, "maintenance": 3, "deprecated": False},
    {"id": "login-happy", "owner": "identity", "risk": 5, "runtime_s": 7, "flake_rate": 0.00, "maintenance": 1, "deprecated": False},
    {"id": "legacy-coupon", "owner": "commerce", "risk": 1, "runtime_s": 34, "flake_rate": 0.14, "maintenance": 4, "deprecated": True},
    {"id": "search-smoke", "owner": "catalog", "risk": 3, "runtime_s": 9, "flake_rate": 0.01, "maintenance": 1, "deprecated": False},
]

predictions = {
    "before": [
        "retiring the deprecated legacy-coupon test should reduce suite runtime without reducing supported product coverage",
        "adding explicit owners and quarantine expiry should make every unstable test accountable and time-bounded"
    ]
}
(root / "predictions.json").write_text(json.dumps(predictions, indent=2), encoding="utf-8")

architecture = '''# Selenium suite architecture\n\nTest intent -> Page/Component services -> fixtures + deterministic data -> WebDriver/Remote/Grid -> browser/AUT -> assertions -> evidence/reporting.\n\nRules:\n- tests own business assertions\n- fixtures own session lifecycle\n- page/component objects own UI locators and services, not WebDriver creation/quit\n- diagnostics observe failures without replacing the original exception\n- CI/Grid infrastructure stays outside test intent\n'''
(root / "architecture.md").write_text(architecture, encoding="utf-8")

checklist = '''# Coding and review checklist\n- [ ] stable semantic locators or test attributes\n- [ ] no fixed sleep as primary synchronization\n- [ ] one explicit WebDriver owner and failure-safe quit\n- [ ] isolated test data/profile/download/evidence paths\n- [ ] assertions express business/UI outcome\n- [ ] first-failure evidence is correlated and redacted\n- [ ] owner and risk tier are declared\n- [ ] no production target, real secret, MFA/CAPTCHA bypass, or public Grid endpoint\n- [ ] Selenium/browser version evidence is recorded\n'''
(root / "coding-review-checklist.md").write_text(checklist, encoding="utf-8")

ownership = {item["id"]: item["owner"] for item in inventory}
(root / "ownership.json").write_text(json.dumps(ownership, indent=2), encoding="utf-8")

budgets = {
    "suite_runtime_minutes": 10,
    "flake_rate_max": 0.02,
    "quarantine_expiry_days": 14,
    "critical_smoke_browser_tier": ["chrome"],
    "nightly_browser_tier": ["chrome", "firefox"],
}
(root / "budgets.json").write_text(json.dumps(budgets, indent=2), encoding="utf-8")

(root / "update-policy.md").write_text(
    "# Dependency and browser update policy\n\nMonthly: review Selenium stable release and supported runtime/browser changes. Rehearse upgrades in an isolated environment, record returned capabilities and browser/driver versions, run critical smoke first, then broad scheduled matrix. Promote only after evidence review; keep rollback/pin available.\n",
    encoding="utf-8",
)
(root / "deprecation-policy.md").write_text(
    "# Deprecation policy\n\nA test may be retired when the product flow is removed or its risk is covered at a more appropriate layer. Retirement requires owner approval, reason, replacement/coverage note, and deletion date. Quarantine is temporary and must not become permanent storage for unexplained failures.\n",
    encoding="utf-8",
)

metrics = {
    "tests": len(inventory),
    "total_runtime_s": sum(i["runtime_s"] for i in inventory),
    "ownerless": sum(not i["owner"] for i in inventory),
    "over_flake_budget": [i["id"] for i in inventory if i["flake_rate"] > budgets["flake_rate_max"]],
    "deprecated_runtime_s": sum(i["runtime_s"] for i in inventory if i["deprecated"]),
}
(root / "metrics-report.json").write_text(json.dumps(metrics, indent=2), encoding="utf-8")

plan = []
for item in inventory:
    reliability = max(item["flake_rate"] / max(budgets["flake_rate_max"], 0.001), 1)
    obsolete_penalty = 3 if item["deprecated"] else 1
    score = round(item["risk"] * reliability * item["maintenance"] * obsolete_penalty, 2)
    action = "retire with owner approval" if item["deprecated"] else "stabilize/refactor"
    plan.append({"id": item["id"], "priority": score, "action": action})
plan.sort(key=lambda x: x["priority"], reverse=True)
(root / "refactoring-plan.json").write_text(json.dumps(plan, indent=2), encoding="utf-8")

runbook = '''# Governance runbook\n1. preserve first-failure evidence\n2. classify product/test/browser/Grid/CI layer\n3. check owner, risk tier, flake/runtime budget, and version evidence\n4. repair root cause or open time-bounded quarantine with owner\n5. rehearse dependency/browser changes in isolation\n6. retire obsolete coverage with explicit replacement/approval\n7. review portfolio metrics monthly\n'''
(root / "governance-runbook.md").write_text(runbook, encoding="utf-8")

required = {
    "architecture.md", "coding-review-checklist.md", "ownership.json", "budgets.json",
    "update-policy.md", "deprecation-policy.md", "metrics-report.json", "refactoring-plan.json",
    "governance-runbook.md", "predictions.json"
}
actual = {p.name for p in root.iterdir() if p.is_file()}
missing = sorted(required - actual)
if missing:
    raise SystemExit(f"missing governance artifacts: {missing}")
print("package files:", len(actual))
print("over flake budget:", metrics["over_flake_budget"])
print("deprecated runtime seconds:", metrics["deprecated_runtime_s"])
print("top refactor:", plan[0])

Run the file in a disposable directory. It creates ten governance artifacts and verifies that none are missing. The synthetic metrics should flag checkout-happy and legacy-coupon above the 2% flake budget; it should also report 34 seconds of runtime attached to a deprecated flow.

5. Architecture diagram requirement

The following diagram visualizes the relationships described in Architecture diagram requirement. Read the nodes in sequence and use the arrows to connect the conceptual state changes to the explanation around the diagram.

flowchart TB
  T[Test intent + business assertions] --> P[Page / Component services]
  T --> D[Fixtures + synthetic data]
  P --> W[WebDriver / RemoteWebDriver]
  D --> W
  W --> G[Local browser or private Grid]
  G --> A[Authorized AUT]
  T --> E[Diagnostics + evidence]
  G --> E
  E --> M[Portfolio metrics]
  M --> R[Review / upgrade / quarantine / deprecation]

Every arrow has an ownership meaning. Tests express outcomes; page/components expose UI services; fixtures create isolated state; WebDriver/Grid executes browser actions; diagnostics observe results; metrics summarize portfolio health; and governance decisions feed changes back into the suite.

6. Verify the coding/review checklist

Open coding-review-checklist.md and confirm it covers stable locators, no fixed sleep as default synchronization, explicit session ownership, isolated state/evidence, meaningful assertions, first-failure evidence, owner/risk metadata, safe-target/secret rules, and runtime version evidence. Add any repository-specific style rule only after these behavior rules are preserved.

7. Verify ownership and budgets

ownership.json must map every test to a domain owner. budgets.json defines suite runtime, maximum flake rate, quarantine expiry, and browser cadence. A budget is an alert/control threshold, not permission to hide failures that exceed it.

8. Verify update and deprecation policy

The update policy should say how a candidate Selenium/browser change moves from isolated rehearsal → critical smoke → broad matrix → promotion, with version/capability evidence and rollback. The deprecation policy must require product/coverage reasoning and owner approval rather than deleting slow tests by fiat.

9. Interpret the metrics report

The checkpoint computes total synthetic runtime, ownerless count, tests above the flake budget, and deprecated runtime. These values are a compact portfolio snapshot. In production, add trend windows, CI/job context, browser tier, failure classification quality, and maintenance effort instead of treating one run as a permanent verdict.

10. Review the risk-prioritized refactoring plan

The generated score deliberately combines business risk, reliability pressure, maintenance cost, and obsolescence. Review whether the ordering makes sense. A deprecated low-risk flow can still rise because it creates avoidable cost; a critical unstable flow rises because it threatens release confidence. Human review remains required.

11. Independent verification checklist

• Exactly ten required package files exist.
• No owner entry is blank.
• At least one test exceeds the synthetic flake budget.
• The deprecated test contributes nonzero runtime.
• The update policy records versions/capabilities before promotion.
• Quarantine has an owner and expiry.
• Deprecation requires coverage reasoning.
• The runbook preserves first-failure evidence before policy changes.

12. Optional live upgrade rehearsal evidence

If Selenium 4.47.0 and a supported local browser are available, run the earlier browser-baseline probe in an isolated environment and attach browser-baseline.json to the package. The point is to record the actual binding/browser/driver/session environment. Do not treat a successful data-URL smoke as evidence that the entire product suite is compatible.

13. Failure injection: blocked governance shortcuts

Simulate one policy exception on paper: “critical flaky test has no owner and someone requests permanent retry.” The correct decision is to block permanent normalization, assign an owner, preserve evidence, diagnose the layer, and if necessary create a temporary visible quarantine with expiry. No browser/Grid restart or production experiment is required.

14. Cleanup and rollback

Delete only the disposable governance-package directory after you have reviewed its artifacts. If you created an isolated virtual environment for an optional upgrade rehearsal, remove that environment separately. Do not change the system browser, global Python installation, production Grid, or repository branch as part of cleanup.

15. What this adds to the production operating model

Chapter 29 adds the organizational feedback loop around all previous Selenium mechanics: who owns browser risk, which architecture boundaries are enforced, how instability/runtime/version drift are measured, how upgrades are rehearsed, and how obsolete coverage leaves the portfolio. Chapter 30 will use this governance layer to assemble and operate a production cross-browser automation platform end to end.

Official references and current-version notes

• Selenium downloads — Stable Selenium clients and Selenium Server/Grid 4.47.0, released August 10, 2026.
• Encouraged testing behaviors — Selenium explicitly frames these as guidelines/recommendations rather than universal best practices.
• Avoid sharing state — Current guidance to isolate test data and create a new WebDriver instance per test.
• Page object models — Current Selenium guidance on clean separation, centralized page services/locators, and keeping business assertions in tests.
• Selenium Manager — Official default driver/browser management path used by modern Selenium bindings when drivers are not explicitly supplied.
• Grid security — Current warning that Grid must be protected from external/public access.
• Grid CLI options — Current Grid configuration surface to re-check during platform/version policy reviews.

Knowledge checks

Summary and next bridge

A maintainable Selenium suite is a governed portfolio: explicit ownership, architecture boundaries, isolated state, measurable reliability/runtime, version evidence, review rules, and a deliberate upgrade/quarantine/deprecation lifecycle.

Next: Chapter 30 — Capstone: Build and Operate a Production Cross-Browser Automation Platform