Screenshots, Logs, Network Evidence, and Failure Diagnostics: Guided Hands-On Workflow

1. Preflight and disposable workspace

Use Python 3.10+, Selenium Python 4.47.0, and a locally installed supported Chromium-family browser. Selenium Manager resolves the normal local driver path. The AUT binds only to 127.0.0.1:8781; the secret string is intentionally fake; no real account, proxy, TLS override, or external service is used.

python -m venv .venv
# Windows PowerShell: .\.venv\Scripts\Activate.ps1
# macOS/Linux: source .venv/bin/activate
python -m pip install "selenium==4.47.0"

2. Generate the local fixture

Save the following as make_fixture.py and run it once. It writes an HTML fixture plus a tiny Python server. The server records its own raw synthetic request log so the network-like case has evidence independent of the browser.

from pathlib import Path

ROOT = Path("chapter17_fixture")
SITE = ROOT / "site"
SITE.mkdir(parents=True, exist_ok=True)

(SITE / "index.html").write_text(r'''<!doctype html>
<html lang="en">
<head>
  <meta charset="utf-8">
  <meta name="viewport" content="width=device-width, initial-scale=1">
  <title>Evidence Engineering Lab</title>
  <style>
    body { font-family: system-ui, sans-serif; margin: 0; background: #10151c; color: #eef4fb; }
    main { max-width: 900px; margin: 0 auto; padding: 28px; }
    .panel { border: 1px solid #415166; border-radius: 14px; padding: 18px; margin: 16px 0; }
    button { margin: 5px; padding: 10px 14px; }
    code { overflow-wrap: anywhere; }
  </style>
</head>
<body>
<main>
  <h1 data-testid="heading">Evidence Engineering Lab</h1>
  <p data-testid="test-id"></p>
  <div class="panel">
    <button data-testid="healthy" type="button">Healthy path</button>
    <button data-testid="app-error" type="button">Application error</button>
    <button data-testid="js-error" type="button">Client JavaScript error</button>
    <button data-testid="network-error" type="button">Network-like failure</button>
  </div>
  <div class="panel">
    <strong>Status</strong>
    <p data-testid="status" data-state="idle">idle</p>
    <strong>Correlation</strong>
    <code data-testid="correlation"></code>
  </div>
</main>
<script>
const params = new URLSearchParams(location.search);
const testId = params.get('test_id') || 'manual';
const syntheticSecret = 'demo-secret-123';
const q = id => document.querySelector(`[data-testid="${id}"]`);
q('test-id').textContent = `test_id=${testId}`;
q('correlation').textContent = testId;

function setState(state, text) {
  q('status').dataset.state = state;
  q('status').textContent = text;
}

q('healthy').addEventListener('click', () => setState('ok', 'healthy'));
q('app-error').addEventListener('click', () => {
  setState('app-error', 'synthetic application validation failed');
  console.warn(`app validation rejected test_id=${testId}`);
});
q('js-error').addEventListener('click', () => {
  setState('js-triggered', 'client exception triggered');
  queueMicrotask(() => {
    throw new Error(`synthetic-client-error test_id=${testId} token=${syntheticSecret}`);
  });
});
q('network-error').addEventListener('click', async () => {
  setState('network-pending', 'request in flight');
  try {
    await fetch(`/api/drop?test_id=${encodeURIComponent(testId)}&token=${encodeURIComponent(syntheticSecret)}`,
      {headers: {'X-Test-Id': testId}});
    setState('unexpected-success', 'unexpected response');
  } catch (error) {
    console.error(`synthetic-network-error test_id=${testId} token=${syntheticSecret}`, error);
    setState('network-error', 'synthetic transport failed');
  }
});
</script>
</body>
</html>''', encoding="utf-8")

(ROOT / "serve.py").write_text(r'''from http.server import ThreadingHTTPServer, SimpleHTTPRequestHandler
from pathlib import Path
from urllib.parse import urlsplit, parse_qs
from datetime import datetime, timezone
import json
import socket

ROOT = Path(__file__).resolve().parent
SITE = ROOT / "site"
RAW_LOG = ROOT / "server-raw.jsonl"
HOST, PORT = "127.0.0.1", 8781

class Handler(SimpleHTTPRequestHandler):
    def translate_path(self, path):
        raw = super().translate_path(path)
        rel = Path(raw).relative_to(Path.cwd())
        return str(SITE / rel)

    def log_message(self, format, *args):
        pass

    def _record(self, outcome):
        parts = urlsplit(self.path)
        row = {
            "ts": datetime.now(timezone.utc).isoformat(),
            "method": self.command,
            "path": parts.path,
            "query": parse_qs(parts.query),
            "x_test_id": self.headers.get("X-Test-Id"),
            "outcome": outcome,
        }
        with RAW_LOG.open("a", encoding="utf-8") as fh:
            fh.write(json.dumps(row) + "\n")

    def do_GET(self):
        if self.path.startswith("/api/drop"):
            self._record("connection-dropped")
            try:
                self.connection.shutdown(socket.SHUT_RDWR)
            except OSError:
                pass
            self.connection.close()
            return
        self._record("static-response")
        return super().do_GET()

server = ThreadingHTTPServer((HOST, PORT), Handler)
print(f"Evidence fixture: http://{HOST}:{PORT}/")
try:
    server.serve_forever()
except KeyboardInterrupt:
    pass
finally:
    server.server_close()
''', encoding="utf-8")

print(ROOT.resolve())

The following example makes the Generate the local fixture behavior concrete. Read it with the stated assumptions, then compare its observable output or state changes with the explanation that follows.

python make_fixture.py
python chapter17_fixture/serve.py
# Expected: Evidence fixture: http://127.0.0.1:8781/

Keep the server terminal visible. Opening the URL manually should show four buttons and an idle status.

3. Understand what each control changes

The following table organizes the key choices and evidence for Understand what each control changes. Use it together with the surrounding prose so the rows serve as a comparison aid rather than standalone rules.

4. Build a privacy-aware first-failure collector

Save the next program as guided_capture.py. It configures Chromium browser logging explicitly, but still checks driver.log_types. It never stores the fixture’s fake token in textual evidence: query parameters and log messages are sanitized before persistence.

from pathlib import Path
from datetime import datetime, timezone
from urllib.parse import urlsplit, parse_qsl, urlencode, urlunsplit
import json
import re
import selenium
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait

BASE_URL = "http://127.0.0.1:8781/"
SECRET = "demo-secret-123"
ARTIFACTS = Path("evidence")
ARTIFACTS.mkdir(exist_ok=True)


def redact_text(value):
    text = str(value)
    text = text.replace(SECRET, "<redacted>")
    text = re.sub(r"(?i)(authorization:\\s*bearer\\s+)[^\\s]+", r"\\1<redacted>", text)
    text = re.sub(r"(?i)(token=)[^&\\s]+", r"\\1<redacted>", text)
    return text


def redact_url(url):
    parts = urlsplit(url)
    pairs = []
    for key, value in parse_qsl(parts.query, keep_blank_values=True):
        pairs.append((key, "<redacted>" if key.lower() in {"token", "secret", "key"} else value))
    return urlunsplit((parts.scheme, parts.netloc, parts.path, urlencode(pairs), parts.fragment))


def safe_caps(caps):
    keep = ["browserName", "browserVersion", "platformName", "pageLoadStrategy"]
    return {key: caps.get(key) for key in keep if key in caps}


def read_browser_logs(driver):
    available = set(getattr(driver, "log_types", []))
    if "browser" not in available:
        return {"available": sorted(available), "entries": [], "note": "browser log type unavailable"}
    entries = []
    for row in driver.get_log("browser"):
        entries.append({k: redact_text(v) for k, v in row.items()})
    return {"available": sorted(available), "entries": entries}


def capture(driver, test_id, label):
    stamp = datetime.now(timezone.utc).strftime("%Y%m%dT%H%M%S.%fZ")
    root = ARTIFACTS / test_id / f"{stamp}-{label}"
    root.mkdir(parents=True, exist_ok=False)

    driver.save_screenshot(str(root / "viewport.png"))
    status = driver.find_element(By.CSS_SELECTOR, '[data-testid="status"]')
    dom = {
        "status_text": status.text,
        "status_state": status.get_attribute("data-state"),
        "correlation": driver.find_element(By.CSS_SELECTOR, '[data-testid="correlation"]').text,
    }
    (root / "dom.json").write_text(json.dumps(dom, indent=2), encoding="utf-8")

    metadata = {
        "captured_at": stamp,
        "test_id": test_id,
        "label": label,
        "selenium": selenium.__version__,
        "session_id": driver.session_id,
        "url": redact_url(driver.current_url),
        "title": driver.title,
        "capabilities": safe_caps(driver.capabilities),
    }
    (root / "metadata.json").write_text(json.dumps(metadata, indent=2), encoding="utf-8")

    logs = read_browser_logs(driver)
    (root / "browser-log.json").write_text(json.dumps(logs, indent=2), encoding="utf-8")

    resources = driver.execute_script("""
      return performance.getEntriesByType('resource').map(e => ({
        name: e.name, initiatorType: e.initiatorType, duration: e.duration
      }));
    """)
    sanitized = [{**row, "name": redact_url(row["name"])} for row in resources]
    (root / "resource-timing.json").write_text(json.dumps(sanitized, indent=2), encoding="utf-8")
    return root


options = webdriver.ChromeOptions()
# Chromium-specific logging capability. Check returned log_types; do not assume parity in every browser.
options.set_capability("goog:loggingPrefs", {"browser": "ALL"})
driver = webdriver.Chrome(options=options)
try:
    test_id = "guided-js-error"
    driver.get(f"{BASE_URL}?test_id={test_id}")
    print("selenium", selenium.__version__)
    print("session", driver.session_id)
    print("capabilities", safe_caps(driver.capabilities))
    print("log_types", driver.log_types)
    print("before", driver.current_url, driver.title)

    driver.find_element(By.CSS_SELECTOR, '[data-testid="js-error"]').click()
    WebDriverWait(driver, 3).until(
        lambda d: d.find_element(By.CSS_SELECTOR, '[data-testid="status"]').get_attribute("data-state") == "js-triggered"
    )
    bundle = capture(driver, test_id, "first-failure")
    print("bundle", bundle)
finally:
    driver.quit()

5. Run and inspect causality

The following example makes the Run and inspect causality behavior concrete. Read it with the stated assumptions, then compare its observable output or state changes with the explanation that follows.

python guided_capture.py
# Inspect:
# evidence/guided-js-error/<UTC>-first-failure/
#   viewport.png
#   dom.json
#   metadata.json
#   browser-log.json
#   resource-timing.json

Before the click, the script prints the negotiated browser version, platform, session ID, available log types, URL, and title. After the click, it waits on the fixture’s explicit data-state—not on a sleep—and captures before quit(). That ordering is the causal contract.

6. Interpret each file without overclaiming

The following table organizes the key choices and evidence for Interpret each file without overclaiming. Use it together with the surrounding prose so the rows serve as a comparison aid rather than standalone rules.

7. Verify redaction independently

Search textual artifacts for the fake secret. The correct result is no match.

from pathlib import Path

secret = "demo-secret-123"
leaks = []
for path in Path("evidence").rglob("*"):
    if path.is_file() and path.suffix in {".json", ".txt", ".md"}:
        if secret in path.read_text(encoding="utf-8"):
            leaks.append(str(path))
assert not leaks, leaks
print("redaction verified")

This check is intentionally independent of the collector. Security controls need verification, not just intent.

8. Optional preview: stream console and JavaScript errors with WebDriver BiDi

Current Selenium exposes high-level BiDi handlers through the script namespace when BiDi is enabled in browser options. This is a preview only; Chapter 18 develops event subscriptions, network events, contexts, lifecycle, and portability in depth.

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait

options = webdriver.ChromeOptions()
options.enable_bidi = True
driver = webdriver.Chrome(options=options)
messages = []
errors = []
try:
    driver.get("http://127.0.0.1:8781/?test_id=bidi-preview")
    console_id = driver.script.add_console_message_handler(messages.append)
    error_id = driver.script.add_javascript_error_handler(errors.append)

    driver.find_element(By.CSS_SELECTOR, '[data-testid="js-error"]').click()
    WebDriverWait(driver, 3).until(lambda _: errors)
    print("console events", len(messages))
    print("javascript errors", [entry.text for entry in errors])

    driver.script.remove_console_message_handler(console_id)
    driver.script.remove_javascript_error_handler(error_id)
finally:
    driver.quit()

The handler IDs matter: subscriptions are state. Remove handlers when their scenario ends so later tests do not inherit duplicate listeners or misleading events.

9. Network evidence: collect only what the hypothesis needs

The fixture’s network-like failure is intentionally diagnosed with three perspectives: the AUT reports network-error, the browser may report a fetch/console failure, and the server raw log proves that /api/drop was reached before the connection was closed. That is stronger than a screenshot and safer than indiscriminately storing every request header/body.

10. What changes on Grid or CI?

The browser may be remote while the evidence bundle is written on the test runner. Preserve returned session ID, browser/platform capabilities, Grid job correlation if available, and the CI attempt/shard. Never assume a node-local driver log is inside the runner workspace. Hosted vendor videos/HARs can be useful optional evidence, but the free local path remains complete.

11. Challenge: choose the evidence control, not a memorized sequence

Change the guided case from js-error to app-error. Before running, predict which artifacts should change and which should not. Your design should not require a browser console error to classify an application/business failure. Then switch to network-error and decide whether resource timing alone is sufficient. Explain your answer using independent layers.

12. Cleanup

The following example makes the Cleanup behavior concrete. Read it with the stated assumptions, then compare its observable output or state changes with the explanation that follows.

# Stop serve.py with Ctrl+C first.
# Keep evidence only if you intentionally want the synthetic lab artifacts.
python - <<'PY'
from pathlib import Path
import shutil
for p in [Path("chapter17_fixture"), Path("evidence")]:
    if p.exists(): shutil.rmtree(p)
print("lab workspace removed")
PY

WebDriver quit() ends the browser session. No personal profile, real credentials, production data, or external service was used.

13. Summary and bridge

You now have a causal capture order and a redacted evidence schema. Lesson 3 turns those mechanics into policy choices: when to capture, which screenshot/DOM/log/network level to retain, how to control cost, and when classic logs should give way to BiDi event streams.

Knowledge check

Official references and version notes

• Selenium 4.47 release notes — stable binding/Grid baseline pinned for this chapter.
• Selenium downloads — current stable clients and Selenium Server/Grid.
• Selenium Python 4.47 API — supported Python versions and WebDriver API surface.
• Chromium WebDriver API — get_log, log_types, screenshot APIs, and Chromium-specific capabilities.
• WebDriver BiDi — W3C bidirectional event direction and enablement.
• WebDriver BiDi logging — current console-message and JavaScript-error handlers.
• WebDriver BiDi network — current network handler concepts; Chapter 18 covers these APIs in depth.
• Selenium Grid — remote session and node boundary.
• Grid getting started — local/free Grid execution and version alignment.
• Docker Selenium — official containerized Grid distribution; optional in this chapter.