Parallel Execution, Isolation, Concurrency, and Test Sharding: Core Concepts and Mental Model

1. Why “run six tests at once” is not one operation

A runner can schedule six functions concurrently even when the machine can safely host only two browsers. A Grid can expose eight slots while the AUT can safely process only three test identities at once. An artifact store can serialize writes even when browsers are healthy. Parallelism therefore has several independent control planes.

2. The concurrency pipeline

flowchart TD
 I[Test inventory] --> S[Shard assignment]
 S --> W1[Worker A]
 S --> W2[Worker B]
 W1 --> D1[WebDriver session A]
 W2 --> D2[WebDriver session B]
 D1 --> G1[Grid slot / local browser]
 D2 --> G2[Grid slot / local browser]
 G1 --> A1[Isolated AUT data A]
 G2 --> A2[Isolated AUT data B]
 W1 --> E1[Evidence namespace A]
 W2 --> E2[Evidence namespace B]

The shard decides which tests belong to a job. The runner decides when those tests are scheduled. A WebDriver session is the browser-control state store and must have one clear owner. Grid decides whether a compatible slot exists. The AUT and fixture layer must give each test isolated data. Evidence must use a collision-proof namespace. These are distinct mechanisms even when one CI job configures all of them.

3. Objects and state stores that must remain distinct

The following table organizes the key choices and evidence for Objects and state stores that must remain distinct. Use it together with the surrounding prose so the rows serve as a comparison aid rather than standalone rules.

4. Read-only inspection before adding workers

Prove what environment you actually have. For a local session, record Selenium version, session ID, returned capabilities, CPU count, and current evidence root. For Grid, inspect /status, UI/GraphQL, Node max-session counts, and queue behavior before raising runner workers.

import os
import selenium
from selenium import webdriver

print("selenium", selenium.__version__)
print("runner logical CPUs", os.cpu_count())
driver = webdriver.Chrome()
try:
    print("session", driver.session_id)
    print("browser", driver.capabilities.get("browserName"), driver.capabilities.get("browserVersion"))
    print("platform", driver.capabilities.get("platformName"))
finally:
    driver.quit()

5. One concurrent unit, one WebDriver owner

The Selenium project explicitly recommends a new WebDriver instance per test because isolation makes parallelization simpler. Python does not provide Selenium Java’s ThreadGuard protection, so ownership is architectural: create the driver inside the worker/test that uses it, keep it local to that scope, and always quit() in a finally block.

from selenium import webdriver


def test_case(case_id):
    driver = webdriver.Chrome()   # created inside this test/worker
    try:
        driver.get("http://127.0.0.1:8765/work?case=" + case_id)
        # assertions belong to this owner
    finally:
        driver.quit()

6. Effective concurrency is the smallest safe capacity

A useful operating model is:

effective concurrency ≤ min(runner workers, available Grid/browser slots, safe AUT concurrency, isolated-data capacity, artifact/resource capacity)

This is not a mathematical guarantee; it is a capacity checklist. If the runner has eight workers, Grid has four slots, and the AUT safely tolerates only three synthetic users, a ceiling of three is the first candidate—not eight and not four.

7. Sharding and parallelism solve different problems

Sharding partitions a test inventory across jobs or machines. Parallelism schedules work inside one job. A deterministic shard function must return the same assignment for the same test ID, shard count, and algorithm version. Do not use Python’s built-in hash() as a cross-process persistent shard key because its randomized seed can change assignments between interpreter runs.

import hashlib


def stable_shard(test_id: str, shard_count: int) -> int:
    digest = hashlib.sha256(test_id.encode("utf-8")).digest()
    return int.from_bytes(digest[:8], "big") % shard_count

for test_id in ["checkout.en", "checkout.fa", "profile.mobile"]:
    print(test_id, stable_shard(test_id, 3))

8. DevOps connection: throughput is an observable system property

Record worker count, Grid slot state, session IDs, wall time, queue time if remote, browser versions, failed-test artifacts, and AUT/resource observations. A speedup smaller than one is still useful evidence: it tells you startup cost or contention dominates. The goal is reliable feedback per unit of infrastructure, not the largest thread count.

Knowledge check

Official references and current-version notes

• Selenium 4.47 release notes
• Selenium downloads — current stable client and Grid versions
• Avoid sharing state — Selenium test practices
• Test independency — Selenium test practices
• Getting started with Selenium Grid — capacity guidance
• Grid CLI options — max sessions and queue controls
• Python concurrent.futures — ThreadPoolExecutor