Parallel Execution, Isolation, Concurrency, and Test Sharding: Configuration, Design Patterns, and Trade-Offs

1. Threads, processes, and CI jobs are scheduling mechanisms

Python threads are convenient for teaching I/O-heavy browser orchestration, but production frameworks may use processes, external workers, or separate CI jobs. Selenium does not require one runner topology. The invariant is simpler: each concurrent test gets exclusive ownership of the WebDriver/session and its mutable test state.

2. Per-test versus per-worker browser lifecycle

Selenium’s encouraged guidance favors a new WebDriver instance per test because it minimizes shared browser state. Reusing one browser across several tests can reduce startup cost, but it turns cookies, storage, windows, downloads, profile state, and browser crashes into shared worker state.

3. Shard by stable test identity, then rebalance intentionally

A simple hash shard is deterministic and cheap, but it may not balance wall time when a few tests are much slower. A larger suite can use historical timing to construct balanced shard manifests. If you do that, version the manifest/algorithm and preserve the original stable test IDs so a failure can still be traced to one owner.

# Example of a versioned explicit shard manifest produced by CI tooling.
SHARDS = {
    0: ["checkout.card", "profile.locale", "search.desktop"],
    1: ["checkout.invoice", "profile.mobile", "search.keyboard"],
}


def tests_for_shard(shard_index: int):
    return list(SHARDS[shard_index])

4. Maximum workers versus safe workers

The Node --max-sessions option limits browser sessions at the Grid layer. The runner’s worker count limits scheduling at the runner layer. Neither knows your AUT’s business-data bottleneck or artifact-store bandwidth. Find the knee of the throughput curve by measuring at a small sequence such as 1 → 2 → 3 workers, stopping when wall time no longer improves or errors/resource pressure appear.

5. Serial markers are a controlled exception, not a repair tool

Some scenarios are genuinely exclusive: a lab that rotates one global test certificate, validates a singleton maintenance banner, or exercises a device that cannot be virtualized. Put those tests in an explicit serial lane with a named reason. Do not mark tests serial merely because they share a badly designed account or filename; fix the isolation contract instead.

6. Keep Selenium, runner, Grid, AUT, and CI configuration in their own layers

Selenium options configure browser sessions. The runner configures workers/fixtures/reports. Grid configures Node slots, queueing, and routing. The AUT controls rate limits and fixture semantics. CI configures job count, matrix, workspace, and artifact aggregation. A worker-count problem is not repaired with a browser capability, and a Grid-slot problem is not repaired with a Selenium explicit wait.

7. Worked decision: eight runner workers, four Grid slots, three safe AUT identities

The initial safe ceiling is three concurrent scenarios because the AUT/test-data boundary is the smallest known capacity. If you can provision six isolated identities but Grid remains four slots, the ceiling becomes four. Only after measuring CPU/RAM and queue behavior should you consider adding Grid capacity or overriding Node defaults.

Knowledge check

Official references and current-version notes

• Selenium 4.47 release notes
• Selenium downloads — current stable client and Grid versions
• Avoid sharing state — Selenium test practices
• Test independency — Selenium test practices
• Getting started with Selenium Grid — capacity guidance
• Grid CLI options — max sessions and queue controls
• Python concurrent.futures — ThreadPoolExecutor