Shadow DOM, Web Components, Dynamic DOMs, and Modern Front Ends: Guided Hands-On Workflow

1. Setup and preflight

Create an empty directory such as selenium-ch10-lab. Use Python 3.10+ and an isolated environment. The mandatory path pins Selenium 4.47.0 and one supported local Chromium-family browser.

python -m venv .venv
# Linux/macOS
. .venv/bin/activate
# Windows PowerShell alternative:
# .\.venv\Scripts\Activate.ps1
python -m pip install --upgrade pip
python -m pip install selenium==4.47.0
python -c "import selenium; print(selenium.__version__)"

2. Generate and serve the custom-element fixture

The following example makes the Generate and serve the custom-element fixture behavior concrete. Read it with the stated assumptions, then compare its observable output or state changes with the explanation that follows.

from pathlib import Path

root = Path("site")
root.mkdir(exist_ok=True)
(root / "frame.html").write_text("""<!doctype html><html lang="en"><head><meta charset="utf-8"><title>Frame Fixture</title></head><body><p id="frame-value">frame:ready</p></body></html>""", encoding="utf-8")
(root / "index.html").write_text("""<!doctype html>
<html lang="en">
<head><meta charset="utf-8"><title>Shadow Component Lab</title>
<style>
body{font-family:system-ui,sans-serif;max-width:850px;margin:2rem auto;padding:0 1rem}button,input{padding:.45rem .65rem;margin:.25rem}.panel{border:1px solid #8886;border-radius:.6rem;padding:1rem;margin:1rem 0}iframe{width:100%;height:90px}
</style></head>
<body>
<h1 id="page-title">Shadow Component Lab</h1>
<p id="app-result">app:idle</p>
<profile-card id="profile-card" data-testid="profile-card"></profile-card>
<div id="ordinary" class="panel"><button data-testid="ordinary-action">Ordinary action</button><span data-testid="ordinary-status">ordinary:ready</span></div>
<iframe id="comparison-frame" src="frame.html" title="Comparison frame"></iframe>
<delayed-widget id="delayed-widget"></delayed-widget>
<closed-card id="closed-card" data-ready="false"></closed-card>
<script>
class ProfileCard extends HTMLElement {
  constructor(){
    super();
    this.attachShadow({mode:'open'});
    this.version = 0;
    this.actions = 0;
  }
  connectedCallback(){ this.render(); }
  render(){
    this.version += 1;
    this.setAttribute('data-version', String(this.version));
    this.shadowRoot.innerHTML = `
      <section data-testid="profile-panel">
        <label>Nickname <input data-testid="nickname" value="learner-${this.version}"></label>
        <button data-testid="advance">Advance component</button>
        <span data-testid="status">version:${this.version};actions:${this.actions}</span>
      </section>`;
    this.shadowRoot.querySelector('[data-testid="advance"]').addEventListener('click', () => {
      this.actions += 1;
      document.querySelector('#app-result').textContent = `app:advanced:${this.actions}`;
      this.render();
    });
  }
}
customElements.define('profile-card', ProfileCard);

document.querySelector('[data-testid="ordinary-action"]').addEventListener('click', () => {
  document.querySelector('[data-testid="ordinary-status"]').textContent = 'ordinary:clicked';
});

class DelayedWidget extends HTMLElement {
  connectedCallback(){
    setTimeout(() => {
      const root = this.attachShadow({mode:'open'});
      root.innerHTML = '<span data-testid="hydrated">hydrated:ready</span>';
      this.setAttribute('data-ready','true');
    }, 450);
  }
}
customElements.define('delayed-widget', DelayedWidget);

class ClosedCard extends HTMLElement {
  constructor(){
    super();
    const root = this.attachShadow({mode:'closed'});
    root.innerHTML = '<span>private implementation</span>';
  }
  connectedCallback(){
    this.setAttribute('data-ready','true');
    this.setAttribute('aria-label','Closed component ready');
  }
}
customElements.define('closed-card', ClosedCard);
</script></body></html>""", encoding="utf-8")
print(root.resolve())

Save as make_fixture.py, then:

The following example makes the Generate and serve the custom-element fixture behavior concrete. Read it with the stated assumptions, then compare its observable output or state changes with the explanation that follows.

python make_fixture.py
python -m http.server 8775 --bind 127.0.0.1 --directory site

The main profile-card has an open root. Every render increments the host's data-version and replaces the internal controls. The page also contains ordinary DOM, an iframe, a delayed open-root component, and a closed component whose public host state is observable.

3. Read the open component before mutating it

The following example makes the Read the open component before mutating it behavior concrete. Read it with the stated assumptions, then compare its observable output or state changes with the explanation that follows.

from selenium import webdriver
from selenium.webdriver.common.by import By

URL = "http://127.0.0.1:8775/index.html"
driver = webdriver.Chrome()
try:
    driver.get(URL)
    host = driver.find_element(By.ID, "profile-card")
    root = host.shadow_root
    nickname = root.find_element(By.CSS_SELECTOR, '[data-testid="nickname"]')
    advance = root.find_element(By.CSS_SELECTOR, '[data-testid="advance"]')
    status = root.find_element(By.CSS_SELECTOR, '[data-testid="status"]')
    print("session", driver.session_id)
    print("version", host.get_attribute("data-version"))
    print("nickname", nickname.get_property("value"))
    print("status", status.text)
finally:
    driver.quit()

The host lookup reads the document. host.shadow_root reads the component boundary. The descendant lookups read the root. No application state has changed yet.

4. Trigger rerender and prove the old node reference is stale

Now keep an old internal reference deliberately. Clicking advance updates application state and causes render() to replace every internal node.

from selenium import webdriver
from selenium.common.exceptions import StaleElementReferenceException
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait

URL = "http://127.0.0.1:8775/index.html"
driver = webdriver.Chrome()
wait = WebDriverWait(driver, 5)
try:
    driver.get(URL)
    host = driver.find_element(By.ID, "profile-card")
    old_root = host.shadow_root
    old_button = old_root.find_element(By.CSS_SELECTOR, '[data-testid="advance"]')
    before = int(host.get_attribute("data-version"))

    old_button.click()
    wait.until(lambda d: int(d.find_element(By.ID, "profile-card").get_attribute("data-version")) == before + 1)

    try:
        print("old text", old_button.text)
        raise AssertionError("Expected old internal node to be stale")
    except StaleElementReferenceException as exc:
        print("expected", type(exc).__name__)

    live_host = driver.find_element(By.ID, "profile-card")
    live_root = live_host.shadow_root
    live_status = live_root.find_element(By.CSS_SELECTOR, '[data-testid="status"]')
    print("after", live_host.get_attribute("data-version"), live_status.text)
finally:
    driver.quit()

The wait observes the host version transition. Reacquisition begins again at the stable host contract, so the test gets a current root and current descendants instead of retrying the detached button object.

5. Rerender a second time without caching the internal control

A maintainable interaction helper stores locators/expected state, not long-lived dynamic nodes.

from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait

HOST = (By.ID, "profile-card")
ADVANCE = (By.CSS_SELECTOR, '[data-testid="advance"]')
STATUS = (By.CSS_SELECTOR, '[data-testid="status"]')

def current_component(driver):
    host = driver.find_element(*HOST)
    return host, host.shadow_root

def advance_once(driver, wait):
    host, root = current_component(driver)
    before = int(host.get_attribute("data-version"))
    root.find_element(*ADVANCE).click()
    wait.until(lambda d: int(d.find_element(*HOST).get_attribute("data-version")) == before + 1)
    host2, root2 = current_component(driver)
    return host2.get_attribute("data-version"), root2.find_element(*STATUS).text

Call advance_once twice in the same session. Each call reacquires live state and returns evidence of the new version.

6. Ordinary nested DOM uses the same document search context

The “ordinary” panel has no shadow boundary. A scoped parent WebElement is enough:

ordinary = driver.find_element(By.ID, "ordinary")
ordinary.find_element(By.CSS_SELECTOR, '[data-testid="ordinary-action"]').click()
assert ordinary.find_element(By.CSS_SELECTOR, '[data-testid="ordinary-status"]').text == "ordinary:clicked"

This is still descendant lookup, but no ShadowRoot reference is involved.

7. Iframe comparison: this one really changes browsing context

The following example makes the Iframe comparison: this one really changes browsing context behavior concrete. Read it with the stated assumptions, then compare its observable output or state changes with the explanation that follows.

frame = driver.find_element(By.ID, "comparison-frame")
driver.switch_to.frame(frame)
try:
    assert driver.find_element(By.ID, "frame-value").text == "frame:ready"
finally:
    driver.switch_to.default_content()

assert driver.find_element(By.ID, "profile-card").is_displayed()

The frame's document cannot be searched until WebDriver switches browsing context. Shadow-root traversal does not use this API.

8. Hydration/readiness: host presence is not component readiness

The delayed widget exists in the document before it attaches its open root. A fixed sleep would guess at timing. Use a bounded condition that re-checks the real contract.

from selenium.common.exceptions import NoSuchElementException, NoSuchShadowRootException, StaleElementReferenceException
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait

def shadow_child(host_locator, child_locator):
    def _ready(driver):
        try:
            host = driver.find_element(*host_locator)
            if host.get_attribute("data-ready") != "true":
                return False
            return host.shadow_root.find_element(*child_locator)
        except (NoSuchElementException, NoSuchShadowRootException, StaleElementReferenceException):
            return False
    return _ready

hydrated = WebDriverWait(driver, 5).until(
    shadow_child((By.ID, "delayed-widget"), (By.CSS_SELECTOR, '[data-testid="hydrated"]'))
)
assert hydrated.text == "hydrated:ready"

The contract is “host reports ready and expected child is findable inside its root,” not “450 ms probably elapsed.”

9. Challenge: choose the correct control, do not copy a sequence

After an action, the host's data-version has advanced, an old internal button raises stale, the iframe is not involved, and the current URL is unchanged. Which control should the test use next?

• A. switch_to.default_content() and retry the same old button.
• B. Add a 5-second sleep and reuse the old ShadowRoot.
• C. Reacquire the host, obtain its current ShadowRoot, then locate the current button/status.
• D. Inject JavaScript to query through the shadow boundary.

Correct reasoning: C. The observed transition is node replacement inside the component, not a frame/context change or timing mystery.

10. Verification and cleanup

• Host/root/descendant lookup worked without JavaScript.
• First rerender produced an expected stale old internal reference.
• Reacquisition found the incremented component version.
• Second rerender used current references rather than cached nodes.
• Ordinary DOM required no shadow-root step; iframe did require context switch.
• Delayed component synchronized on readiness state, not a fixed sleep.

Stop the HTTP server with Ctrl+C. Remove only the disposable lab directory after preserving any desired screenshots/logs.

11. Summary and next step

The workflow proves that component reliability comes from search-context ownership and reference lifetime. The next lesson turns that into design policy: what should be public, what should be cached, and which layer should own the test.

Knowledge check

Official references and version notes

• Selenium 4.47 release notes — stable baseline pinned for this chapter.
• Selenium downloads — current stable bindings and Selenium Server/Grid versions.
• Finding web elements — Shadow DOM — official shadow-host/root/descendant pattern.
• Selenium Python WebElement API — shadow_root and WebElement semantics.
• Selenium Python ShadowRoot API — root-scoped find_element(s).
• Selenium Python exceptions — stale, no-shadow-root, and detached-shadow-root errors.
• Selenium troubleshooting — stale elements — why node replacement invalidates an element reference.
• W3C WebDriver Working Draft — Shadow Roots — protocol references and detached-shadow-root semantics.