CI/CD Integration with GitHub Actions, GitLab CI, and Jenkins: Diagnostics, Failure Modes, and Production Practices

1. Broken example: the workflow passes despite a red suite

This shell shortcut is intentionally wrong:

- name: Broken test step
  run: python ci/run_suite.py --browser chrome || true

The shell converts the failing test command into exit code zero. Downstream stages see green even though Selenium failed. A similar mistake is marking the critical test step continue-on-error: true without a separate gate that interprets the outcome.

2. Missing browser or driver is provisioning, not synchronization

If session creation fails because the requested browser binary is absent, collect the safe version inventory and session-creation exception. Do not add an element wait; no page exists yet. On hosted runners, compare the failing job’s image/browser evidence with a known-good run. On self-hosted runners, inspect the agent image, PATH, browser installation, Selenium Manager output, and permissions.

Failure occurs before driver.session_id exists
-> classify as browser/driver/runner provisioning
-> inspect browser binary + Selenium Manager/driver evidence
-> repair image/agent
-> rerun smallest session-creation scenario

3. Secrets printed in logs are an incident

CI secret stores reduce accidental disclosure only if workflows and tests avoid printing them. Do not include tokens in query strings, command-line echoes, screenshots of secret-bearing pages, raw network traces, or broad environment dumps. Course examples use no real secret. In a real authorized test environment, use a least-privilege secret reference and redact before artifact persistence.

# Safe shape: inject without echoing the value.
env:
  TEST_PASSWORD: ${{ secrets.TEST_PASSWORD }}
# Test code consumes it; diagnostics log only that a secret was present, never the value.

4. Container localhost points at the wrong process

If the CI test process, Grid, browser Node, and AUT are in different containers, each has its own loopback interface. A browser container navigating to http://127.0.0.1:8765 reaches itself, not the job container. Confirm the network topology, service aliases, published ports, and browser-side reachability before changing locators or waits.

5. Artifacts uploaded only on success delete the most useful evidence

Place artifact collection in failure-independent control flow. GitHub Actions uses if: always(); GitLab can use artifacts: when: always; Jenkins Declarative Pipeline can archive in post { always { ... } }. Give parallel jobs unique names so attempts do not overwrite one another.

- name: Upload evidence even after a failed test step
  if: always()
  uses: actions/upload-artifact@v7
  with:
    name: selenium-${{ matrix.browser }}-${{ github.run_id }}-${{ github.run_attempt }}
    path: evidence/

6. Concurrent jobs mutating one environment create cross-job flakes

Two CI jobs can each own a clean browser and still collide through the same user, order, database row, file, or singleton feature flag. Chapter 21’s isolation model extends across CI jobs: include shard/job identity in synthetic test data and artifact namespaces, or provision isolated test tenants where authorized.

7. Platform retry and rerun features must not erase the first attempt

A manual rerun can help confirm whether a failure is intermittent, but it is additional evidence—not a repair. Keep attempt 1 artifacts, use unique run_attempt/job identifiers, and classify pass-after-rerun as a stability signal. Do not configure blanket platform retries that turn flaky gates green without investigation.

8. Invisible browser drift can look like an application regression

Hosted runner images intentionally evolve. Record browser versions and returned capabilities on every run. If a failure begins after the image/browser changed, reproduce with the smallest scenario and compare behavior. The right correction might be product compatibility, test adaptation, or a temporary support-matrix decision; it is not automatically “pin an old browser forever.”

9. CI diagnostic sequence

1. Preserve first-failure artifacts before cleanup or rerun.
2. Confirm Selenium/binding/browser/driver/Grid and CI runner/agent versions.
3. Confirm commit, job, browser lane, base URL, and synthetic test identity.
4. Inspect whether a WebDriver session was created and record its capabilities/context.
5. Inspect locator/element/synchronization evidence only after session/context is valid.
6. Inspect AUT and network evidence, then Grid/runner resource state if remote.
7. Apply the least destructive correction to the smallest failing layer.
8. Rerun the smallest controlled scenario; keep the original evidence.

Knowledge check

Official references and current-version notes

• Selenium 4.47 release notes
• Selenium downloads — current stable client and Grid versions
• GitHub-hosted runners reference
• GitHub Ubuntu 24.04 runner image inventory
• actions/checkout
• actions/setup-python
• actions/upload-artifact
• GitLab CI job artifacts
• GitLab CI/CD YAML reference
• Jenkins Pipeline tests and artifacts
• Jenkins Declarative Pipeline syntax