Monorepos, Path Filters, Changed-File Detection, and Selective Pipelines: Guided Hands-On Workflow

1. Disposable scenario and preflight

Create a throwaway repository named gha-monorepo-lab with branches inside the same repository. Keep this exercise away from proprietary code because changed paths and component rules become part of the evidence. The repository contains two tiny Python applications, apps/api and apps/web, and both depend conceptually on libs/shared.

• GitHub-hosted ubuntu-24.04; no self-hosted runner.
• actions/checkout v7.0.1 pinned at 3d3c42e5aac5ba805825da76410c181273ba90b1.
• actions/setup-python v7.0.0 pinned at 5fda3b95a4ea91299a34e894583c3862153e4b97; Python 3.13.
• actions/upload-artifact v7.0.1 pinned at 043fb46d1a93c77aae656e7c1c64a875d1fc6a0a.
• Workflow starts on every lab pull request; branch protection may require only Monorepo CI / required.
• No token write permission, secrets, deployment, package, cache or cloud account is required.

2. Build the smallest useful monorepo fixture

apps/
  api/test_component.py
  web/test_component.py
libs/shared/contract.txt
.github/scripts/select_components.py
.github/workflows/monorepo-ci.yml
README.md

Each component test can simply read libs/shared/contract.txt and its own local fixture. The point is not Python testing; it is to make dependency fan-out observable. Commit the selector itself so its logic changes are reviewed like application code.

3. First compare native path filtering

For an optional non-required workflow, native path filters are excellent: GitHub can avoid creating runs that have no relevant paths. For the required monorepo CI lane, however, filtering the entire workflow can create a missing/Pending required-check problem.

# Demonstration only: do not make this disappearing workflow the sole required check.
on:
  pull_request:
    paths:
      - 'apps/**'
      - 'libs/**'
      - '.github/workflows/monorepo-ci.yml'
      - '.github/scripts/select_components.py'

4. Checkout history must contain the comparison commits

The default shallow checkout is optimized for the current revision, not merge-base analysis. On the tiny lab repository, use fetch-depth: 0 so both branch histories and their common ancestor are definitely available. Then prove both event SHAs exist with git cat-file before calculating anything.

BASE="$PR_BASE_SHA"
HEAD="$PR_HEAD_SHA"
git cat-file -e "$BASE^{commit}"
git cat-file -e "$HEAD^{commit}"
MERGE_BASE="$(git merge-base "$BASE" "$HEAD")"
printf 'base=%s
head=%s
merge_base=%s
' "$BASE" "$HEAD" "$MERGE_BASE"

In a very large repository, fetch-depth: 0 may cost too much. A production selector can fetch the exact base/head refs and deepen history incrementally until a merge base is proven. The safety requirement is evidence of a valid comparison, not a particular fetch-depth number.

5. Produce changed paths without turning them into commands

Use the merge base and head to derive the pull-request change set. The lab passes paths through a file, not through command substitution. -z makes the delimiter NUL, which is the only byte Git filenames cannot contain. --no-renames is conservative: a move is seen as a deletion plus an addition, so ownership rules can react to both old and new locations.

git diff --name-only --no-renames -z "$MERGE_BASE" "$HEAD" > changed-paths.z
python -c "from pathlib import Path; print([p.decode('utf-8','surrogateescape') for p in Path('changed-paths.z').read_bytes().split(b'\0') if p])"

6. Map direct paths and shared dependencies

The selector treats file paths as data. Direct API paths select API, direct Web paths select Web, and a shared-library path selects both. Documentation-only changes select neither component. Any path outside the explicitly harmless/documented set triggers a full fallback because the selector cannot prove it is irrelevant.

import json, os, pathlib
raw = pathlib.Path("changed-paths.z").read_bytes().split(b"\0")
paths = [p.decode("utf-8", "surrogateescape") for p in raw if p]
selected = set(); reasons = {}; full = False

def add(component, reason):
    selected.add(component); reasons.setdefault(component, []).append(reason)

for path in paths:
    if path.startswith("apps/api/"):
        add("api", f"direct:{path}")
    elif path.startswith("apps/web/"):
        add("web", f"direct:{path}")
    elif path.startswith("libs/shared/"):
        add("api", f"depends-on-shared:{path}"); add("web", f"depends-on-shared:{path}")
    elif path.startswith("docs/") or path in {"README.md"}:
        pass
    else:
        full = True; reasons.setdefault("fallback", []).append(f"unowned-or-global:{path}")
if full:
    selected = {"api", "web"}
components = sorted(selected)
out = {"base": os.environ["BASE"], "head": os.environ["HEAD"], "merge_base": os.environ["MERGE_BASE"], "paths": paths, "components": components, "full_fallback": full, "reasons": reasons}
pathlib.Path("selection.json").write_text(json.dumps(out, indent=2), encoding="utf-8")
with open(os.environ["GITHUB_OUTPUT"], "a", encoding="utf-8") as f:
    f.write("components=" + json.dumps(components, separators=(",", ":")) + "\n")
    f.write("count=" + str(len(components)) + "\n")
    f.write("full_fallback=" + str(full).lower() + "\n")

This “unknown means all” rule is intentionally conservative. Teams often begin with a permissive default that skips unknown paths; over time, a new build file or shared directory is introduced and silently receives no CI. Failing closed toward more tests makes architecture growth safe.

7. Turn selection into a dynamic matrix

The selector emits compact JSON such as ["api","web"] through GITHUB_OUTPUT. A downstream job converts that JSON with fromJSON and creates one matrix cell per component. fail-fast: false keeps the other selected component running after one failure so the evidence packet shows the full affected set rather than only the first defect.

strategy:
  fail-fast: false
  matrix:
    component: ${{ fromJSON(needs.select.outputs.components) }}
steps:
  - run: python "apps/$COMPONENT/test_component.py"
    env:
      COMPONENT: ${{ matrix.component }}

8. Keep one stable required check present

The required job runs after the selector and matrix using always() only for non-privileged result aggregation. It explicitly accepts component-tests=skipped only when the selector output count is zero. If the selector fails, or if any selected matrix cell fails, the aggregate check fails. This creates a stable branch-protection name while still allowing selective work.

Do not make an arbitrary skipped job the authority. A skipped job can report success, so the aggregate step must compare the skip with selector evidence. The evidence says why there was no component job.

9. Complete lab workflow

name: Monorepo CI
on:
  pull_request:
  workflow_dispatch:
permissions: {}
jobs:
  select:
    name: Detect affected components
    runs-on: ubuntu-24.04
    permissions:
      contents: read
    outputs:
      components: ${{ steps.selector.outputs.components }}
      count: ${{ steps.selector.outputs.count }}
      full_fallback: ${{ steps.selector.outputs.full_fallback }}
    steps:
      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
        with:
          fetch-depth: 0
      - uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
        with:
          python-version: "3.13"
      - id: revisions
        name: Prove the revision pair
        shell: bash
        env:
          EVENT_NAME: ${{ github.event_name }}
          PR_BASE_SHA: ${{ github.event.pull_request.base.sha }}
          PR_HEAD_SHA: ${{ github.event.pull_request.head.sha }}
        run: |
          set -euo pipefail
          if [ "$EVENT_NAME" = pull_request ]; then
            BASE="$PR_BASE_SHA"; HEAD="$PR_HEAD_SHA"
          else
            HEAD="$(git rev-parse HEAD)"; BASE="$(git rev-parse HEAD^)"
          fi
          git cat-file -e "$BASE^{commit}"
          git cat-file -e "$HEAD^{commit}"
          MERGE_BASE="$(git merge-base "$BASE" "$HEAD")"
          printf 'BASE=%s\nHEAD=%s\nMERGE_BASE=%s\n' "$BASE" "$HEAD" "$MERGE_BASE" | tee revisions.env
          printf 'base=%s\nhead=%s\nmerge_base=%s\n' "$BASE" "$HEAD" "$MERGE_BASE" >> "$GITHUB_OUTPUT"
      - name: Derive NUL-safe changed paths
        shell: bash
        env:
          HEAD: ${{ steps.revisions.outputs.head }}
          MERGE_BASE: ${{ steps.revisions.outputs.merge_base }}
        run: |
          set -euo pipefail
          git diff --name-only --no-renames -z "$MERGE_BASE" "$HEAD" > changed-paths.z
          python -c "from pathlib import Path; p=[x.decode('utf-8','surrogateescape') for x in Path('changed-paths.z').read_bytes().split(b'\\0') if x]; Path('changed-paths.txt').write_text('\\n'.join(repr(x) for x in p)+'\\n',encoding='utf-8')"
      - id: selector
        name: Map paths through the dependency graph
        shell: bash
        env:
          BASE: ${{ steps.revisions.outputs.base }}
          HEAD: ${{ steps.revisions.outputs.head }}
          MERGE_BASE: ${{ steps.revisions.outputs.merge_base }}
        run: python .github/scripts/select_components.py
      - uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
        if: ${{ always() }}
        with:
          name: selector-${{ github.run_id }}-${{ github.run_attempt }}
          path: |
            revisions.env
            changed-paths.txt
            selection.json
          retention-days: 7
  component-tests:
    name: Test ${{ matrix.component }}
    needs: select
    if: ${{ needs.select.outputs.count != '0' }}
    runs-on: ubuntu-24.04
    permissions:
      contents: read
    strategy:
      fail-fast: false
      matrix:
        component: ${{ fromJSON(needs.select.outputs.components) }}
    steps:
      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
      - uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
        with:
          python-version: "3.13"
      - name: Run component contract
        shell: bash
        env:
          COMPONENT: ${{ matrix.component }}
        run: python "apps/$COMPONENT/test_component.py"
  required:
    name: Monorepo CI / required
    needs: [select, component-tests]
    if: ${{ always() }}
    runs-on: ubuntu-24.04
    permissions: {}
    steps:
      - name: Aggregate selector and selected work
        shell: bash
        env:
          SELECT_RESULT: ${{ needs.select.result }}
          TEST_RESULT: ${{ needs.component-tests.result }}
          COUNT: ${{ needs.select.outputs.count }}
        run: |
          set -euo pipefail
          test "$SELECT_RESULT" = success
          if [ "$COUNT" = 0 ]; then test "$TEST_RESULT" = skipped; else test "$TEST_RESULT" = success; fi

For workflow_dispatch, the example compares HEAD^ only to keep manual experimentation simple; the production PR semantics are the pull_request branch. A production push selector should use the push event’s before/after SHAs and handle new-branch/all-zero before explicitly rather than reusing PR logic.

10. Run four change experiments

11. Compare shallow versus sufficient history

Create a topic branch with several commits and an older divergence point. Temporarily change checkout to fetch-depth: 1. The expected failure is an explicit inability to prove the base/head/merge-base object. Preserve that log. Then restore sufficient history and confirm the same revision pair produces a deterministic changed-file set.

12. Challenge: choose the layer, do not copy YAML

A change under libs/shared/ selects only API even though Web imports the library. The changed-file set may be correct; the defect is in the dependency graph/closure. Repair that architecture model, add a selector contract test, and rerun the smallest equivalent change set.

13. Guided workflow summary

You now have a selector whose input revisions, history, paths, dependency reasons, matrix and required-check conclusion can be inspected independently. The workflow saves time without allowing an absent workflow to masquerade as a passing required check.

Knowledge check

Official references and version notes

• Workflow syntax: path filters — Current paths/paths-ignore matching, two-dot versus three-dot diff rules, pattern ordering and GitHub.com diff limits.
• Troubleshooting workflows — Current path-filter diff limits and workflow-run troubleshooting.
• Troubleshooting required status checks — Why a path-filtered workflow can leave a required check Pending and how skipped jobs differ.
• Matrix jobs — Dynamic matrix concepts and per-cell jobs.
• Passing information between jobs — Selector job outputs used by downstream matrix and aggregate jobs.
• Events that trigger workflows — Event-specific base/head/ref semantics.
• actions/checkout — Current checkout behavior; v7.0.1 is pinned by full commit SHA in executable examples.
• Git diff documentation — Two-dot/three-dot revision comparison and NUL-safe path output options.
• Git merge-base — Computing the common ancestor for dependency-aware pull-request change detection.
• actions/checkout v7.0.1 — Full-SHA-pinned checkout used by the lab.
• actions/setup-python v7.0.0 — Full-SHA-pinned Python 3.13 setup.
• actions/upload-artifact v7.0.1 — Full-SHA-pinned selector evidence upload.