Large Repositories, Monorepos, Search, Actions Cost Controls, and Platform Performance: Guided Hands-On Workflow and Core Operations

1. Lab boundary and preflight

This lesson creates a public disposable repository named ch31-scale-lab. The lab uses standard GitHub-hosted Ubuntu runners, which are free for public repositories. It does not require an organization, larger runner, Git LFS purchase, PAT, self-hosted runner, or paid cache expansion. Commands are Bash/Git Bash; PowerShell users can run the shell blocks in Git Bash or translate heredocs and shell tests.

gh auth status
git --version
python --version
OWNER="$(gh api -H 'X-GitHub-Api-Version: 2026-03-10' user --jq .login)"
REPO="ch31-scale-lab"
FULL="$OWNER/$REPO"
printf 'owner=%s repo=%s\n' "$OWNER" "$REPO"

Predict before acting: (1) an API-only file change should select API validation and the gate, but not Web validation; (2) a change under libs/shared/ should select both components; (3) a short-lived demo artifact/cache should appear in hosted inventory but must not justify a paid runner or large storage allowance.

2. Create a synthetic monorepo

mkdir "$REPO" && cd "$REPO"
git init -b main
mkdir -p services/api services/web libs/shared docs tools tests .github/workflows .github
cat > services/api/app.py <<'PY'
from pathlib import Path
print("api", Path("libs/shared/version.txt").read_text().strip())
PY
cat > services/web/app.py <<'PY'
from pathlib import Path
print("web", Path("libs/shared/version.txt").read_text().strip())
PY
printf 'shared-v1\n' > libs/shared/version.txt
printf '# Scale lab\n\nSynthetic monorepo.\n' > docs/index.md
printf '{"schema":1,"global":true}\n' > build-config.json
printf '.cache/\n' > .gitignore
cat > .github/CODEOWNERS <<EOF
/services/api/ @$OWNER
/services/web/ @$OWNER
/libs/shared/ @$OWNER
/.github/workflows/ @$OWNER
/build-config.json @$OWNER
EOF

git add .
git commit -m 'Create synthetic monorepo'
gh repo create "$REPO" --public --source=. --remote=origin --push

The repository has two components and one shared library. CODEOWNERS expresses review ownership; it does not know that both components depend on libs/shared/version.txt. That dependency belongs in the affected-project model.

3. Measure an intentionally broad baseline

Start with a workflow that runs both component jobs and creates tiny storage objects. This produces a measurable baseline before optimization.

name: ch31-baseline
on:
  workflow_dispatch:
permissions:
  contents: read
jobs:
  api:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@d23441a48e516b6c34aea4fa41551a30e30af803 # v6
      - run: python -m py_compile services/api/app.py
  web:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@d23441a48e516b6c34aea4fa41551a30e30af803 # v6
      - run: python -m py_compile services/web/app.py
  storage-sample:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@d23441a48e516b6c34aea4fa41551a30e30af803 # v6
      - name: Prepare tiny generated cache payload
        run: |
          mkdir -p .cache/demo
          python - <<'PY'
          from pathlib import Path
          Path('.cache/demo/payload.bin').write_bytes(b'0' * (64 * 1024))
          PY
      - uses: actions/cache@0057852bfaa89a56745cba8c7296529d2fc39830 # v4.3.0
        with:
          path: .cache/demo
          key: ch31-demo-${{ hashFiles('build-config.json') }}
      - name: Create tiny evidence file
        run: printf 'run=%s sha=%s\n' "$GITHUB_RUN_ID" "$GITHUB_SHA" > ch31-evidence.txt
      - uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
        with:
          name: ch31-evidence
          path: ch31-evidence.txt
          retention-days: 1

cat > .github/workflows/baseline-ci.yml <<'YAML'
name: ch31-baseline
on:
  workflow_dispatch:
permissions:
  contents: read
jobs:
  api:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@d23441a48e516b6c34aea4fa41551a30e30af803
      - run: python -m py_compile services/api/app.py
  web:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@d23441a48e516b6c34aea4fa41551a30e30af803
      - run: python -m py_compile services/web/app.py
  storage-sample:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@d23441a48e516b6c34aea4fa41551a30e30af803
      - name: Prepare tiny generated cache payload
        run: |
          mkdir -p .cache/demo
          python - <<'PY'
          from pathlib import Path
          Path('.cache/demo/payload.bin').write_bytes(b'0' * (64 * 1024))
          PY
      - uses: actions/cache@0057852bfaa89a56745cba8c7296529d2fc39830
        with:
          path: .cache/demo
          key: ch31-demo-${{ hashFiles('build-config.json') }}
      - run: printf 'run=%s sha=%s\n' "$GITHUB_RUN_ID" "$GITHUB_SHA" > ch31-evidence.txt
      - uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a
        with:
          name: ch31-evidence
          path: ch31-evidence.txt
          retention-days: 1
YAML

git add .github/workflows/baseline-ci.yml
git commit -m 'Add measurable baseline workflow'
git push
gh workflow run ch31-baseline --repo "$FULL"
sleep 3
gh run list --repo "$FULL" --workflow ch31-baseline --limit 3

After the run completes, record the run rather than relying on memory:

RUN_ID="$(gh run list --repo "$FULL" --workflow ch31-baseline --limit 1 --json databaseId --jq '.[0].databaseId')"
gh run view "$RUN_ID" --repo "$FULL" --json databaseId,headSha,createdAt,updatedAt,jobs

gh cache list --repo "$FULL" --key ch31-demo

gh api -H 'Accept: application/vnd.github+json' \
  -H 'X-GitHub-Api-Version: 2026-03-10' \
  "repos/$FULL/actions/artifacts?per_page=100" \
  --jq '.artifacts[] | {id,name,size_in_bytes,expires_at,workflow_run}'

4. Build an affected-project router

The router consumes an exact base/head diff and emits three booleans. Cross-cutting paths expand the affected set to both components.

#!/usr/bin/env python3
import json, sys

CROSS_PREFIXES = ("libs/shared/", ".github/workflows/")
CROSS_FILES = {"build-config.json", "pyproject.toml", "package-lock.json"}

def affected(paths):
    paths = set(paths)
    global_change = any(p in CROSS_FILES or p.startswith(CROSS_PREFIXES) for p in paths)
    api = global_change or any(p.startswith("services/api/") for p in paths)
    web = global_change or any(p.startswith("services/web/") for p in paths)
    return {"api": api, "web": web, "global": global_change}

if __name__ == "__main__":
    result = affected([line.strip() for line in sys.stdin if line.strip()])
    print(json.dumps(result, sort_keys=True))

from tools.affected import affected

def test_api_only():
    assert affected(["services/api/app.py"]) == {"api": True, "web": False, "global": False}

def test_shared_is_cross_cutting():
    assert affected(["libs/shared/version.txt"]) == {"api": True, "web": True, "global": True}

cat > tools/affected.py <<'PY'
#!/usr/bin/env python3
import json, sys
CROSS_PREFIXES=("libs/shared/", ".github/workflows/")
CROSS_FILES={"build-config.json","pyproject.toml","package-lock.json"}
def affected(paths):
    paths=set(paths)
    global_change=any(p in CROSS_FILES or p.startswith(CROSS_PREFIXES) for p in paths)
    return {
      "api": global_change or any(p.startswith("services/api/") for p in paths),
      "web": global_change or any(p.startswith("services/web/") for p in paths),
      "global": global_change,
    }
if __name__ == "__main__":
    print(json.dumps(affected([x.strip() for x in sys.stdin if x.strip()]), sort_keys=True))
PY
cat > tests/test_affected.py <<'PY'
from tools.affected import affected
assert affected(["services/api/app.py"]) == {"api": True, "web": False, "global": False}
assert affected(["libs/shared/version.txt"]) == {"api": True, "web": True, "global": True}
print("affected routing tests: PASS")
PY
python tests/test_affected.py

5. Add selective jobs but keep one stable gate

name: ch31-monorepo-ci
on:
  pull_request:
    branches: [main]
permissions:
  contents: read
concurrency:
  group: ch31-pr-${{ github.event.pull_request.number }}
  cancel-in-progress: true
jobs:
  route:
    runs-on: ubuntu-latest
    outputs:
      api: ${{ steps.route.outputs.api }}
      web: ${{ steps.route.outputs.web }}
    steps:
      - uses: actions/checkout@d23441a48e516b6c34aea4fa41551a30e30af803 # v6
        with:
          fetch-depth: 0
      - id: route
        env:
          BASE_SHA: ${{ github.event.pull_request.base.sha }}
          HEAD_SHA: ${{ github.event.pull_request.head.sha }}
        run: |
          git diff --name-only "$BASE_SHA" "$HEAD_SHA" > changed.txt
          cat changed.txt
          result="$(python tools/affected.py < changed.txt)"
          python - <<'PY' "$result" >> "$GITHUB_OUTPUT"
          import json,sys
          r=json.loads(sys.argv[1])
          print(f"api={str(r['api']).lower()}")
          print(f"web={str(r['web']).lower()}")
          PY
  api:
    needs: route
    if: needs.route.outputs.api == 'true'
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@d23441a48e516b6c34aea4fa41551a30e30af803
        with:
          sparse-checkout: |
            services/api
            libs/shared
      - run: python -m py_compile services/api/app.py
  web:
    needs: route
    if: needs.route.outputs.web == 'true'
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@d23441a48e516b6c34aea4fa41551a30e30af803
        with:
          sparse-checkout: |
            services/web
            libs/shared
      - run: python -m py_compile services/web/app.py
  monorepo-gate:
    if: always()
    needs: [route, api, web]
    runs-on: ubuntu-latest
    steps:
      - env:
          ROUTE: ${{ needs.route.result }}
          API: ${{ needs.api.result }}
          WEB: ${{ needs.web.result }}
        run: |
          test "$ROUTE" = success
          test "$API" = success -o "$API" = skipped
          test "$WEB" = success -o "$WEB" = skipped

Why this shape? The workflow itself starts for every pull request to main; the cheap route job computes exact scope; selected component jobs use sparse checkout; the aggregate gate is stable enough to become a required check later. The lab does not change repository rules.

Save that workflow as .github/workflows/monorepo-ci.yml, commit it with the router/tests, and push main. Then create an API-only PR:

git add tools tests .github/workflows/monorepo-ci.yml
git commit -m 'Add affected-project routing and stable gate'
git push

git switch -c api-only-change
printf '\nprint("api-only-change")\n' >> services/api/app.py
git add services/api/app.py
git commit -m 'Change API only'
git push -u origin api-only-change
gh pr create --repo "$FULL" --base main --head api-only-change --title 'API-only routing exercise' --body 'Chapter 31 disposable routing lab.'
PR_NUM="$(gh pr view --repo "$FULL" --json number --jq .number)"
gh pr checks "$PR_NUM" --repo "$FULL" --watch

Expected state: route, api, and monorepo-gate succeed; web is skipped. That is causal evidence that selectivity changed work without removing the stable gate.

6. Search and API inventory with explicit scope

gh search code 'version.txt' --repo "$FULL" --limit 20 --json path,repository,url

gh issue create --repo "$FULL" --title 'Performance evidence exercise' --body 'Synthetic issue for Chapter 31 search.'
gh issue list --repo "$FULL" --search 'is:open "Performance evidence exercise"'
gh pr list --repo "$FULL" --search 'is:open "API-only routing exercise"'

gh api -H 'X-GitHub-Api-Version: 2026-03-10' rate_limit \
  --jq '.resources | {core,search,code_search,graphql}'

Fresh code can take time to index. If Code Search returns zero results immediately, verify the file locally with git grep and retry later. Do not convert indexing lag into a false conclusion that the file is absent.

7. Challenge: choose the control, not a memorized command

For each change, decide which control should react and why:

1. docs/index.md only: component jobs or documentation-only validation?
2. libs/shared/version.txt: which components become affected?
3. .github/workflows/monorepo-ci.yml: why should it be cross-cutting?
4. A reviewer complains they were not requested: is that router logic, CODEOWNERS, team permissions, or a ruleset?

The lesson is successful when you can separate dependency routing, review ownership, and merge enforcement instead of solving all three with one path glob.

8. Summary

You measured broad work, introduced an explicit dependency router, preserved a stable gate, used sparse checkout only after routing, and collected run/storage/search/rate-limit evidence. Lesson 3 turns these mechanics into platform-level architecture decisions.

Knowledge check

Further reading — current primary sources

• Repository limits
• About large files on GitHub
• About Git Large File Storage
• Triggering workflows and path-filter behavior
• Dependency caching reference
• GitHub Actions billing
• Understanding GitHub Code Search syntax
• REST API — rate limits