Checkpoint Lab — Matrix Strategies, Dynamic Matrices, Fail-Fast, and Parallel Test Design

1. Checkpoint scenario and invariant

You are qualifying a tiny Python utility. Three cells are part of the support contract; one cell is experimental. The lab deliberately fails supported-b and experimental-future. Your invariant is: every mandatory cell must produce a successful observed test result; an experimental failure is recorded but does not itself block support.

Before running, predict: four generated jobs; at most two matrix cells active simultaneously; one mandatory failure; one tolerated experimental failure; four per-cell artifacts; and an aggregate decision of BLOCK.

2. Preflight and current assumptions

3. Create the tiny app

In the disposable repository, create these two files. They are intentionally dependency-free so a test failure is about the controlled matrix policy, not a package registry.

def normalize(value: str) -> str:
    return value.strip().lower()

from app import normalize
assert normalize("  GHA  ") == "gha"
print("base test passed")

4. Exact checkpoint workflow

Create .github/workflows/chapter12-checkpoint.yml with the following workflow. The test step records its own outcome before a final gate converts the controlled failure into the generated job’s failure. That lets the evidence-upload step run even for failed cells.

name: chapter12-compatibility-checkpoint
on:
  workflow_dispatch:
permissions: {}

jobs:
  test:
    name: ${{ matrix.mode }} | ${{ matrix.os }} | py-${{ matrix.python }}
    runs-on: ${{ matrix.os }}
    continue-on-error: ${{ matrix.experimental }}
    strategy:
      fail-fast: false
      max-parallel: 2
      matrix:
        include:
          - os: ubuntu-24.04
            python: '3.12'
            mode: supported-a
            experimental: false
            inject_failure: false
          - os: ubuntu-24.04
            python: '3.13'
            mode: supported-b
            experimental: false
            inject_failure: true
          - os: windows-2025
            python: '3.13'
            mode: supported-c
            experimental: false
            inject_failure: false
          - os: ubuntu-24.04
            python: '3.13'
            mode: experimental-future
            experimental: true
            inject_failure: true
    steps:
      - name: Checkout exact workflow source
        uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
        with:
          persist-credentials: false

      - name: Set up pinned runtime action
        uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
        with:
          python-version: ${{ matrix.python }}

      - name: Run base test and controlled failure
        id: test
        continue-on-error: true
        shell: bash
        run: |
          python test_app.py
          if [ '${{ matrix.inject_failure }}' = 'true' ]; then
            echo 'controlled matrix failure for checkpoint evidence'
            exit 1
          fi

      - name: Write per-cell evidence
        if: ${{ always() }}
        shell: bash
        env:
          CELL_OS: ${{ matrix.os }}
          CELL_PYTHON: ${{ matrix.python }}
          CELL_MODE: ${{ matrix.mode }}
          CELL_EXPERIMENTAL: ${{ matrix.experimental }}
          TEST_OUTCOME: ${{ steps.test.outcome }}
        run: |
          mkdir -p evidence
          python - <<'PY_EVIDENCE'
          import json, os, platform
          data = {
              "run_id": os.environ["GITHUB_RUN_ID"],
              "run_attempt": os.environ["GITHUB_RUN_ATTEMPT"],
              "sha": os.environ["GITHUB_SHA"],
              "os": os.environ["CELL_OS"],
              "python_requested": os.environ["CELL_PYTHON"],
              "python_resolved": platform.python_version(),
              "mode": os.environ["CELL_MODE"],
              "experimental": os.environ["CELL_EXPERIMENTAL"] == "true",
              "test_outcome": os.environ["TEST_OUTCOME"],
          }
          name = data["mode"] + ".json"
          with open("evidence/" + name, "w", encoding="utf-8") as f:
              json.dump(data, f, indent=2, sort_keys=True)
          PY_EVIDENCE

      - name: Upload unique cell evidence
        if: ${{ always() }}
        uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
        with:
          name: cell-${{ strategy.job-index }}-${{ matrix.mode }}
          path: evidence/*.json
          if-no-files-found: error
          retention-days: 7

      - name: Convert recorded test failure into cell failure
        if: ${{ steps.test.outcome == 'failure' }}
        shell: bash
        run: exit 1

  aggregate:
    name: Aggregate compatibility evidence
    if: ${{ always() }}
    needs: test
    runs-on: ubuntu-24.04
    steps:
      - name: Download all per-cell evidence
        uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
        with:
          pattern: cell-*
          path: collected
          merge-multiple: true

      - name: Produce support decision
        shell: bash
        run: |
          python - <<'PY_AGG'
          import glob, json, sys
          rows = [json.load(open(p, encoding="utf-8")) for p in glob.glob("collected/*.json")]
          rows.sort(key=lambda r: r["mode"])
          mandatory_failures = [r for r in rows if not r["experimental"] and r["test_outcome"] != "success"]
          experimental_failures = [r for r in rows if r["experimental"] and r["test_outcome"] != "success"]
          with open("support-decision.txt", "w", encoding="utf-8") as out:
              out.write(f"cells_observed={len(rows)}\n")
              out.write(f"mandatory_failures={len(mandatory_failures)}\n")
              out.write(f"experimental_failures={len(experimental_failures)}\n")
              out.write("decision=" + ("BLOCK" if mandatory_failures else "SUPPORTED") + "\n")
          print(open("support-decision.txt", encoding="utf-8").read())
          # Deliberately fail because supported-b was designed to fail.
          sys.exit(1 if mandatory_failures else 0)
          PY_AGG

5. Predictions before dispatch

6. Execute and preserve the first run

1. Commit the fixture and workflow; record the exact commit SHA.
2. Dispatch the workflow once. Do not rerun immediately.
3. Record run ID and run attempt.
4. Open the generated job graph and confirm all four cell names.
5. Capture the first failing supported cell log before changing anything.
6. Record which cells completed and which were tolerated.

Because fail-fast:false, the expected outcome is full evidence rather than sibling cancellation.

7. Verification checklist

• The checkout step ran the exact github.sha and did not persist credentials.
• Each cell records requested and resolved Python version.
• Four unique artifacts exist; a missing artifact is a failed evidence invariant.
• supported-b.json says experimental:false and test_outcome:failure.
• experimental-future.json says experimental:true and test_outcome:failure.
• The aggregator downloads all cell artifacts and prints decision=BLOCK.
• The workflow’s final red state is explained by a mandatory compatibility failure, not by artifact transport or runner setup.

8. Repair without hiding the original failure

Preserve the original run. Then change only the controlled supported failure: set inject_failure:false for supported-b. Leave the experimental failure in place. Commit the new workflow revision and dispatch again. Predict that mandatory failures become zero, experimental failures remain one, and the aggregate decision changes to SUPPORTED.

This second run demonstrates why experimental tolerance and mandatory support must be separate dimensions. Do not “repair” the first run by making every cell continue-on-error:true.

9. Required evidence packet

10. Cleanup and rollback

After the evidence packet is saved, delete the disposable repository if desired. Do not delete the original run/artifacts before you have copied the evidence you intend to retain. No external infrastructure, credential, package or deployment needs rollback because the mandatory lab never creates one.

11. What Chapter 12 adds to the operating model

You can now turn one job definition into a controlled compatibility experiment: the matrix source is explicit, each generated cell has a defined trust/runtime identity, failure tolerance is policy rather than wishful thinking, cost is bounded, and downstream decisions are based on retained cell evidence. Chapter 13 deepens the artifact side of this model: retention, cross-job transport and workflow-result management.

Knowledge check

Official references and version notes

• GitHub Docs — Running variations of jobs in a workflow — matrix expansion, contexts, include/exclude, dynamic outputs, failure handling and max-parallel.
• GitHub Docs — workflow syntax: strategy.matrix — current matrix limit and strategy semantics.
• GitHub Docs — expressions: fromJSON — converting job-output JSON into arrays/objects used by a later matrix.
• GitHub Docs — strategy context — fail-fast, job-index, job-total and max-parallel for the current generated job.
• GitHub Docs — matrix strategy with reusable workflows — matrix-driven reusable workflow calls and output caveats.
• actions/upload-artifact v7.0.1 and actions/download-artifact v8.0.1 — action releases pinned by full commit SHA in the checkpoint.