Compliance Pipelines, Pipeline Execution Policies, Governance, Audit Evidence, and Enterprise Controls: Guided Hands-On Workflow and Core Operations

1. Scenario and why each layer exists

You operate the fictional Acme Labs namespace. Application project acme-labs/ch36-app owns normal build/test jobs. A central governance team owns policy version ch36-policy-v1.0.0, whose mandatory gate is policy:governance-proof. One project may receive a short waiver, but every waiver must have an identity, owner, reason, and expiry.

The exercise separates three layers on purpose: project intent, central policy intent, and the effective result. If you only compare two YAML files by eye, you have not proved what pipeline should exist.

2. Preflight: tools, path, and before-state evidence

python --version
git --version
LAB="${TMPDIR:-/tmp}/gitlab-ch36-governance"
rm -rf "$LAB"
mkdir -p "$LAB"/{project,policy,governance,tools,evidence}
cd "$LAB"
printf 'lab=%s\n' "$PWD"
find . -maxdepth 2 -type f -print

Expected observation: the directory exists and contains no configuration yet. The rm -rf is guarded by a fixed temporary path in this disposable lab. If you change LAB, print and inspect it before cleanup.

3. Create project intent — no policy yet

The project configuration is intentionally ordinary. It builds one synthetic file and tests it. This is what the application team owns.

stages: [build, test]

build:
  stage: build
  script:
    - printf 'source_sha=%s\n' "$CI_COMMIT_SHA"
    - printf 'artifact=demo-only\n' > build.txt
  artifacts:
    paths: [build.txt]
    expire_in: 1 day

test:
  stage: test
  script:
    - test -f build.txt

cat > project/.gitlab-ci.yml <<'YAML'
stages: [build, test]
build:
  stage: build
  script:
    - printf 'artifact=demo-only\n' > build.txt
  artifacts:
    paths: [build.txt]
    expire_in: 1 day
test:
  stage: test
  script:
    - test -f build.txt
YAML
sha256sum project/.gitlab-ci.yml

4. Create the central policy model and the real GitLab mapping

For the free simulation, policy intent is represented in JSON so a standard-library Python program can evaluate it without installing packages. Beside it, keep the real GitLab YAML mapping so you can see exactly which product layer the simulation represents.

{
  "name": "ch36 mandatory governance gate",
  "version": "ch36-policy-v1.0.0",
  "control": "governance-proof",
  "strategy": "inject_policy",
  "suffix": "never",
  "scope": {"projects": ["acme-labs/ch36-app"]},
  "job": {
    "name": "policy:governance-proof",
    "stage": ".pipeline-policy-pre",
    "control": "governance-proof"
  }
}

pipeline_execution_policy:
  - name: ch36 mandatory governance gate
    description: Enforce a visible, versioned governance check.
    enabled: true
    pipeline_config_strategy: inject_policy
    content:
      include:
        - project: acme-labs/ch36-policy-config
          file: policy-ci.yml
          ref: ch36-policy-v1.0.0
    policy_scope:
      projects:
        including:
          - id: 123456
    suffix: never
    skip_ci:
      allowed: false
    no_pipeline:
      allowed: false
    variables_override:
      allowed: false
      exceptions:
        - CH36_NON_SECRET_MODE

In a real security policy project, the policy belongs in .gitlab/security-policies/policy.yml. The referenced policy-ci.yml belongs in the CI configuration project named by content.include.project. The explicit ref prevents silent drift to that repository’s current HEAD.

policy:governance-proof:
  stage: .pipeline-policy-pre
  script:
    - test -n "$CI_COMMIT_SHA"
    - printf 'policy_version=%s\n' 'ch36-policy-v1.0.0'
    - printf 'project=%s\n' "$CI_PROJECT_PATH"
    - printf 'source_sha=%s\n' "$CI_COMMIT_SHA"
  artifacts:
    when: always
    expire_in: 7 days
    paths:
      - governance-evidence/

5. Build a transparent local compiler — not a fake GitLab server

The simulator answers only the chapter’s governance question: which jobs are project-owned, which job is centrally enforced, whether a time-bounded waiver exists, and which hashes identify the inputs. It does not pretend to emulate GitLab rules, runners, artifacts, or the full YAML compiler.

#!/usr/bin/env python3
import argparse, hashlib, json
from datetime import datetime, timezone
from pathlib import Path

def load(path):
    return json.loads(Path(path).read_text(encoding="utf-8"))

def sha(path):
    return hashlib.sha256(Path(path).read_bytes()).hexdigest()

p=argparse.ArgumentParser()
p.add_argument("--project", required=True)
p.add_argument("--policy", required=True)
p.add_argument("--exceptions", required=True)
p.add_argument("--out", required=True)
a=p.parse_args()
project=load(a.project); policy=load(a.policy); exceptions=load(a.exceptions)
now=datetime.now(timezone.utc)
valid=[]
for e in exceptions:
    try:
        expiry=datetime.fromisoformat(e["expires_at"].replace("Z", "+00:00"))
    except Exception:
        continue
    if (e.get("project") == project["project"] and
        e.get("control") == policy["control"] and
        e.get("owner") and expiry > now):
        valid.append(e)

project_names={j["name"] for j in project["jobs"]}
policy_name=policy["job"]["name"]
if policy_name in project_names and policy.get("suffix") == "never":
    result={"status":"rejected","reason":"job-name-conflict","policy_version":policy["version"]}
    Path(a.out).write_text(json.dumps(result, indent=2)+"\n", encoding="utf-8")
    raise SystemExit(3)

waiver=valid[0] if valid else None
effective=[dict(j, origin="project") for j in project["jobs"]]
effective.insert(0, {**policy["job"], "origin":"policy", "policy_version":policy["version"],
                     "enforcement":"waived" if waiver else "blocking"})
result={
  "status":"compiled",
  "project":project["project"],
  "source_revision":project["source_revision"],
  "project_model_sha256":sha(a.project),
  "policy_model_sha256":sha(a.policy),
  "policy_version":policy["version"],
  "policy_scope":policy["scope"],
  "exception": waiver,
  "jobs":effective,
  "audit_event":{
    "type":"simulated_policy_evaluation",
    "at":now.isoformat(),
    "policy_version":policy["version"],
    "project":project["project"]
  }
}
Path(a.out).parent.mkdir(parents=True, exist_ok=True)
Path(a.out).write_text(json.dumps(result, indent=2)+"\n", encoding="utf-8")
print(json.dumps(result, indent=2))

cat > tools/compile_effective_pipeline.py <<'PY'
# Paste the Python simulator from the lesson block above verbatim.
PY
chmod +x tools/compile_effective_pipeline.py

6. Record source revision, project jobs, and an empty exception set

git init -q
git config user.name "Chapter 36 Learner"
git config user.email "learner@example.invalid"
cat > project/project-model.json <<'JSON'
{
  "project": "acme-labs/ch36-app",
  "source_revision": "PENDING_GIT_SHA",
  "jobs": [
    {"name":"build","stage":"build"},
    {"name":"test","stage":"test"}
  ]
}
JSON
cat > policy/policy-model.json <<'JSON'
{
  "name":"ch36 mandatory governance gate",
  "version":"ch36-policy-v1.0.0",
  "control":"governance-proof",
  "strategy":"inject_policy",
  "suffix":"never",
  "scope":{"projects":["acme-labs/ch36-app"]},
  "job":{"name":"policy:governance-proof","stage":".pipeline-policy-pre","control":"governance-proof"}
}
JSON
printf '[]\n' > governance/exceptions.json
# Save the complete simulator shown in section 5 as tools/compile_effective_pipeline.py.
git add .
git commit -qm "lab: baseline governance model"
SHA="$(git rev-parse HEAD)"
python - "$SHA" <<'PY'
import json,sys
p='project/project-model.json'; d=json.load(open(p)); d['source_revision']=sys.argv[1]
open(p,'w').write(json.dumps(d,indent=2)+'\n')
PY
git add project/project-model.json && git commit -qm "lab: bind source revision evidence"
printf 'source_sha=%s\n' "$(git rev-parse HEAD)"

7. Compile the effective result and inspect where enforcement entered

Run the simulator and inspect origin. The mandatory job must appear even though the project model never declared it. That distinction is the point of the exercise: project source and effective governed configuration are different states.

python tools/compile_effective_pipeline.py \
  --project project/project-model.json \
  --policy policy/policy-model.json \
  --exceptions governance/exceptions.json \
  --out evidence/effective-pipeline.json
python - <<'PY'
import json
x=json.load(open('evidence/effective-pipeline.json'))
for j in x['jobs']:
    print(f"{j['name']:28} origin={j['origin']:7} enforcement={j.get('enforcement','n/a')}")
print('policy_version=',x['policy_version'])
print('exception=',x['exception'])
PY
sha256sum evidence/effective-pipeline.json

Expected: policy:governance-proof has origin policy and enforcement blocking; build and test remain project-owned. The output also records hashes of both input models.

8. Prove that a project override is a conflict, not a successful bypass

Add a project job with the same name. Because the model uses suffix: never, the compiler rejects the result instead of silently choosing one definition.

cp project/project-model.json evidence/project-model.clean.json
python - <<'PY'
import json
p='project/project-model.json'; d=json.load(open(p))
d['jobs'].append({'name':'policy:governance-proof','stage':'test'})
open(p,'w').write(json.dumps(d,indent=2)+'\n')
PY
set +e
python tools/compile_effective_pipeline.py \
  --project project/project-model.json --policy policy/policy-model.json \
  --exceptions governance/exceptions.json --out evidence/conflict.json
RC=$?
set -e
printf 'exit=%s\n' "$RC"
cat evidence/conflict.json
cp evidence/project-model.clean.json project/project-model.json

In current GitLab PEP behavior, suffix: never likewise causes name conflicts to fail rather than allowing a project definition to replace the enforced one. With the default on_conflict, GitLab gives conflicting jobs distinct names. This is a configuration-layer problem, not a token or runner problem.

9. Create a narrow, owned, expiring waiver

The exception is separate governance data. It names one project, one control, one owner, one approver, one reason, and one expiry. That makes the exception observable and reviewable instead of hiding it in an application variable.

python - <<'PY'
import json
from datetime import datetime,timedelta,timezone
now=datetime.now(timezone.utc)
x=[{
  'id':'EX-CH36-001',
  'project':'acme-labs/ch36-app',
  'control':'governance-proof',
  'owner':'training-governance@example.invalid',
  'approved_by':'chapter36-approver@example.invalid',
  'reason':'bounded training waiver',
  'created_at':now.isoformat().replace('+00:00','Z'),
  'expires_at':(now+timedelta(hours=2)).isoformat().replace('+00:00','Z')
}]
open('governance/exceptions.json','w').write(json.dumps(x,indent=2)+'\n')
PY
python tools/compile_effective_pipeline.py \
  --project project/project-model.json --policy policy/policy-model.json \
  --exceptions governance/exceptions.json --out evidence/waived-pipeline.json
python - <<'PY'
import json
x=json.load(open('evidence/waived-pipeline.json'))
print('exception_id=',x['exception']['id'])
print('policy_job=',x['jobs'][0])
PY

The simulator leaves the policy-origin job visible but marks the control waived. A real organization may implement the technical exception as a temporary project exclusion in policy_scope or as explicit policy logic. GitLab does not supply a generic owner/reason/expiry waiver object for you, so the business lifecycle must be governed separately and reconciled with the technical scope.

10. Compare local configuration with the governed result

11. Optional real GitLab inspection — read-only first

If you have an authorized disposable Ultimate namespace, create the policy through normal review, then inspect it rather than immediately changing it again. Keep credentials in your shell environment and never print them.

export GITLAB_URL="https://gitlab.example.test"
export PROJECT_ID="123456"
# export GITLAB_TOKEN="..."  # narrow, authorized token only

curl --fail --silent --show-error \
  --header "PRIVATE-TOKEN: $GITLAB_TOKEN" \
  "$GITLAB_URL/api/v4/projects/$PROJECT_ID/pipelines?per_page=3" | jq .

PIPELINE_ID="<disposable-pipeline-id>"
curl --fail --silent --show-error \
  --header "PRIVATE-TOKEN: $GITLAB_TOKEN" \
  "$GITLAB_URL/api/v4/projects/$PROJECT_ID/pipelines/$PIPELINE_ID/jobs?per_page=100" | jq .

# Project audit events are paid/tier/role dependent.
curl --fail --silent --show-error \
  --header "PRIVATE-TOKEN: $GITLAB_TOKEN" \
  "$GITLAB_URL/api/v4/projects/$PROJECT_ID/audit_events?pagination=keyset&per_page=20&order_by=id&sort=desc" | jq .

12. Cleanup and layer-selection challenge

cd /tmp 2>/dev/null || cd /
printf 'about_to_remove=%s\n' "$LAB"
case "$LAB" in
  */gitlab-ch36-governance) rm -rf "$LAB" ;;
  *) echo "Refusing unexpected cleanup path" >&2; exit 2 ;;
esac

Challenge: the application repository does not contain policy:governance-proof, but the actual pipeline does. Which layer should you inspect? The policy/effective-configuration layer. A repository-only search cannot prove absence of policy-injected jobs.

Knowledge check

Official references and version notes

Further reading — current official GitLab sources

Behavior and tier assumptions in this chapter were checked against the current documentation on 2026-09-13. Re-check these pages before applying the optional enterprise path because policy schemas and product tiers evolve.

• GitLab Docs — Pipeline execution policies
• GitLab Docs — Security policy projects
• GitLab Docs — Policy enforcement and separation of duties
• GitLab Docs — Compliance pipelines (deprecated)
• GitLab Docs — CI/CD variables and precedence
• GitLab Docs — CI Lint API
• GitLab Docs — Audit events
• GitLab Docs — Audit events API
• GitLab Docs — Compliance frameworks
• GitLab Docs — Deprecations and removals

Current assumptions used in this chapter: Version-sensitive YAML, Runner/executor behavior, APIs, security features, policy controls, tiers, and deprecations must be verified against the exact GitLab, GitLab Runner, tool, and external-system versions used in production. Preserve the exact project/ref/SHA, compiled configuration, pipeline/job IDs, runner/tool versions, and external target evidence used for reproducible work.