Production Capstone: Build and Govern a Secure GitLab Software Delivery Platform: Security, Governance, and Reliability Validation

1. Validate control interactions, not isolated checkboxes

Most production failures happen between controls. A protected branch can still be undermined by an overly permissive matching rule. A protected variable can still be exposed to trusted-branch code running on an unsafe persistent runner. A security scanner can run successfully but cover the wrong source or produce evidence that no policy consumes. A release can exist while pointing to the wrong artifact.

2. Prove the release tuple independently

The core traceability tuple is tag → commit SHA → pipeline/source → artifact digest → release record. Each link should be independently observable. If any link disagrees, stop promotion until the discrepancy is explained.

from pathlib import Path
import json, hashlib, subprocess
root=Path('gitlab-ch34-capstone'); ev=root/'evidence'
artifact=json.loads((ev/'artifact_manifest.json').read_text())
release=json.loads((ev/'release_manifest.json').read_text())
actual=hashlib.sha256((root/artifact['file']).read_bytes()).hexdigest()
checks={
  'artifact_digest_matches_bytes': actual == artifact['sha256'],
  'release_digest_matches_artifact': release['artifact_sha256'] == artifact['sha256'],
  'security_report_hash_matches': release['security_report_sha256'] == (ev/'security_report.sha256').read_text().strip(),
}
try:
    tag_sha=subprocess.check_output(['git','-C',str(root),'rev-list','-n','1','v0.1.0-capstone'],text=True).strip()
    checks['tag_resolves_to_release_commit']= release['commit_sha'] in {'LOCAL_COMMIT_UNKNOWN',tag_sha} or release['commit_sha']==tag_sha
except Exception:
    checks['tag_resolves_to_release_commit']='TAG_NOT_AVAILABLE_IN_LOCAL_FALLBACK'
print(json.dumps(checks,indent=2))
if any(v is False for v in checks.values()): raise SystemExit('traceability validation failed')

A GitLab pipeline ID alone does not prove that the retained artifact came from that pipeline. The digest binds the bytes. A Git tag alone does not bind an external deployment target to those bytes. The deployment record or manifest must carry the immutable identity forward.

3. Access and merge-governance validation

Review the effective member roles at the project and inherited group levels. The target invariant is least privilege: normal application delivery should not require Owner, and elevated roles should be deliberate. Then inspect branch rules. GitLab’s current protection-rule semantics matter: when multiple rules match, the most permissive rule can determine push/merge behavior, while Code Owner approval uses a different, restrictive combination rule where that paid feature is enabled.

ACCESS VALIDATION RECORD
Project: asterline-lab/delivery-platform/app
Expected Developer capability: branch/MR/pipeline work
Expected Maintainer capability: repository/project configuration and release governance
Owner required for routine delivery: NO
Inherited memberships reviewed: YES
main direct push allowed: NO
release tag creation: restricted to intended release role
Required approvals enforced by GitLab: NO on Free path / verify on paid path
CODEOWNERS enforcement: NO on Free path / verify on paid path
Evidence IDs: <member snapshot>, <branch rule>, <MR>, <tag rule>

4. Pipeline, variable, job-token, and runner validation

Review four separate dimensions. First, pipeline source: does the configuration run the intended jobs for merge-request, default-branch, tag, scheduled, or triggered pipelines? Second, variables: are sensitive values protected, hidden/masked where appropriate, and scoped only to eligible pipelines/environments? Third, CI_JOB_TOKEN: is cross-project access constrained to the documented allowlist/resource model? Fourth, runner trust: could repository-controlled code reach a persistent host, sibling project, Docker socket, or production network?

5. Validate security evidence without overclaiming coverage

For the local fixture, inspect the report hash and scope. For GitLab-native SAST or pipeline secret detection, capture analyzer job, report artifact, commit SHA, and coverage assumptions. On Free/Premium, downloadable report artifacts are useful evidence even when Ultimate-only vulnerability-management dashboards or approval policies are unavailable.

from pathlib import Path
import json, hashlib
p=Path('gitlab-ch34-capstone/evidence/security_report.json')
report=json.loads(p.read_text())
print('scanner =',report['scanner'])
print('scope   =',report['scope'])
print('findings=',len(report['findings']))
print('sha256  =',hashlib.sha256(p.read_bytes()).hexdigest())
print('ASSERTION: this report covers only its declared scope; it is not proof of vulnerability absence')

6. Registry and release controls: mutable names versus immutable identity

A package version, container tag, or release name is useful for humans but may be mutable or re-creatable depending on the registry and policy. Use a digest/checksum for exact identity. If a container image is used, record the OCI manifest digest. If using the GitLab Container Registry, cleanup and protection rules affect who can mutate or delete tags/repositories; cleanup is destructive and should never be the only retention mechanism for release evidence.

7. Validate tier/offering assumptions explicitly

8. Test the major architectural tradeoffs

9. Reliability and cost review

Efficiency is a reliability property when it reduces queueing and resource exhaustion. Inspect duplicate branch/MR pipelines, unnecessary retries, artifact/cache retention, runner concurrency, serial deployment needs, API pagination/rate limits, registry growth, and Self-Managed dependencies. Do not “optimize” by deleting the evidence required to investigate failure.

RELIABILITY / COST REVIEW
Pipelines
  [ ] workflow:rules prevents accidental duplicate sources
  [ ] interruptible only where cancellation is safe
  [ ] retry used only for transient/idempotent failures
Runners
  [ ] capacity and trust class documented
  [ ] no privileged persistent runner for untrusted code
Artifacts / cache
  [ ] artifact retention matches evidence need
  [ ] cache is acceleration, not release evidence
API / hooks
  [ ] pagination and rate limits handled
  [ ] webhook duplicate delivery is idempotent
Registry
  [ ] retention/cleanup preserves release identities
Self-Managed
  [ ] DB, Gitaly, object storage, registry, Sidekiq, network/TLS dependencies mapped

10. Build the evidence matrix

from pathlib import Path
import csv
root=Path('gitlab-ch34-capstone'); ev=root/'evidence'
rows=[
 ['least_privilege','group/project membership','membership snapshot','unexpected inherited elevated role','group owner'],
 ['reviewed_default_branch','branch rule + MR','direct-push test + MR record','direct push succeeds / review absent','maintainer'],
 ['pipeline_identity','.gitlab-ci.yml + pipeline','source + commit SHA','release job on unintended source','platform'],
 ['runner_isolation','runner config/metadata','trust-class review','sensitive job on unsafe runner','runner operator'],
 ['artifact_identity','artifact manifest','independent SHA-256','checksum mismatch','release owner'],
 ['release_traceability','tag/release/manifest','tag->commit->digest check','tuple disagreement','release owner'],
 ['security_evidence','scanner/report','report hash + scope','missing/stale/out-of-scope report','security'],
 ['webhook_integrity','webhook receiver','signature + scope + duplicate test','invalid/stale/duplicate event causes action','integration owner'],
 ['recovery','backup/runbook','restore drill evidence','backup exists but restore fails','Self-Managed operator'],
]
with (ev/'evidence_matrix.csv').open('w',newline='') as f:
    w=csv.writer(f); w.writerow(['invariant','resource','verification','failure_signal','owner']); w.writerows(rows)
print((ev/'evidence_matrix.csv').read_text())

Knowledge check

11. Lesson summary and bridge

• GitLab controls are validated at their interactions: source, authorization, runner, credential, artifact, policy, and evidence boundaries.
• Traceability requires an immutable identity chain rather than confidence in labels or green pipeline status.
• Tier-gated controls are explicitly separated from Free-path process simulations.
• Reliability and cost reviews optimize duplicate work, retention, capacity, and retries without deleting necessary evidence.
• The evidence matrix turns architecture requirements into operational accountability.

Next, you will deliberately break four parts of the platform, preserve evidence, diagnose root cause, apply the least destructive correction, and prove recovery.

Primary sources and version notes

These lessons were finalized against current official GitLab documentation on 2026-08-22 with GitLab 19.3 as the reference release. Availability can vary by GitLab.com, Self-Managed, or Dedicated offering; Free, Premium, or Ultimate tier; namespace settings; administrator policy; runner type; and feature status. Re-check current documentation before applying a production design. Current protection-rule behavior and paid approval/environment-policy boundaries are especially important in this lesson because a visual “protected” or “approved” state can still be misinterpreted if overlapping rules or tier limitations are ignored.

• GitLab 19.3 release
• GitLab roles and permissions
• Protected branches
• Protection rules and permissions
• Merge requests
• Merge request approvals
• Code Owners
• CI/CD pipelines
• CI/CD variables
• CI_JOB_TOKEN
• Runner security
• Job artifacts
• Caching in GitLab CI/CD
• Protected environments
• Deployment approvals
• Container Registry
• Protected container repositories
• Releases
• Release evidence
• SAST
• Pipeline secret detection
• Security policies
• REST API
• Webhooks
• GitLab agent for Kubernetes
• Audit events
• Health check endpoints
• Back up GitLab
• Restore GitLab
• GitLab Duo Agent Platform
• Agent tool governance