Chapter 34Lesson 02~520 minutes

Production Capstone: Build and Govern a Secure GitLab Software Delivery Platform: Implementation and Automation Build-Out

Build the capstone incrementally: repository governance, CI/CD, runner boundaries, immutable artifact evidence, release traceability, security evidence, API inspection, webhook validation, and operational evidence.

ImplementationCI/CDArtifactsReleaseAutomation

Learning objectives

  • Build the capstone in small reversible increments with evidence captured before and after each change.
  • Create a governed Git workflow and CI/CD configuration that makes pipeline source, commit identity, job output, and release eligibility explicit.
  • Produce a synthetic package artifact, checksum manifest, security evidence, and release metadata tied to the same commit identity.
  • Use a read-only GitLab API/glab path and a locally validated signed-webhook pattern without exposing real credentials.
  • Maintain an evidence manifest containing refs, pipeline/job identifiers, runner assumptions, artifact digests, release identifiers, policy decisions, and cleanup obligations.
Availability baseline — verified 2026-08-22 against GitLab 19.3. The required build-out has two layers: a GitLab Free disposable project for repository/MR/ref/release state, and a local no-runner fallback for CI/security/artifact evidence. If GitLab.com hosted-runner quota is available, the tiny pipeline can run there; otherwise execute the equivalent local commands and keep fixture evidence. SAST and pipeline secret-detection report artifacts are available across tiers, but require supported runners; richer vulnerability-management UI and policy enforcement are tier-gated.

1. Create the local capstone workspace before touching GitLab

Start with a local repository that can be deleted safely. The example application is intentionally small: one Python function, one unit test, one packaging script, one local security check, and an evidence directory. The goal is traceability, not application complexity.

from pathlib import Path
import json, textwrap
root=Path("gitlab-ch34-capstone")
(root/"app").mkdir(parents=True, exist_ok=True)
(root/"tests").mkdir(exist_ok=True)
(root/"scripts").mkdir(exist_ok=True)
(root/"evidence").mkdir(exist_ok=True)
(root/"dist").mkdir(exist_ok=True)

(root/"app"/"service.py").write_text("""def normalize(name: str) -> str:\n    return name.strip().lower().replace(' ', '-')\n""")
(root/"tests"/"test_service.py").write_text("""import unittest\nfrom app.service import normalize\n\nclass TestNormalize(unittest.TestCase):\n    def test_normalize(self):\n        self.assertEqual(normalize(' Hello World '), 'hello-world')\n\nif __name__ == '__main__':\n    unittest.main()\n""")
(root/"README.md").write_text("# Asterline delivery-platform capstone\n\nSynthetic training repository.\n")
(root/"evidence"/"scope.json").write_text(json.dumps({
  "organization":"Asterline Software (fictional)",
  "project":"asterline-lab/delivery-platform/app",
  "data_classification":"synthetic-only",
  "production_access":False,
  "gitlab_reference":"19.3 / 2026-08-22"
}, indent=2))
print(root.resolve())

2. Initialize Git and establish the exact source identity

Initialize the local repository, commit the seed application, and record the commit SHA. Do not create a release tag yet; the tag comes only after review and verification. If you create the disposable GitLab project, use a dedicated training namespace and verify the remote URL before pushing.

cd gitlab-ch34-capstone
git init
git add .
git commit -m "capstone: seed synthetic service"
git branch -M main
git rev-parse HEAD
git status --short --branch

# Optional after creating the disposable GitLab Free project:
# git remote add origin <COPY_THE_DISPOSABLE_PROJECT_URL>
# git remote -v
# git push -u origin main
Before any push: confirm the destination namespace and repository path. Never paste a token into the remote URL or shell history. Use SSH/credential-manager authentication or glab auth login through the official flow.

3. Build the merge-governance contract

Create a feature branch for the implementation and a CODEOWNERS file that documents ownership intent. On GitLab Premium/Ultimate, Code Owner approval can be enforced when combined with protected branches. On Free, the file and reviewer assignment remain useful documentation, but they are not equivalent to a blocking Code Owner approval rule.

# .gitlab/CODEOWNERS
* @platform-reviewers
/app/ @application-reviewers
/.gitlab-ci.yml @platform-reviewers
/scripts/security_check.py @security-reviewers
FREE-PATH MERGE EVIDENCE
Target branch: main
Direct push: prohibited by branch rule
Source branch: feature/capstone-build
Reviewer requested: yes
Reviewer decision: APPROVE / REQUEST CHANGES
Pipeline commit SHA recorded: yes
Artifact digest recorded: yes
Security evidence reviewed: yes
Paid enforcement present: NO (process evidence only)

In a live disposable project, protect main so direct push is denied and merge is permitted only to the intended role. Required approval rules and required Code Owner approval are optional paid extensions; do not claim they blocked a merge on the Free path.

4. Build a least-privilege CI/CD pipeline with explicit sources

The pipeline distinguishes merge-request validation from release-tag work. It never prints environment variables or tokens. The build produces an artifact and checksum. A separate security job runs a deterministic local check. Release metadata is generated only for tags. The jobs are intentionally tiny.

workflow:
  rules:
    - if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
    - if: '$CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH'
    - if: '$CI_COMMIT_TAG'
    - when: never

stages: [verify, package, security, release]

default:
  interruptible: true
  before_script:
    - python --version

unit_test:
  stage: verify
  script:
    - python -m unittest discover -s tests -v

package:
  stage: package
  script:
    - python scripts/build_package.py
  artifacts:
    paths:
      - dist/
      - evidence/artifact_manifest.json
    expire_in: 7 days

security_evidence:
  stage: security
  script:
    - python scripts/security_check.py
  artifacts:
    when: always
    paths:
      - evidence/security_report.json
    expire_in: 14 days

release_manifest:
  stage: release
  rules:
    - if: '$CI_COMMIT_TAG'
  script:
    - python scripts/release_manifest.py
  artifacts:
    paths:
      - evidence/release_manifest.json
    expire_in: 30 days

If your runner requires a container image, choose a reviewed image and pin it as tightly as your environment supports; mutable image tags are supply-chain dependencies. The capstone intentionally avoids privileged Docker-in-Docker. A production runner should be isolated, non-privileged where practical, and matched to trusted pipeline sources.

5. Build package, security, and release evidence scripts

# scripts/build_package.py
from pathlib import Path
import hashlib, json, tarfile, os
root=Path(__file__).resolve().parents[1]
dist=root/'dist'; dist.mkdir(exist_ok=True)
out=dist/'asterline-app.tar.gz'
with tarfile.open(out,'w:gz') as tf:
    tf.add(root/'app',arcname='app')
sha=hashlib.sha256(out.read_bytes()).hexdigest()
commit=os.environ.get('CI_COMMIT_SHA','LOCAL_COMMIT_UNKNOWN')
manifest={'file':str(out.relative_to(root)),'sha256':sha,'commit_sha':commit}
(root/'evidence'/'artifact_manifest.json').write_text(json.dumps(manifest,indent=2))
print(json.dumps(manifest,indent=2))
# scripts/security_check.py
from pathlib import Path
import hashlib, json, re
root=Path(__file__).resolve().parents[1]
findings=[]
patterns={
  'dangerous_eval': re.compile(r'\beval\s*\('),
  'fake_secret_assignment': re.compile(r'(?i)(password|token|secret)\s*=\s*[\"\'][^\"\']+[\"\']')
}
for p in sorted((root/'app').rglob('*.py')):
    text=p.read_text()
    for rule,pat in patterns.items():
        for m in pat.finditer(text):
            findings.append({'rule':rule,'file':str(p.relative_to(root)),'offset':m.start()})
report={'scanner':'capstone-local-fixture','scope':'app/**/*.py','findings':findings}
raw=json.dumps(report,indent=2)
(root/'evidence'/'security_report.json').write_text(raw)
(root/'evidence'/'security_report.sha256').write_text(hashlib.sha256(raw.encode()).hexdigest()+'\n')
print(raw)
raise SystemExit(1 if findings else 0)
# scripts/release_manifest.py
from pathlib import Path
import json, os
root=Path(__file__).resolve().parents[1]
artifact=json.loads((root/'evidence'/'artifact_manifest.json').read_text())
manifest={
  'tag':os.environ.get('CI_COMMIT_TAG','LOCAL_TAG_NOT_SET'),
  'commit_sha':os.environ.get('CI_COMMIT_SHA',artifact['commit_sha']),
  'pipeline_id':os.environ.get('CI_PIPELINE_ID','LOCAL_PIPELINE'),
  'artifact_sha256':artifact['sha256'],
  'security_report_sha256':(root/'evidence'/'security_report.sha256').read_text().strip()
}
(root/'evidence'/'release_manifest.json').write_text(json.dumps(manifest,indent=2))
print(json.dumps(manifest,indent=2))

6. Run the no-runner fallback and capture before/after evidence

Even if no GitLab runner quota is available, you can execute the same deterministic checks locally. Record that the evidence is local fallback evidence; do not label it as a GitLab job.

cd gitlab-ch34-capstone
python -m unittest discover -s tests -v
python scripts/build_package.py
python scripts/security_check.py
python scripts/release_manifest.py
sha256sum dist/asterline-app.tar.gz || shasum -a 256 dist/asterline-app.tar.gz
cat evidence/artifact_manifest.json
cat evidence/security_report.json
cat evidence/release_manifest.json

Expected: tests pass, the security fixture contains zero findings, and the artifact SHA-256 in artifact_manifest.json exactly matches the independently calculated checksum. The release manifest will show local placeholder identifiers until executed in a tagged GitLab pipeline.

7. Optional GitLab-native security evidence

Current GitLab documentation makes basic SAST scanning and downloadable SAST JSON reports available across tiers, and pipeline secret detection report artifacts can also be produced across tiers. Rich merge-request security widgets, vulnerability-management workflows, advanced scanning, and policy enforcement depend on higher tiers.

# Optional extension; validate in CI Lint before use.
include:
  - template: Jobs/SAST.gitlab-ci.yml
  - template: Jobs/Secret-Detection.gitlab-ci.yml

# If you define stages explicitly, keep the test stage required by these templates.
stages: [verify, package, test, security, release]
Coverage warning: an empty report is not proof of security. Record what languages/files/commits the analyzer actually scanned, which analyzer version ran, and which classes of defects remain outside its scope.

8. Create a tag and GitLab Release tied to exact identity

After the merge request is reviewed and the target commit is known, create a release candidate tag that points to that exact commit. In the disposable project, create a GitLab Release from that tag. Releases are available across GitLab tiers. If the tag is protected, release creation also depends on permission to create that protected tag.

# Run only after review and after confirming the target commit.
COMMIT="$(git rev-parse HEAD)"
printf 'reviewed commit: %s
' "$COMMIT"
git tag -a v0.1.0-capstone "$COMMIT" -m "Synthetic capstone release"
git rev-list -n 1 v0.1.0-capstone

# Optional push to the disposable project after verifying origin:
# git remote -v
# git push origin v0.1.0-capstone

In GitLab, create the release for v0.1.0-capstone and include the artifact SHA-256 from your evidence manifest in the release notes. Do not upload a different build merely because the filename matches. If you publish a generic package or container image, capture its package/version or OCI digest and add that immutable identity to the manifest.

9. Add read-only API/glab verification

Automation begins with read-only state inspection. Resolve the project explicitly and request machine-readable fields. Authentication proves identity; project membership and resource rules still decide authorization.

# glab path: authenticated locally; token value is never printed.
glab auth status
glab api projects/:id --paginate --output json

# Or read specific project fields with an explicit URL-encoded project path:
# glab api 'projects/asterline-lab%2Fdelivery-platform%2Fapp'

For direct REST clients, handle HTTP status codes, pagination, rate limits, and request IDs. Do not scrape GitLab HTML. If a list endpoint returns multiple pages, follow documented pagination links/headers instead of assuming the first page is complete.

10. Validate an event-driven integration locally with the current signing model

Current GitLab webhook documentation supports signing tokens that produce a Standard Webhooks HMAC-SHA256 signature over webhook-id.webhook-timestamp.body. The receiver must verify the signature with constant-time comparison, reject stale/replayed events according to its policy, validate event type and object scope, and make handlers idempotent because delivery can be retried.

from pathlib import Path
import base64, hashlib, hmac, json, time
root=Path('gitlab-ch34-capstone'); ev=root/'evidence'
# Synthetic token: base64("FAKE_SECRET_NOT_REAL")
signing_token='whsec_RkFLRV9TRUNSRVRfTk9UX1JFQUw='
message_id='evt-capstone-001'
timestamp='1787360400'
body=json.dumps({'event_name':'release','project_path':'asterline-lab/delivery-platform/app','tag':'v0.1.0-capstone'},separators=(',',':'))
raw_key=base64.b64decode(signing_token.removeprefix('whsec_'))
message=f'{message_id}.{timestamp}.{body}'.encode()
digest=hmac.new(raw_key,message,hashlib.sha256).digest()
signature='v1,'+base64.b64encode(digest).decode()

def valid(sig):
    expected='v1,'+base64.b64encode(hmac.new(raw_key,message,hashlib.sha256).digest()).decode()
    return any(hmac.compare_digest(expected,item) for item in sig.split(' '))

record={'webhook_id':message_id,'timestamp':timestamp,'signature_valid':valid(signature),'event':json.loads(body)}
(ev/'webhook_verification.json').write_text(json.dumps(record,indent=2))
print(json.dumps(record,indent=2))
Production receiver: never log the signing token, and avoid logging an entire payload when it can contain personal or sensitive project data. Record only the event identifiers and sanitized fields needed for troubleshooting.

11. Build the capstone evidence manifest

from pathlib import Path
import json, subprocess, hashlib
root=Path('gitlab-ch34-capstone'); ev=root/'evidence'
def git(*args):
    try: return subprocess.check_output(['git','-C',str(root),*args],text=True).strip()
    except Exception: return 'UNAVAILABLE'
artifact=json.loads((ev/'artifact_manifest.json').read_text())
security=json.loads((ev/'security_report.json').read_text())
hook=json.loads((ev/'webhook_verification.json').read_text())
manifest={
  'commit_sha':git('rev-parse','HEAD'),
  'branch':git('branch','--show-current'),
  'pipeline_id':'LIVE_PIPELINE_ID_OR_LOCAL_FALLBACK',
  'runner_identity':'HOSTED_OR_ISOLATED_RUNNER_OR_LOCAL_FALLBACK',
  'artifact':artifact,
  'security_findings':len(security['findings']),
  'security_report_sha256':hashlib.sha256((ev/'security_report.json').read_bytes()).hexdigest(),
  'release_tag':'v0.1.0-capstone',
  'release_id':'GITLAB_RELEASE_URL_OR_FIXTURE',
  'webhook_verified':hook['signature_valid'],
  'policy_notes':['Free path: reviewer evidence is process control, not required-approval enforcement'],
  'cleanup':['delete disposable GitLab project after evidence export','delete local workspace after review']
}
(ev/'evidence_manifest.json').write_text(json.dumps(manifest,indent=2))
print(json.dumps(manifest,indent=2))

12. Challenge: choose the correct control instead of copying a sequence

You discover that the release tag points to commit A, but the package checksum in the release notes was generated from commit B. Which surface do you change first?

Do not “fix” the release notes to make the values match. First stop promotion, preserve the tag/release/artifact evidence, determine which identity is authoritative, rebuild or retag only through the governed workflow, and verify the new tuple tag → commit → pipeline → digest. The mismatch is a provenance failure, not a documentation typo.

Knowledge check

Why does the pipeline use workflow: rules?

Why is the artifact checksum calculated independently after the build?

What does the Free-path CODEOWNERS file prove?

Why is webhook signature verification not enough by itself?

If GitLab-hosted runner quota is unavailable, what must change in the evidence?

13. Lesson summary and bridge

  • The platform was built incrementally so every consequential resource has an explicit identity and verification step.
  • The pipeline separates source eligibility, deterministic verification, packaging, security evidence, and release metadata.
  • The artifact digest and security-report hash create evidence that can be compared independently later.
  • Read-only API automation and signed webhook verification are separate trust boundaries from repository mutation.
  • The evidence manifest now gives Lesson 3 a concrete object to validate rather than relying on green status indicators.

Next, you will test the architecture as an interacting control system, prove traceability, challenge tier assumptions, and build an evidence matrix with explicit owners and failure signals.

Primary sources and version notes

These lessons were finalized against current official GitLab documentation on 2026-08-22 with GitLab 19.3 as the reference release. Availability can vary by GitLab.com, Self-Managed, or Dedicated offering; Free, Premium, or Ultimate tier; namespace settings; administrator policy; runner type; and feature status. Re-check current documentation before applying a production design. The GitLab release documentation now requires modern glab for release jobs as release-cli is being replaced. The mandatory path uses the release UI/tag model so the lab does not depend on a particular runner image or CLI release job.

Next lesson

Security, Governance, and Reliability Validation

Challenge every control interaction and build an evidence matrix that maps each invariant to independent proof and an accountable owner.

Keep the academy open

Support free, practical DevOps education.

Every lesson is designed to remain readable in a browser, downloadable from GitHub, and usable without a paid learning platform. Contributions help expand and maintain the curriculum.

Ethereum / ERC-20
0x716c4Ab160C4B66F31a28AE2448BfF68fc3a2ef0Send only Ethereum/ERC-20 compatible assets to this address.