Runner Security, Isolation Boundaries, Privileged Containers, Fork Pipelines, Untrusted Code, and Threat Modeling: Concepts, Architecture, and Mental Model
Threat-model every CI job as repository-controlled code: classify pipeline trust, prove runner eligibility, inspect executor and host exposure, constrain credentials/network/devices, preserve incident evidence, and reset trust by cleanup or destruction.
Learning objectives
- Model a GitLab Runner as privileged infrastructure that executes repository-controlled code, not as a neutral build box.
- Trace trust classification → runner eligibility → executor boundary → host/network/device/credential exposure → job execution → evidence → trust reset.
- Explain why runner tags, protection, executor isolation and ephemeral lifecycle solve different parts of the threat model.
- Distinguish fork merge-request context, same-project protected-ref context, and post-merge/release context before exposing protected resources.
- Preserve pipeline/job/runner evidence before destroying or reimaging a suspicious worker.
1. The practical problem: Chapter 30's capacity is also an attack surface
Chapter 30 treated a runner fleet as a capacity system. Chapter 31 keeps that model, but changes the default assumption about the job: repository code, test scripts, build plugins, downloaded dependencies and CI configuration can all be controlled by someone who should not control your runner host. A fast fleet that lets a forked merge request reach a privileged Docker daemon or production credential is not a reliable fleet.
The security question therefore comes before “Which executor is fastest?” First classify who controls the code and ref, then determine which runner may accept it, then inspect what the executor exposes. Only after those boundaries are known should you optimize cache reuse, warm workers or privileged image-building.
2. Mental model: code trust flows into infrastructure trust
A pipeline starts with an exact source event, project, ref and SHA. Those facts determine a trust class; they do not automatically make code benign. The compiled CI configuration decides which jobs exist and which tags they request. GitLab then matches jobs to eligible runners using runner scope, tags and protection rules. The runner manager prepares an executor. That executor exposes some amount of host kernel, filesystem, devices, network and credentials. The job can behave correctly or maliciously. Finally, evidence must survive long enough to investigate, while the execution boundary is cleaned, destroyed or reimaged before it is trusted again.
flowchart TD
A[Pipeline source + project + ref + SHA] --> B[Trust classification]
B --> C[Compiled rules + requested tags]
C --> D[Runner scope + tags + protected eligibility]
D --> E[Executor / worker boundary]
E --> F[Host + network + device + credential exposure]
F --> G[Repository-controlled job executes]
G --> H[Job trace + runner/manager evidence]
H --> I[Destroy / reimage / verified cleanup]
I --> J[Trust reset before reuse]
Each arrow is a distinct control. A protected runner cannot repair an unsafe host socket mount. A non-privileged container cannot compensate for a job that receives a broad production token. An ephemeral VM does not help if its logs disappear before incident collection.
3. Keep the security state domains separate
| State layer | Read-only evidence | Security question |
|---|---|---|
| Pipeline/source |
CI_PIPELINE_SOURCE, project IDs/paths, MR
source/target project, ref,
CI_COMMIT_REF_PROTECTED,
CI_COMMIT_SHA
|
Who controls the code and which immutable revision is executing? |
| Compiled configuration |
Merged YAML, workflow:rules, job
rules, requested tags, job graph
|
Did configuration create the job and ask for a sensitive runner pool? |
| Runner eligibility |
Runner ID/scope, protected flag, tags, paused/online state,
CI_JOB_TAGS, CI_RUNNER_TAGS
|
Could this job legally match this runner? Tags are routing labels, not an authorization proof by themselves. |
| Runner manager | Runner version, authentication identity, host, logs, concurrency | Which trusted control process requested and dispatched the job? |
| Executor/worker | Executor, container/pod/VM ID, privileged/rootless state, namespaces/capabilities, reuse count | What isolation boundary actually contains repository-controlled code? |
| Host/device exposure | Docker socket/pipe, host mounts, devices, host PID/network namespaces, kernel | Can job code cross the executor boundary into the worker or manager host? |
| Network exposure | Routes, DNS, metadata endpoints, internal services, egress policy | What can malicious code reach even without escaping the container? |
| Identity/credentials |
Protected-variable eligibility,
CI_JOB_TOKEN scope, OIDC audience/claims,
deploy credentials
|
What authority is reachable during this exact job? Never capture values. |
| Reuse state | Git worktree strategy, caches, image pull policy, volumes, home directories | Can one job read or poison residue that another job will trust? |
| Incident/cleanup | Pipeline/job/runner IDs, manager logs, worker identity, preserved artifacts, credential-rotation record, destruction/reimage proof | Can you investigate the first failure and prove the compromised trust boundary was reset? |
4. Trust class is about control, not about branch names alone
| Example | Conservative trust class | Why |
|---|---|---|
| Fork merge request | Untrusted | A contributor controls the fork branch and CI content. GitLab documents that fork MR pipelines normally run in the fork; a parent-project member can deliberately run one in the parent, where parent resources become relevant. |
| Same-project feature branch | Review/untrusted by default | Project membership does not make every commit safe; Developers can intentionally or accidentally submit dangerous code. |
| Same-project MR with both branches protected and authorized triggerer | Eligible for protected resources only if the project setting allows it | GitLab 18.1+ adds a controlled path, but the conditions are explicit and the code still deserves review. |
| Protected release tag created by restricted maintainers | Higher trust | Protected ref governance narrows who can create the ref; runner protection and tags can further narrow execution. |
| Post-merge protected main deployment | Higher trust, not infinite trust | It can use narrowly scoped deployment identity, but dependencies/configuration can still be compromised. |
5. Read-only inspection before changing runners
Start with evidence. In a real GitLab job, these values are safe metadata; do not print any token or secret value:
printf 'source=%s\n' "$CI_PIPELINE_SOURCE"
printf 'project=%s id=%s\n' "$CI_PROJECT_PATH" "$CI_PROJECT_ID"
printf 'ref=%s protected=%s sha=%s\n' "$CI_COMMIT_REF_NAME" "$CI_COMMIT_REF_PROTECTED" "$CI_COMMIT_SHA"
printf 'pipeline=%s job=%s\n' "$CI_PIPELINE_ID" "$CI_JOB_ID"
printf 'job_tags=%s runner_tags=%s\n' "$CI_JOB_TAGS" "$CI_RUNNER_TAGS"
printf 'runner_id=%s runner_version=%s executor=%s\n' "$CI_RUNNER_ID" "$CI_RUNNER_VERSION" "$CI_RUNNER_EXECUTABLE_ARCH"
CI_JOB_TAGS was added in GitLab 19.3 and exposes the
job's configured tags; CI_RUNNER_TAGS identifies the
runner's tags. Compare them, but do not mistake a tag match for
authorization. Also record runner scope/protection in GitLab and the
executor/manager configuration outside the job.
6. Runner eligibility is an intersection, not a single switch
A sensitive job should require tags that only the intended runner pool advertises. The runner itself should be protected when it must not accept ordinary branches. Scope should be as narrow as practical. GitLab documents that protected runners run jobs on protected branches/tags; jobs intended for protected runners should also use tags so they do not fall back to ordinary runners.
For merge-request pipelines, current GitLab behavior is stricter: protected variables/runners can be exposed only when the opt-in setting is enabled, both branches are protected, the triggerer has target-branch permissions, and both branches belong to the same project. A fork MR cannot access those protected resources.
7. Executors change the isolation boundary
| Execution model | Primary boundary | Key risk for untrusted code | Safer use |
|---|---|---|---|
| Shell executor | Runner host user/process permissions | Jobs share the host and can steal code/data from other projects or alter the host. GitLab calls this high risk. | Trusted builds only; dedicated host and minimal credentials/network. |
| Docker, non-privileged | Container namespaces/capabilities on a shared kernel | Kernel, volumes, caches, network and host configuration remain shared boundaries. | Default container path for ordinary untrusted tests, with non-root user, capability reduction and no host socket. |
| Docker privileged | Container security mechanisms are largely disabled | Container breakout/host root impact becomes plausible. | Only isolated, ephemeral, dedicated workers for a narrowly protected workload if unavoidable. |
| Docker socket binding | Container talks to host Docker daemon | GitLab warns this effectively disables container security and can permit host privilege escalation. | Avoid for untrusted jobs; prefer an isolated build mechanism. |
| Kubernetes executor | Per-job pod plus cluster/node controls | Privileged pods, hostPath, broad service accounts, shared nodes/network can collapse isolation. | Namespace/RBAC/pod security/network controls plus dedicated node pools for high-risk workloads. |
| Single-use VM/instance | VM boundary per job | Base-image or hypervisor/provider compromise remains possible, but cross-job residue is reduced. | Strong pattern for privileged or high-risk jobs when destroyed after one use. |
8. Credentials and network are part of the executor
A container can remain perfectly contained and still exfiltrate a
credential it was legitimately given. Model
credential reachability and
network reachability as first-class runner properties. An
untrusted pool should have no protected deployment variables, no
broad cloud identity, no runner-host SSH keys and no route to
production/control networks. If a job needs GitLab access, use the
narrowest supported identity and remember that even
CI_JOB_TOKEN can be stolen while the job runs if the
worker is compromised.
Network segmentation should block unnecessary internal services and cloud metadata endpoints while allowing only the dependencies that the job genuinely needs. “Containerized” is not synonymous with “network-isolated.”
9. Fork pipelines have two materially different execution contexts
By default, a fork merge request pipeline is created and runs in the fork project, using the fork project's CI configuration, runners/resources and project variables. A member of the parent project can deliberately run the fork MR pipeline in the parent; that pipeline uses CI configuration from the fork branch but the parent's CI/CD settings/resources/variables and the triggering member's permissions. GitLab shows a warning in the UI because the code can be malicious.
ci_allow_fork_pipelines_to_run_in_parent_project to
disable new ones.
10. Persistent state can turn yesterday's job into today's attack
GitLab warns specifically about GIT_STRATEGY: fetch on
shared environments because it reuses the prior worktree. A
malicious user can leave executable Git state, and submodule content
can remain accessible through reflogs. Use fetch only when all users
of that shared environment are trusted. For security-sensitive
shared runners, prefer fresh disposable workers; where static
workers are unavoidable, consider a fresh clone/empty strategy
appropriate to the job and GitLab's
FF_ENABLE_JOB_CLEANUP cleanup feature.
Caches, container-image layers and persistent volumes are also reuse channels. They improve performance; they are not trust boundaries.
11. Teardown must not erase the first-failure evidence
For an ephemeral worker, preserve manager logs, pipeline/job IDs, source SHA, runner ID/version/executor, worker identity, job trace, relevant network/provider events and non-secret artifact metadata before the worker disappears. If compromise is plausible, stop assigning new work, isolate the worker, revoke/rotate credentials that could have been exposed, preserve evidence according to policy, then destroy/reimage the worker. A retry creates new evidence and can repeat side effects; it is not an incident-response substitute.
12. Misconceptions that cause runner incidents
| Claim | Why it fails | Safer model |
|---|---|---|
| “The job has the right tag, so it is trusted.” | A repository author can request a tag if CI configuration permits it; tags match capabilities. | Combine tags with protected runner/ref controls and trust-aware job rules. |
| “Non-privileged Docker is a sandbox.” | Shared kernel, volumes, network, caches and daemon integrations remain attack surfaces. | Harden the container and surrounding host/network; use ephemeral workers for stronger reset. |
| “Protected variables are enough.” | A protected runner can still have host credentials or network reachability outside GitLab variables. | Inventory every credential and reachable service on the manager/worker. |
| “Fork pipelines are always isolated in the fork.” | Parent members can run a fork MR pipeline in the parent context when enabled. | Treat that action as an explicit trust decision and review first. |
| “Destroying a worker solves the incident.” | Destruction can also delete evidence; stolen credentials remain valid until revoked/expired. | Preserve evidence, revoke authority, then prove destruction/reimage. |
Knowledge check
Why are runner tags insufficient as a security boundary?
Tags are job-to-runner routing labels. A secure design also constrains runner scope/protection, pipeline/ref trust, executor privilege, credentials and network exposure.
Can a fork merge-request pipeline access protected runners under the GitLab 18.1 protected-resource MR rules?
No. The protected-resource path requires both branches to belong to the same project; fork merge-request pipelines cannot access the protected resources.
Why is Docker socket binding dangerous even when the job container itself is not marked privileged?
The job gains control of the host Docker daemon. GitLab warns that this effectively disables the container security boundary and can allow host privilege escalation/container breakout.
Why is GIT_STRATEGY: fetch risky on a shared
untrusted runner?
It reuses the previous working copy. Git state or submodule content left by one job can influence or leak into later jobs.
What must happen before an ephemeral suspicious worker is destroyed?
Preserve first-failure evidence and identify/revoke any credentials or authority that could have been exposed; then destroy/reimage and record proof of trust reset.
Version and compatibility note
GitLab and GitLab Runner evolve continuously. Treat version-sensitive YAML, runner/executor behavior, APIs, security features, policy controls, tiers, and deprecations as assumptions to verify against the current official GitLab documentation before production use. Preserve the exact project/ref/SHA, compiled configuration, pipeline/job IDs, runner/tool versions, and external target evidence used for any reproducible lab or incident record.
Official references and version notes
Documentation verification date: 2026-09-12. The
current stable GitLab Runner patch used as this chapter's
reproducibility baseline is 19.3.2 (tagged 2026-09-10).
GitLab Runner security guidance treats self-managed runners as
remote-code-execution infrastructure, rates Shell executor as high
risk for untrusted builds, warns that privileged containers and
Docker-socket binding can collapse host isolation, and states that
GIT_STRATEGY: fetch on a shared environment is
appropriate only when all users are trusted. Since GitLab 18.1,
same-project merge-request pipelines can be allowed to use protected
variables/runners only when both source and target branches are
protected, the triggering user has suitable target-branch access,
and both branches belong to the same project; fork merge-request
pipelines cannot access those protected resources. The mandatory
exercises are local simulations: they require no runner registration
token, real secret, privileged container, Docker socket, cloud
account, or production network access. The chapter deliberately
avoids executing privileged containers or mounting host Docker
sockets. Those configurations are shown only as threat-model
examples, never as mandatory lab actions.
- Security for self-managed runners — official reference.
- Configure runners — official reference.
- Runner executors — official reference.
- Shell executor — official reference.
- Docker executor — official reference.
- Use Docker to build Docker images — official reference.
- Merge request pipelines and forks — official reference.
- CI/CD pipelines and protected runner behavior — official reference.
- Pipeline types — official reference.
- CI/CD variables — official reference.
- Predefined CI/CD variables — official reference.
- Advanced Runner configuration — official reference.
- Docker Autoscaler executor — official reference.
- GitLab Runner tags — official reference.
- GitLab 19.3 release notes — official reference.
Current assumptions used in this chapter: Version-sensitive YAML, Runner/executor behavior, APIs, security features, policy controls, tiers, and deprecations must be verified against the exact GitLab, GitLab Runner, tool, and external-system versions used in production. Preserve the exact project/ref/SHA, compiled configuration, pipeline/job IDs, runner/tool versions, and external target evidence used for reproducible work.
Keep the academy open
Support free, practical DevOps education.
Every lesson is designed to remain readable in a browser, downloadable from GitHub, and usable without a paid learning platform. Contributions help expand and maintain the curriculum.
0x716c4Ab160C4B66F31a28AE2448BfF68fc3a2ef0
Send only Ethereum/ERC-20 compatible assets to this
address.