Chapter 31Lesson 01~195 minutes

Runner Security, Isolation Boundaries, Privileged Containers, Fork Pipelines, Untrusted Code, and Threat Modeling: Concepts, Architecture, and Mental Model

Threat-model every CI job as repository-controlled code: classify pipeline trust, prove runner eligibility, inspect executor and host exposure, constrain credentials/network/devices, preserve incident evidence, and reset trust by cleanup or destruction.

Runner securityThreat modelTrust boundaryProtected runnersIsolation

Learning objectives

  • Model a GitLab Runner as privileged infrastructure that executes repository-controlled code, not as a neutral build box.
  • Trace trust classification → runner eligibility → executor boundary → host/network/device/credential exposure → job execution → evidence → trust reset.
  • Explain why runner tags, protection, executor isolation and ephemeral lifecycle solve different parts of the threat model.
  • Distinguish fork merge-request context, same-project protected-ref context, and post-merge/release context before exposing protected resources.
  • Preserve pipeline/job/runner evidence before destroying or reimaging a suspicious worker.

1. The practical problem: Chapter 30's capacity is also an attack surface

Chapter 30 treated a runner fleet as a capacity system. Chapter 31 keeps that model, but changes the default assumption about the job: repository code, test scripts, build plugins, downloaded dependencies and CI configuration can all be controlled by someone who should not control your runner host. A fast fleet that lets a forked merge request reach a privileged Docker daemon or production credential is not a reliable fleet.

The security question therefore comes before “Which executor is fastest?” First classify who controls the code and ref, then determine which runner may accept it, then inspect what the executor exposes. Only after those boundaries are known should you optimize cache reuse, warm workers or privileged image-building.

Chapter invariant: treat every self-managed runner as remote-code-execution infrastructure. A job is safe enough to run only when its trust class, runner eligibility, executor boundary, credential scope, network reachability and teardown policy are all explicit.

2. Mental model: code trust flows into infrastructure trust

A pipeline starts with an exact source event, project, ref and SHA. Those facts determine a trust class; they do not automatically make code benign. The compiled CI configuration decides which jobs exist and which tags they request. GitLab then matches jobs to eligible runners using runner scope, tags and protection rules. The runner manager prepares an executor. That executor exposes some amount of host kernel, filesystem, devices, network and credentials. The job can behave correctly or maliciously. Finally, evidence must survive long enough to investigate, while the execution boundary is cleaned, destroyed or reimaged before it is trusted again.

Causal runner-security path
            flowchart TD
            A[Pipeline source + project + ref + SHA] --> B[Trust classification]
            B --> C[Compiled rules + requested tags]
            C --> D[Runner scope + tags + protected eligibility]
            D --> E[Executor / worker boundary]
            E --> F[Host + network + device + credential exposure]
            F --> G[Repository-controlled job executes]
            G --> H[Job trace + runner/manager evidence]
            H --> I[Destroy / reimage / verified cleanup]
            I --> J[Trust reset before reuse]
          

Each arrow is a distinct control. A protected runner cannot repair an unsafe host socket mount. A non-privileged container cannot compensate for a job that receives a broad production token. An ephemeral VM does not help if its logs disappear before incident collection.

3. Keep the security state domains separate

State layer Read-only evidence Security question
Pipeline/source CI_PIPELINE_SOURCE, project IDs/paths, MR source/target project, ref, CI_COMMIT_REF_PROTECTED, CI_COMMIT_SHA Who controls the code and which immutable revision is executing?
Compiled configuration Merged YAML, workflow:rules, job rules, requested tags, job graph Did configuration create the job and ask for a sensitive runner pool?
Runner eligibility Runner ID/scope, protected flag, tags, paused/online state, CI_JOB_TAGS, CI_RUNNER_TAGS Could this job legally match this runner? Tags are routing labels, not an authorization proof by themselves.
Runner manager Runner version, authentication identity, host, logs, concurrency Which trusted control process requested and dispatched the job?
Executor/worker Executor, container/pod/VM ID, privileged/rootless state, namespaces/capabilities, reuse count What isolation boundary actually contains repository-controlled code?
Host/device exposure Docker socket/pipe, host mounts, devices, host PID/network namespaces, kernel Can job code cross the executor boundary into the worker or manager host?
Network exposure Routes, DNS, metadata endpoints, internal services, egress policy What can malicious code reach even without escaping the container?
Identity/credentials Protected-variable eligibility, CI_JOB_TOKEN scope, OIDC audience/claims, deploy credentials What authority is reachable during this exact job? Never capture values.
Reuse state Git worktree strategy, caches, image pull policy, volumes, home directories Can one job read or poison residue that another job will trust?
Incident/cleanup Pipeline/job/runner IDs, manager logs, worker identity, preserved artifacts, credential-rotation record, destruction/reimage proof Can you investigate the first failure and prove the compromised trust boundary was reset?

4. Trust class is about control, not about branch names alone

Example Conservative trust class Why
Fork merge request Untrusted A contributor controls the fork branch and CI content. GitLab documents that fork MR pipelines normally run in the fork; a parent-project member can deliberately run one in the parent, where parent resources become relevant.
Same-project feature branch Review/untrusted by default Project membership does not make every commit safe; Developers can intentionally or accidentally submit dangerous code.
Same-project MR with both branches protected and authorized triggerer Eligible for protected resources only if the project setting allows it GitLab 18.1+ adds a controlled path, but the conditions are explicit and the code still deserves review.
Protected release tag created by restricted maintainers Higher trust Protected ref governance narrows who can create the ref; runner protection and tags can further narrow execution.
Post-merge protected main deployment Higher trust, not infinite trust It can use narrowly scoped deployment identity, but dependencies/configuration can still be compromised.

5. Read-only inspection before changing runners

Start with evidence. In a real GitLab job, these values are safe metadata; do not print any token or secret value:

printf 'source=%s\n' "$CI_PIPELINE_SOURCE"
printf 'project=%s id=%s\n' "$CI_PROJECT_PATH" "$CI_PROJECT_ID"
printf 'ref=%s protected=%s sha=%s\n' "$CI_COMMIT_REF_NAME" "$CI_COMMIT_REF_PROTECTED" "$CI_COMMIT_SHA"
printf 'pipeline=%s job=%s\n' "$CI_PIPELINE_ID" "$CI_JOB_ID"
printf 'job_tags=%s runner_tags=%s\n' "$CI_JOB_TAGS" "$CI_RUNNER_TAGS"
printf 'runner_id=%s runner_version=%s executor=%s\n'   "$CI_RUNNER_ID" "$CI_RUNNER_VERSION" "$CI_RUNNER_EXECUTABLE_ARCH"

CI_JOB_TAGS was added in GitLab 19.3 and exposes the job's configured tags; CI_RUNNER_TAGS identifies the runner's tags. Compare them, but do not mistake a tag match for authorization. Also record runner scope/protection in GitLab and the executor/manager configuration outside the job.

6. Runner eligibility is an intersection, not a single switch

A sensitive job should require tags that only the intended runner pool advertises. The runner itself should be protected when it must not accept ordinary branches. Scope should be as narrow as practical. GitLab documents that protected runners run jobs on protected branches/tags; jobs intended for protected runners should also use tags so they do not fall back to ordinary runners.

For merge-request pipelines, current GitLab behavior is stricter: protected variables/runners can be exposed only when the opt-in setting is enabled, both branches are protected, the triggerer has target-branch permissions, and both branches belong to the same project. A fork MR cannot access those protected resources.

7. Executors change the isolation boundary

Execution model Primary boundary Key risk for untrusted code Safer use
Shell executor Runner host user/process permissions Jobs share the host and can steal code/data from other projects or alter the host. GitLab calls this high risk. Trusted builds only; dedicated host and minimal credentials/network.
Docker, non-privileged Container namespaces/capabilities on a shared kernel Kernel, volumes, caches, network and host configuration remain shared boundaries. Default container path for ordinary untrusted tests, with non-root user, capability reduction and no host socket.
Docker privileged Container security mechanisms are largely disabled Container breakout/host root impact becomes plausible. Only isolated, ephemeral, dedicated workers for a narrowly protected workload if unavoidable.
Docker socket binding Container talks to host Docker daemon GitLab warns this effectively disables container security and can permit host privilege escalation. Avoid for untrusted jobs; prefer an isolated build mechanism.
Kubernetes executor Per-job pod plus cluster/node controls Privileged pods, hostPath, broad service accounts, shared nodes/network can collapse isolation. Namespace/RBAC/pod security/network controls plus dedicated node pools for high-risk workloads.
Single-use VM/instance VM boundary per job Base-image or hypervisor/provider compromise remains possible, but cross-job residue is reduced. Strong pattern for privileged or high-risk jobs when destroyed after one use.

8. Credentials and network are part of the executor

A container can remain perfectly contained and still exfiltrate a credential it was legitimately given. Model credential reachability and network reachability as first-class runner properties. An untrusted pool should have no protected deployment variables, no broad cloud identity, no runner-host SSH keys and no route to production/control networks. If a job needs GitLab access, use the narrowest supported identity and remember that even CI_JOB_TOKEN can be stolen while the job runs if the worker is compromised.

Network segmentation should block unnecessary internal services and cloud metadata endpoints while allowing only the dependencies that the job genuinely needs. “Containerized” is not synonymous with “network-isolated.”

9. Fork pipelines have two materially different execution contexts

By default, a fork merge request pipeline is created and runs in the fork project, using the fork project's CI configuration, runners/resources and project variables. A member of the parent project can deliberately run the fork MR pipeline in the parent; that pipeline uses CI configuration from the fork branch but the parent's CI/CD settings/resources/variables and the triggering member's permissions. GitLab shows a warning in the UI because the code can be malicious.

Security consequence: running a fork MR in the parent project is not “just another test.” Review the CI/configuration changes and the code before granting parent execution context. If the organization does not need parent-project pipelines for forks, the project API exposes ci_allow_fork_pipelines_to_run_in_parent_project to disable new ones.

10. Persistent state can turn yesterday's job into today's attack

GitLab warns specifically about GIT_STRATEGY: fetch on shared environments because it reuses the prior worktree. A malicious user can leave executable Git state, and submodule content can remain accessible through reflogs. Use fetch only when all users of that shared environment are trusted. For security-sensitive shared runners, prefer fresh disposable workers; where static workers are unavoidable, consider a fresh clone/empty strategy appropriate to the job and GitLab's FF_ENABLE_JOB_CLEANUP cleanup feature.

Caches, container-image layers and persistent volumes are also reuse channels. They improve performance; they are not trust boundaries.

11. Teardown must not erase the first-failure evidence

For an ephemeral worker, preserve manager logs, pipeline/job IDs, source SHA, runner ID/version/executor, worker identity, job trace, relevant network/provider events and non-secret artifact metadata before the worker disappears. If compromise is plausible, stop assigning new work, isolate the worker, revoke/rotate credentials that could have been exposed, preserve evidence according to policy, then destroy/reimage the worker. A retry creates new evidence and can repeat side effects; it is not an incident-response substitute.

12. Misconceptions that cause runner incidents

Claim Why it fails Safer model
“The job has the right tag, so it is trusted.” A repository author can request a tag if CI configuration permits it; tags match capabilities. Combine tags with protected runner/ref controls and trust-aware job rules.
“Non-privileged Docker is a sandbox.” Shared kernel, volumes, network, caches and daemon integrations remain attack surfaces. Harden the container and surrounding host/network; use ephemeral workers for stronger reset.
“Protected variables are enough.” A protected runner can still have host credentials or network reachability outside GitLab variables. Inventory every credential and reachable service on the manager/worker.
“Fork pipelines are always isolated in the fork.” Parent members can run a fork MR pipeline in the parent context when enabled. Treat that action as an explicit trust decision and review first.
“Destroying a worker solves the incident.” Destruction can also delete evidence; stolen credentials remain valid until revoked/expired. Preserve evidence, revoke authority, then prove destruction/reimage.

Knowledge check

Why are runner tags insufficient as a security boundary?

Can a fork merge-request pipeline access protected runners under the GitLab 18.1 protected-resource MR rules?

Why is Docker socket binding dangerous even when the job container itself is not marked privileged?

Why is GIT_STRATEGY: fetch risky on a shared untrusted runner?

What must happen before an ephemeral suspicious worker is destroyed?

Next lesson

Continue to the guided workflow — Lesson 2 — Local runner-policy simulation

You will turn the mental model into a deterministic local policy evaluator, prove a fork-like job is denied protected capacity, and inspect executor-risk examples without running a privileged container.

Version and compatibility note

GitLab and GitLab Runner evolve continuously. Treat version-sensitive YAML, runner/executor behavior, APIs, security features, policy controls, tiers, and deprecations as assumptions to verify against the current official GitLab documentation before production use. Preserve the exact project/ref/SHA, compiled configuration, pipeline/job IDs, runner/tool versions, and external target evidence used for any reproducible lab or incident record.

Official references and version notes

Documentation verification date: 2026-09-12. The current stable GitLab Runner patch used as this chapter's reproducibility baseline is 19.3.2 (tagged 2026-09-10). GitLab Runner security guidance treats self-managed runners as remote-code-execution infrastructure, rates Shell executor as high risk for untrusted builds, warns that privileged containers and Docker-socket binding can collapse host isolation, and states that GIT_STRATEGY: fetch on a shared environment is appropriate only when all users are trusted. Since GitLab 18.1, same-project merge-request pipelines can be allowed to use protected variables/runners only when both source and target branches are protected, the triggering user has suitable target-branch access, and both branches belong to the same project; fork merge-request pipelines cannot access those protected resources. The mandatory exercises are local simulations: they require no runner registration token, real secret, privileged container, Docker socket, cloud account, or production network access. The chapter deliberately avoids executing privileged containers or mounting host Docker sockets. Those configurations are shown only as threat-model examples, never as mandatory lab actions.

Current assumptions used in this chapter: Version-sensitive YAML, Runner/executor behavior, APIs, security features, policy controls, tiers, and deprecations must be verified against the exact GitLab, GitLab Runner, tool, and external-system versions used in production. Preserve the exact project/ref/SHA, compiled configuration, pipeline/job IDs, runner/tool versions, and external target evidence used for reproducible work.

Keep the academy open

Support free, practical DevOps education.

Every lesson is designed to remain readable in a browser, downloadable from GitHub, and usable without a paid learning platform. Contributions help expand and maintain the curriculum.

Ethereum / ERC-20
0x716c4Ab160C4B66F31a28AE2448BfF68fc3a2ef0 Send only Ethereum/ERC-20 compatible assets to this address.