Caches, Cache Keys, Fallback Keys, Distributed Cache, Dependency Reuse, and Cache Correctness: Concepts, Architecture, and Mental Model

1. The practical problem: a fast dependency restore can still be wrong

Chapter 10 established that artifacts carry retained build outputs and evidence. Cache solves a different problem: avoiding repeated work such as downloading or unpacking dependencies. That distinction is operationally important. A cache is allowed to miss, be evicted, be unavailable on another runner, or contain data created by an earlier pipeline. A correct job must still be able to reconstruct and verify its required dependencies.

The dangerous mental shortcut is “cache hit = dependencies are correct.” A hit only means GitLab Runner found an archive under a matching cache key and restored it. Correctness still depends on whether the key encoded every input that changes dependency compatibility, whether the restored files are trusted for that scope, and whether the dependency manager or build verifies its own lockfile/checksums.

2. Mental model: correctness inputs → key → lookup → restore → verify → optional update

Begin with the inputs that determine whether cached dependency data is reusable: lockfile content, language/runtime version, OS/architecture when native binaries are involved, dependency-manager format, and sometimes build flags. Those inputs produce a cache key. Runner asks its local cache or configured distributed backend for that key. A miss is normal. A hit restores files into the job workspace. The job then performs its normal dependency verification and work. Only after successful or policy-allowed execution should a job update cache state.

            flowchart TD
              A[Lockfile + toolchain + platform inputs] --> B[Cache key]
              B --> C[Local or distributed lookup]
              C -->|miss| D[Recreate dependencies]
              C -->|hit| E[Restore cached files]
              D --> F[Dependency verification]
              E --> F
              F --> G[Tests/build output]
              G --> H{Policy allows update?}
              H -->|yes| I[Archive/upload cache]
              H -->|no| J[Leave cache unchanged]
              G --> K[Artifacts/reports prove output]
          

The final split matters: cache update is not the evidence product. Artifacts/reports from Chapter 10 remain the retained evidence path. A cache can accelerate the next job or next pipeline, but a deployable binary should not be trusted because it happened to be present in a cache archive.

3. Separate the states before changing anything

4. Cache versus artifact: similar syntax, different contract

Both cache and artifact paths are relative to the project directory, but their contracts are different. Cache is for reusable inputs or intermediate dependency data whose absence should only cost time. Artifacts are explicit outputs/evidence associated with a producing job and pipeline. Caches can span pipelines; artifacts are tied to producer identity and retention rules.

5. A cache key is an invalidation contract

A key should change when a correctness-relevant input changes and remain stable when only irrelevant source files change. A branch-only key often over-invalidates or under-specifies. A single global key can allow unrelated jobs to overwrite each other. GitLab provides cache:key:files to derive a key from file contents and cache:key:files_commits to derive it from the most recent commits touching specified files. Current docs allow up to two file paths/patterns in either form.

default:
  image: python:3.11.9-slim-bookworm
  cache:
    key:
      files:
        - deps.lock
      prefix: deps-py311-linux-amd64
    paths:
      - .deps-cache/
    policy: pull-push

The prefix carries toolchain/platform identity while files carries dependency-content identity. A native dependency cache might need a more specific platform/ABI prefix. A pure download cache might need less. The key should describe reuse safety, not merely convenience.

6. Fallback keys are ordered warm-start hints, not correctness exceptions

Each cache entry can currently define up to five ordered fallback_keys. Runner tries the primary key first, then each fallback in order, and finally the optional global CACHE_FALLBACK_KEY. The first successful retrieval stops the search. A fallback is useful when a feature branch can safely start from a default-branch dependency cache, but the job must still validate or refresh data against the feature branch lockfile.

cache:
  key:
    files:
      - deps.lock
    prefix: deps-py311-linux-amd64
  fallback_keys:
    - deps-py311-main
    - deps-py311-seed
  paths:
    - .deps-cache/
  policy: pull

7. Pull/push policy controls who is allowed to mutate performance state

The default pull-push policy restores a cache at job start and uploads updates at job end. That is convenient but often too broad. A safer pattern is a small number of trusted “writer” jobs using pull-push or push, while many test jobs use pull. Consumers then gain speed without every parallel job racing to overwrite the same key.

8. Protected and non-protected cache namespaces are a security boundary

GitLab separates protected and non-protected cache use by default. Current documentation also records role/ref-sensitive key suffix behavior in newer GitLab versions. The reason is straightforward: an untrusted feature branch should not be able to populate dependency files that a protected pipeline later consumes as though they came from the same trust domain.

cache:unprotect: true deliberately shares a cache across that boundary. Treat that as a security decision, not a performance toggle. Use it only when all writers are trusted and the cached content is independently verified strongly enough that poisoning cannot change execution.

9. Local versus distributed cache is runner topology state

By default, cache lives with the runner. A job scheduled to another standalone runner may not see the first runner’s archive. Distributed cache makes the archive reachable to a fleet, commonly through S3/S3-compatible storage, Google Cloud Storage, or Azure Blob storage. GitLab.com instance runners already use distributed cache patterns.

Distributed cache improves availability but adds transfer latency, object-storage cost, credentials, lifecycle policy, and a larger poisoning blast radius if keys/scopes are careless. Therefore “more shared” is not automatically “better.” Measure archive size, compression time, upload/download time, and hit rate before deciding.

10. Read-only inspection before changing cache state

Start with evidence that does not mutate anything. Record the pipeline ID, job ID, source SHA, pipeline source, runner ID/version/executor, compiled cache configuration, lockfile digest, and the cache lines in the job trace. A trace normally reveals whether the runner searched for, extracted, or failed to find a cache. Do not dump all environment variables and do not clear caches as the first diagnostic step.

printf 'source=%s
sha=%s
pipeline=%s
job=%s
runner=%s
'   "$CI_PIPELINE_SOURCE" "$CI_COMMIT_SHA" "$CI_PIPELINE_ID" "$CI_JOB_ID" "${CI_RUNNER_ID:-unknown}"
sha256sum deps.lock
python --version

11. Misconceptions to remove now

• “Hit means correct.” No. It means an archive was restored for the key.
• “Cache replaces artifacts.” No. Cache is optional acceleration; artifacts are retained outputs/evidence.
• “One key is simpler.” It can make unrelated jobs overwrite each other or reuse incompatible binaries.
• “Clear all caches when anything looks stale.” That destroys evidence and unrelated warm state; first identify the exact key/scope.
• “Distributed cache is globally shared truth.” It is still mutable runner-managed performance state and requires scoped credentials/lifecycle controls.

12. Micro-lab: predict the safe key before writing YAML

Suppose a Python project has deps.lock, runs on Python 3.11 and 3.12, and sometimes executes on Linux/amd64 and Windows/amd64. Decide which dimensions belong in the cache key and which do not. Then answer: if only README.md changes, should the dependency cache invalidate? If deps.lock changes? If the Python minor version changes? If the runner architecture changes and the cache contains native wheels?

The point is not to make the longest possible key. The point is to make reuse correspond to compatibility.

Knowledge check

Official references and version notes

Documentation verification date: 2026-09-11. Cache behavior is version-sensitive at both GitLab and GitLab Runner layers. Re-check the deployed GitLab/Runner versions for Self-Managed or Dedicated installations, especially when runner topology, object storage, protected-cache behavior, or cache archive implementation differs from GitLab.com.

• Caching in GitLab CI/CD — cache-versus-artifact boundary, fallback keys, protected-cache separation, availability, storage, clearing, and troubleshooting.
• CI/CD YAML syntax reference — authoritative cache, cache:key, cache:key:files, cache:key:files_commits, cache:key:prefix, cache:fallback_keys, cache:policy, cache:when, and cache:unprotect semantics.
• CI/CD caching examples — dependency-manager patterns and lockfile-aware examples.
• GitLab Runner advanced configuration — distributed cache backend configuration, sharing, paths, and archive limits.
• Speed up job execution — distributed cache backends and transfer-performance considerations.
• Job artifacts — the retained-output mechanism that must not be confused with cache.