Caches, Cache Keys, Fallback Keys, Distributed Cache, Dependency Reuse, and Cache Correctness: Configuration, Design Choices, and Tradeoffs

1. Design from invalidation and trust boundaries, not from cache syntax

A cache design is good when a reviewer can answer four questions: what makes the content reusable, who can write it, where it is stored, and what independent evidence proves the job is correct if the cache is stale or missing. These questions scale from one runner to a fleet.

2. One cache versus multiple caches

GitLab currently supports up to four cache entries per job. Multiple caches are useful when dependency families have different invalidation inputs or sizes—for example Python package downloads and frontend package downloads. They are harmful when split mechanically into many archives whose compression/transfer overhead exceeds their saved work.

3. Exact keys, lockfile keys, prefixes, and fallback keys

A static exact key is appropriate only when some external process changes the key deliberately. A branch key isolates branches but can waste reuse when lockfiles are identical. cache:key:files often gives a better dependency contract. A prefix can add toolchain/OS/architecture identity. Fallback keys provide warm starts but broaden the set of possible restored inputs and therefore increase the importance of validation.

4. Pull, push, and pull-push: separate readers from writers

Cache poisoning risk grows with the number of writers. A common production pattern is to let a dedicated dependency-preparation job use pull-push while parallel test jobs use pull. A special cache-warmer can use push after reconstructing dependencies from trusted sources. This reduces upload contention and makes mutation ownership reviewable.

Policy choice is independent from pipeline authorization. A low-trust branch with pull-push can still be dangerous if it reaches a shared key. Protected-cache separation, runner trust, key scope, and dependency verification must align.

5. Local versus distributed cache

Local cache is operationally simple and can be extremely fast when the same runner executes related jobs. It is unreliable as a fleet-wide optimization because another runner may not have the archive. Distributed cache moves the archive to object storage reachable by multiple runners. Current GitLab Runner docs support S3/S3-compatible, GCS, and Azure backends.

6. Protected separation versus broad reuse

Default protected/non-protected separation should be viewed as a security feature. Disabling it or using cache:unprotect may improve reuse but increases the writer population that can influence a cache later read by trusted jobs. If you cross that boundary, compensate with content-addressed packages, strong package-manager verification, read-only consumers, and a narrow set of trusted writers.

7. Speed versus archive/transfer cost

A cache is a net win only when saved dependency work exceeds compression, upload, download, extraction, and backend latency. A 3 GB cache that saves 20 seconds of dependency work but adds 45 seconds of transfer has negative value. Measure the critical path, not only cache hit percentage.

Also consider backend lifecycle cost: distributed caches accumulate mutable archives. Apply object-storage lifecycle rules and runner-side limits instead of assuming GitLab project artifact retention settings manage runner cache objects.

8. Cached downloads versus deployable outputs

Use cache for dependency reuse. Use artifacts/packages/container registry objects for outputs that need producer identity, retention, promotion, or deployment. Do not take a binary from a mutable dependency cache and call it a release simply because its key contains a source SHA. If the output matters to deployment, Chapter 10’s build-once/digest-verification model applies.

9. Worked scenario: choose a cache for a polyglot monorepo

A repository has Python and Node services. Python jobs use CPython 3.11 on Linux; Node jobs use Node 24. Both share an autoscaled runner fleet. Feature branches are untrusted, while main produces release candidates. Choose an approach and justify it.

10. Decision checklist

1. List correctness-relevant dependency and toolchain inputs.
2. Decide which inputs become key content versus key prefix.
3. Define writer and reader jobs separately.
4. Keep protected/unprotected scopes separated unless there is a reviewed reason not to.
5. Choose local/distributed storage from actual runner topology.
6. Measure archive size, hit rate, transfer time, and end-to-end critical-path impact.
7. Verify that every consumer remains correct after a miss and rejects incompatible restored data.
8. Keep deployable outputs in artifacts/registry/package systems with immutable identity.

Knowledge check

Official references and version notes

Documentation verification date: 2026-09-11. Cache behavior is version-sensitive at both GitLab and GitLab Runner layers. Re-check the deployed GitLab/Runner versions for Self-Managed or Dedicated installations, especially when runner topology, object storage, protected-cache behavior, or cache archive implementation differs from GitLab.com.

• Caching in GitLab CI/CD — cache-versus-artifact boundary, fallback keys, protected-cache separation, availability, storage, clearing, and troubleshooting.
• CI/CD YAML syntax reference — authoritative cache, cache:key, cache:key:files, cache:key:files_commits, cache:key:prefix, cache:fallback_keys, cache:policy, cache:when, and cache:unprotect semantics.
• CI/CD caching examples — dependency-manager patterns and lockfile-aware examples.
• GitLab Runner advanced configuration — distributed cache backend configuration, sharing, paths, and archive limits.
• Speed up job execution — distributed cache backends and transfer-performance considerations.
• Job artifacts — the retained-output mechanism that must not be confused with cache.