CI/CD Build Patterns, Caching, Test Sharding, Artifact Promotion, and Ephemeral Build Agents: Concepts, Architecture, and Mental Model

1. CI turns hidden workstation assumptions into production failures

A local build can succeed because the developer already has a compatible JDK, a warm dependency cache, previously generated files, locally installed artifacts, credentials, or an old build//target/ tree. An ephemeral CI agent removes those accidental conveniences. That is useful: if the pipeline can rebuild from a reviewed commit plus explicitly approved reusable state, the team gains stronger evidence that the build is actually reproducible.

The goal is not “always delete everything.” The goal is to classify state. Dependencies may be cached. Cacheable task outputs may be shared under a controlled Build Cache. But a release JAR should enter deployment because the pipeline exported and verified that exact JAR—not because a later agent happened to rebuild something with the same filename.

2. Mental model: source → verified toolchain → evidence → immutable promotion

flowchart TD
  A[Reviewed commit] --> B[Clean ephemeral workspace]
  B --> C[Verified Wrapper + JDK/toolchain]
  D[Approved dependency/build caches] --> C
  C --> E[Test shards]
  E --> F[Complete report set]
  C --> G[Package once]
  G --> H[Artifact + SHA-256 + run identity]
  F --> I[Verification gate]
  H --> I
  I --> J[Promotion stage]
  J --> K[Same artifact bytes in release repository]
  L[CI secrets] --> C
  L --> J

The arrows describe trust transitions. The cache can accelerate resolution or task execution, but it does not replace source identity, verification metadata, tests, or artifact checksums. Promotion has no arrow back to compilation: that is deliberate.

3. Ephemeral agents: isolate mutable state, preserve deliberate evidence

An ephemeral agent is a worker whose workspace and process state are treated as disposable after the job. It may still restore approved caches. The distinction is that the pipeline does not depend on undocumented leftovers from a previous job. A clean workspace therefore becomes part of the evidence model, not a ritual.

For Gradle, use the committed Wrapper and consider an isolated GRADLE_USER_HOME in labs or tightly managed CI images. For Maven, the Wrapper plus -Dmaven.repo.local=... can isolate the local repository. Do not globally purge a developer's normal ~/.gradle or ~/.m2 just to simulate CI.

4. Record four identities before trusting the build

Every run should be able to answer four questions: which source revision? which Wrapper/build-tool distribution? which JVM ran the build? and which compiler/toolchain target produced the artifact? Chapter 23 separated those JVM identities; CI must record them again because stages may run on different workers.

git rev-parse HEAD
java -version
./gradlew --version
# Maven lane:
./mvnw --version

Capturing these lines as a small text artifact is stronger than assuming a runner image tag means every stage used the same toolchain.

5. Cache taxonomy: reusable state is not all the same thing

6. Cache keys and poisoning: speed is a trust decision

A cache key determines which jobs may reuse a stored namespace. Correctness inputs commonly include build-tool version, JDK/toolchain identity, dependency lock/verification state, OS/architecture where relevant, and sometimes the source branch or trust class. A cache key that allows an untrusted pull request to write entries later consumed by a protected release branch creates a poisoning path.

Gradle's remote Build Cache is specifically designed for reusable task outputs. Current Gradle guidance commonly has CI populate a remote cache while developers read it. The same idea generalizes: keep cache write authority narrower than cache read authority. Dependency caches and build caches are also different systems; copying build/ into a dependency cache is not a shortcut—it erases provenance.

7. Test sharding: partition execution without partitioning responsibility

A test shard is a deterministic subset of the verification suite assigned to one worker/job. Sharding is useful only if the pipeline can prove which tests were assigned, which ran, and whether the union of all shards equals the intended suite. A fast pipeline that silently omits 5% of tests is a false green.

8. Build once, promote the same artifact bytes

Build once/promote same artifact means the candidate artifact is created in one controlled build, hashed, tested, and then copied or repository-promoted unchanged through environments. Rebuilding for staging and production creates two artifacts that may differ because of time, toolchain, dependency, environment, or configuration drift—even if both claim version 1.0.0.

The handoff record should minimally bind the artifact checksum to source revision, build run, Wrapper/build-tool identity, JDK/toolchain identity, and test evidence. Environment-specific configuration belongs outside the immutable application binary where possible.

9. CI secrets and repository credentials: inject late, expose minimally

Credentials are inputs to repository access or publication, not ordinary build metadata. Do not put tokens in build files, command lines that echo into logs, test reports, or Build Scan values. Prefer the CI platform's secret mechanism, environment/provider APIs that avoid printing the value, least-privilege repository accounts, and stages that only receive secrets when they actually need them.

Untrusted pull requests should not automatically receive release credentials or cache-push authority. A build script is executable code: running an unreviewed change with a production token is equivalent to handing that code the token.

10. Read-only first: inspect before optimizing or mutating

Before introducing caches or shards, record a clean baseline. These commands inspect tool identity and the selected task/test model without changing shared infrastructure:

# Gradle
./gradlew --version
./gradlew tasks --group verification
./gradlew dependencies

# Maven
./mvnw --version
./mvnw -DskipTests help:effective-pom
./mvnw dependency:tree

Only after the baseline is explainable should you cache, shard, or promote. Otherwise performance changes can conceal an existing modeling defect.

11. Production CI build contract

A defensible build pipeline can state: clean source identity; verified Wrapper/tool distribution; explicit JDK/toolchain; reviewed dependency/plugin repositories and verification; cache namespaces with documented trust; deterministic test partition and report collection; package exactly once; checksum the candidate; and promote the same bytes. CI-platform YAML is merely one implementation of that operating contract.

Knowledge check

Official references and version notes

• Gradle 9.7.1 release notes — pinned Gradle baseline.
• Gradle dependency caching — dependency-cache state and guidance for ephemeral builds.
• Gradle Build Cache — local/remote task-output cache and CI push/read trust model.
• Build Cache use cases — CI-produced cache entries and cross-machine reuse.
• Gradle task outcomes — UP-TO-DATE versus FROM-CACHE.
• Gradle on GitLab CI — current Wrapper-first CI guidance and cache considerations.
• Maven release history — Maven 3.9.16 GA baseline.
• Maven Wrapper 3.3.4 — current stable Wrapper baseline.
• Maven Surefire test goal — explicit test selection and failure behavior.

Version-sensitive statements were rechecked against primary documentation on 2026-08-24. Mandatory labs remain local/free; hosted CI, remote caches, artifact repositories, and secret stores are represented as optional production mappings only.