CI/CD Build Patterns, Caching, Test Sharding, Artifact Promotion, and Ephemeral Build Agents: Configuration, Design Choices, and Tradeoffs

1. Dependency cache versus Build Cache

A dependency cache avoids redownloading external modules/plugins. A Build Cache reuses outputs of cacheable tasks whose modeled inputs produce the same cache key. They solve different bottlenecks and carry different poisoning risks. Restoring dependencies cannot legitimately make compileJava FROM-CACHE; a Build Cache can.

2. One large job versus test shards

Sharding reduces wall-clock time only when independent test work is large enough to outweigh duplicate startup, checkout, dependency resolution, JVM startup, and report-merge cost. A two-minute suite may become slower when split across ten cold agents. A forty-minute suite with stable class-level partitions may benefit substantially.

Prefer deterministic assignments based on tags, test classes, historical timing data with a reviewed manifest, or a stable hash. Never define shards by nondeterministic filesystem enumeration without validating the final union.

3. Persistent versus ephemeral agents

Persistent agents can have warm local caches and long-lived processes, but they accumulate hidden state. Ephemeral agents make isolation easier to reason about but may pay more bootstrap cost. Neither model eliminates the need for Wrapper/JDK verification, secret scoping, or cache policy.

4. Rebuild per environment versus immutable promotion

Rebuilding for dev, staging, and production can appear “clean,” but it destroys artifact identity. The correct isolation boundary is environment configuration and deployment authorization, not compilation. Build one candidate, test it, hash/sign it, and promote or copy that same object. If policy requires environment-specific assembly, then those assemblies are distinct artifacts and need distinct identities—they should not masquerade as the same release.

5. Provider YAML versus portable build scripts

CI YAML should orchestrate jobs, permissions, artifacts, caches, approvals, and runner placement. Build correctness—tasks, dependency policy, test suite composition, reproducible packaging—belongs in Maven/Gradle project configuration or repository-owned scripts. This keeps ./gradlew ciTest jar or an equivalent Maven command meaningful on a developer machine and another CI provider.

6. Trusted and untrusted pipeline classes

Pull requests from forks, protected branches, release tags, and scheduled dependency updates often have different trust levels. Design cache write authority, secret access, and publication rights accordingly. A useful policy is: untrusted jobs may read only carefully selected public/reviewed cache state; protected jobs may publish validated Build Cache entries and release artifacts. Exact CI features differ by platform, but the boundary remains.

7. Decision table for a mixed Maven/Gradle estate

8. Worked scenario

A 20-module Gradle service takes 18 minutes: 3 minutes dependency download on cold agents, 12 minutes tests, 3 minutes compile/package. First, enable a verified dependency cache and measure cold/warm differences. Second, split the 12-minute suite into two balanced deterministic shards, preserving XML reports and a coverage manifest. Third, export the single packaged artifact and checksum. Do not add a broad cache containing build/ or rebuild the JAR in deployment merely because it seems simpler.

If later measurement shows compile tasks are stable/cacheable, introduce a protected remote Build Cache as a separate change. One variable at a time keeps performance causality visible.

9. Boundaries this chapter does not own

JDK image patching belongs to image/runner management; repository-manager retention and promotion rules belong to artifact-repository governance; branch protection and secret approval rules belong to CI-platform governance; deployment rollout belongs to delivery/platform courses. This chapter defines the build-tool contract those systems must preserve.

Knowledge check

Official references and version notes

• Gradle 9.7.1 release notes — pinned Gradle baseline.
• Gradle dependency caching — dependency-cache state and guidance for ephemeral builds.
• Gradle Build Cache — local/remote task-output cache and CI push/read trust model.
• Build Cache use cases — CI-produced cache entries and cross-machine reuse.
• Gradle task outcomes — UP-TO-DATE versus FROM-CACHE.
• Gradle on GitLab CI — current Wrapper-first CI guidance and cache considerations.
• Maven release history — Maven 3.9.16 GA baseline.
• Maven Wrapper 3.3.4 — current stable Wrapper baseline.
• Maven Surefire test goal — explicit test selection and failure behavior.

Version-sensitive statements were rechecked against primary documentation on 2026-08-24. Mandatory labs remain local/free; hosted CI, remote caches, artifact repositories, and secret stores are represented as optional production mappings only.