Advanced Compose: Profiles, depends_on, Health Conditions, Overrides, Includes, Watch, and Multi-File Design: Diagnostics, Failure Modes, Security, and Performance

1. Evidence-first diagnostic sequence

1. Preserve first-failure command output, timestamps, ordered -f files, active profiles, and relevant environment inputs.
2. Capture docker version, docker info, context, and docker compose version.
3. Run docker compose ... config --quiet and save the normalized model.
4. Confirm exact image/build identity and project labels.
5. Inspect container/process/health state.
6. Inspect mounts, path ownership, networks, and resources only as relevant.
7. Inspect external dependency evidence if the model/runtime is correct.
8. Change the smallest causal input and rerun only the affected scope.

docker compose -f compose.yaml -f overlays/compose.dev.yaml config   > evidence/first-failure.normalized.yaml

docker compose -f compose.yaml -f overlays/compose.dev.yaml ps -a   > evidence/first-failure.ps.txt

docker ps -a --filter label=com.docker.compose.project=da-compose16   --no-trunc > evidence/first-failure.engine-containers.txt

2. Intentionally broken example: an override path resolved from the wrong directory

Suppose overlays/compose.dev.yaml contains:

services:
  app:
    build:
      context: ../app   # WRONG assumption: "relative to overlays/"

If the base file is the repository root compose.yaml, ordinary multi-file merge resolves this relative path from the base file. The result may point outside the intended project or fail with a missing build context.

Preserve the failure:

docker compose -f compose.yaml -f overlays/compose.dev.yaml config   > evidence/bad-path.normalized.yaml 2> evidence/bad-path.stderr.txt || true

grep -n "context:" evidence/bad-path.normalized.yaml || true

Interpretation: this is model/path state, not an Engine networking, runtime, or image-store failure. Correct the path to ./app (relative to the base file), normalize again, then build only the affected app if needed.

3. “The service is missing” can be correct profile behavior

A debug service with profiles: [debug] will not appear in ordinary baseline execution. Before assuming Compose ignored the service, compare model views:

docker compose config --services
docker compose --profile debug config --services
docker compose config --profiles

If the service appears only with the profile active, there is no daemon failure. The decision layer is profile selection. If another service references a profile-gated service in a way that makes the model invalid under the active profile set, Compose should report the configuration inconsistency rather than silently inventing the dependency.

4. A healthcheck can be healthy and still test the wrong contract

A common anti-pattern is a database healthcheck that only proves the process exists while the dependent application requires a specific database/schema/authentication path. Conversely, an overly deep healthcheck can create load or require credentials that should not be exposed merely for liveness.

Preserve:

docker inspect <dependency-container>   --format '{{json .State.Health}}' > evidence/dependency-health.json

docker compose logs --timestamps dependency app   > evidence/dependency-app-logs.txt

Then compare what the healthcheck proves with what the app requires. Correct the probe or the application's retry/error handling; do not merely increase retries until the symptom disappears.

5. Watch loops and accidental sensitive synchronization

Watch should target a narrow source tree. Dangerous patterns include watching the entire repository when it contains .env, generated credentials, evidence artifacts, build outputs, package caches, or files that the container itself rewrites into a synchronized path.

Example of a poor rule:

develop:
  watch:
    - action: sync
      path: .
      target: /workspace

Repair it by selecting only the source directory and ignoring generated/high-churn content:

develop:
  watch:
    - action: sync
      path: ./app
      target: /site
      ignore:
        - Dockerfile
        - "*.key"
        - "*.pem"

Do not test this lesson with real secrets. The patterns are illustrative; production secret handling belongs to explicit secret/configuration mechanisms, not Watch sync.

6. YAML looks plausible, normalized model is wrong

Compose merge behavior is field-aware; it is not safe to reason from generic YAML merge intuition. If an override adds ports, volumes, environment keys, or commands, inspect the normalized service:

docker compose -f compose.yaml -f compose.dev.yaml config app
# Or capture all services and compare revisions:
docker compose -f compose.yaml -f compose.dev.yaml config > evidence/model-a.yaml
docker compose -f compose.yaml -f compose.test.yaml config > evidence/model-b.yaml

Fix the source model, not the runtime container. Editing a running container cannot repair the declarative cause.

7. Circular or fragile dependency design

If service A waits for B while B operationally depends on A, no amount of longer startup delay makes the architecture robust. Use dependency graphs to represent genuine startup contracts and redesign circular readiness assumptions. For eventual consistency, let applications retry and converge instead of encoding every relationship as a hard startup gate.

8. Reconciliation evidence: did Compose recreate the object?

docker compose ps -q app > evidence/app-id.before.txt
docker image inspect da-compose16-app:lab --format '{{.Id}}' > evidence/app-image.before.txt

# Apply only the intended corrected model scope:
docker compose -f compose.yaml -f overlays/compose.dev.yaml up -d app

docker compose ps -q app > evidence/app-id.after.txt
docker image inspect da-compose16-app:lab --format '{{.Id}}' > evidence/app-image.after.txt

diff -u evidence/app-id.before.txt evidence/app-id.after.txt || true

A changed container ID proves recreation. An unchanged ID can prove no replacement happened, but still inspect the relevant runtime state before concluding the application is correct.

9. Troubleshooting shortcuts to reject

10. Smallest-scope correction checklist

• Model error: fix YAML/input, rerun config; do not touch Engine state yet.
• Profile error: change active profile set and inspect services before up.
• Health error: correct probe or application retry, then recreate/restart only the affected service if necessary.
• Watch rule error: stop Watch, narrow the rule, validate, resume only the affected service.
• Image/build mismatch: record current digest/ID, rebuild only intended service, then compare identity.

Knowledge check

Official references and version notes

• Docker Docs — Merge Compose files: ordered-file merge rules and base-file path resolution.
• Docker Docs — Include Compose files: per-included-model project-directory semantics and modular application models.
• Docker Docs — Profiles: conditional services, explicit profile activation, and targeted-service behavior.
• Docker Docs — Startup order: service_started, service_healthy, and service_completed_successfully.
• Docker Docs — Compose Watch: prerequisites, sync, rebuild, and sync+restart.
• Docker Docs — docker compose config: normalized model, profiles, services, variables, hashes, and quiet validation.
• Compose services reference: profiles, depends_on, health checks, and service attributes.
• Docker Compose v5.5.1 (released 2026-09-03), current upstream baseline checked for this chapter.