Docker, Containers, Persistent Storage, and Production Deployment: Diagnostics, Failure Modes, and Production Practices

1. Evidence-first diagnostic sequence

1. Freeze change. Preserve docker ps -a, inspect output, image digests, logs, Compose/run configuration, and timestamps.
2. Confirm exact SonarQube/PostgreSQL image identities and architecture.
3. Confirm effective JDBC/server environment without printing sensitive values into shared artifacts.
4. Inspect mounts and volume identities.
5. Inspect Docker network membership, aliases, and DNS assumptions.
6. Inspect DB readiness and DB logs.
7. Inspect SonarQube web/CE/search logs and /api/system/status.
8. Inspect host sysctl/ulimit/resource state if search or resource failures appear.
9. Apply the least destructive correction, then rerun the same smallest scenario.

2. Preserve first-failure evidence before touching the container

mkdir -p evidence

docker ps -a --no-trunc > evidence/docker-ps.txt
docker image inspect sonarqube:26.9.0.129388-community > evidence/sonarqube-image.json
docker inspect sq-lab > evidence/sq-lab-inspect.json 2>&1 || true
docker inspect sq-db > evidence/sq-db-inspect.json 2>&1 || true
docker network inspect sq_net > evidence/sq-net.json 2>&1 || true
docker logs sq-lab > evidence/sq-lab.log 2>&1 || true
docker logs sq-db > evidence/sq-db.log 2>&1 || true

Before sharing evidence, redact passwords, tokens, private hostnames, and sensitive labels. Preserve raw evidence securely if needed; publish only the minimum sanitized subset.

3. Intentionally broken example: localhost points to the wrong process

Assume PostgreSQL is healthy in sq-db on sq_net, but SonarQube was started with:

SONAR_JDBC_URL=jdbc:postgresql://localhost:5432/sonar

Expected evidence: SonarQube cannot connect to PostgreSQL because TCP 5432 inside sq-lab is not the sibling database container. Confirm before fixing:

docker network inspect sq_net
docker exec sq-db pg_isready -U sonar -d sonar
docker logs --tail 160 sq-lab

The least destructive correction is to recreate only sq-lab with jdbc:postgresql://db:5432/sonar. Do not publish PostgreSQL broadly, lower a quality gate, delete volumes, or switch project keys. Those changes are unrelated to the failing layer.

4. Volume and ownership failures

A SonarQube startup can fail because a bind-mounted directory is not writable by the effective container user, because a volume was mounted at the wrong destination, or because an operator deleted a needed named volume. Inspect rather than guessing:

docker inspect sq-lab --format '{{json .Mounts}}'
docker image inspect sonarqube:26.9.0.129388-community --format 'user={{.Config.User}}'
docker volume inspect sq_data sq_extensions sq_logs

Do not solve ownership by running the whole container as root. Align volume semantics and permissions with the supported image behavior.

5. “The project disappeared” is usually a database-identity question

If the SonarQube container recreates successfully but projects/users/settings disappear, first verify the JDBC endpoint and PostgreSQL volume identity. A fresh Postgres container without the original data volume is a fresh durable store. Reusing sonarqube_data cannot reconstruct missing database rows.

6. Embedded-search failures: container logs can point back to the host

Symptoms such as bootstrap-check failures, inability to use /tmp, file-descriptor exhaustion, or search process exits belong to host/search/runtime state. Inspect current Linux limits and resource pressure before changing SonarQube policy:

sysctl vm.max_map_count
sysctl fs.file-max
ulimit -n
ulimit -u
docker stats --no-stream sq-lab
docker logs --tail 200 sq-lab

Do not normalize SONAR_ES_BOOTSTRAP_CHECKS_DISABLE=true as a production shortcut. Fix the owning host/runtime prerequisite.

7. Floating-tag drift: “same Compose file” may not mean same software

If a stack uses sonarqube:latest or sonarqube:community, a later pull can select new image content without changing YAML. Compare RepoDigests, container image IDs, deployment timestamps, and release notes. The correction is a controlled version/digest policy, not deleting logs that expose the change.

8. Secret failures versus secret exposure

An authentication error can come from a wrong DB password, wrong user, wrong JDBC endpoint, or database-side permissions. Diagnose without printing passwords. Conversely, a correct password embedded in a Dockerfile or committed Compose file is an incident even if the container starts. Correctness and confidentiality are separate properties.

9. Destructive cleanup is not troubleshooting

Commands such as docker system prune, docker volume prune, or docker compose down -v can remove evidence and durable state. Preserve logs/inspect output and identify exact resource names first. SonarSource’s container update guidance explicitly warns operators about volume-removal commands because database state can be lost.

10. Symptom-to-layer matrix

11. Production practice: make rollback evidence explicit

A production container change record should include source configuration revision, old/new image tags and digests, supported upgrade path, database backup point, plugin compatibility, host/resource preflight, maintenance window, expected migration behavior, health checks, and rollback/recovery decision criteria. An older image is not automatically a safe rollback after a database migration.

Knowledge check

Official references and version notes

• SonarQube Community Build downloads — current Community Build release identity.
• Official SonarQube Docker image tags — current official image tags and digests.
• Prepare the Docker installation — pre-installation checks and named-volume guidance.
• Set up and start your container — docker run/Compose, JDBC environment variables, ports, and volume warnings.
• Linux pre-installation — embedded-search host limits and writable /tmp requirements.
• Configuration methods — preferred Docker environment-variable configuration model.
• System properties — current JDBC and web-server property names.
• Installing database — supported database versions and H2 non-production boundary.
• Updating Community Build — container recreation and persistent-state upgrade guidance.
• SonarQube official image overview — official image usage, host prerequisites, port and edition tags.